Search by job, company or skills

Azure Data Engineer

3-5 Years
Quick Apply
  • Posted 23 days ago
  • Over 400 applicants have applied

Job Description

Responsibilities:

  • Design, develop, and maintain scalable and efficient data pipelines using PySpark, Scala Spark, Databricks, Python, and SQL.
  • Write optimized, reusable, and high-quality code for data processing and transformation.
  • Optimize SQL queries for high-performance data extraction, manipulation, and analysis.
  • Demonstrate strong expertise in Databricks, including workflow management, job orchestration, and data exploration.
  • Collaborate with cross-functional teams to gather and understand business and data requirements.
  • Implement best practices for ETL, data pipeline optimization, and query performance tuning.
  • Develop and maintain comprehensive documentation for all data pipelines, workflows, and related processes.
  • Troubleshoot, debug, and resolve data pipeline issues promptly to ensure data accuracy and minimal downtime.
  • Continuously explore opportunities for automation, performance improvements, and scalability enhancements in data workflows.

Requirements:

  • Strong programming skills in Python, PySpark, and SQL.
  • Experience with Databricks and Spark-based data processing frameworks.
  • Good understanding of ETL design principles, data modeling, and data architecture.
  • Hands-on experience with workflow orchestration tools and version control systems (e.g., Git).
  • Familiarity with cloud-based data platforms (AWS, Azure, or GCP) is an added advantage.

More Info

Job Type:
Function:
Employment Type:

About Company

Colan Infotech is among the 50 fastest growing companies” according to Silicon India and “The 50 best companies to work for” rated by Silicon Review. At Colan Infotech, you would walk across tech-minded people who conquer the frontiers of technology for a successful business. For over a decade, since Colan Infotech was started in 2009, we are in the business of digitizing firms and keep businesses relevant in today’s age. The company is headquartered in Chennai with 500+ employees, always prepared to meet the technology needs of any kind of business anywhere in the world. Our company holds the recognition of CMMI level 3 and being a NASSCOM partner. With a wide range of technology expertise under our belt, starting from mobile, web technologies, AI all the way to Blockchain solutions – we are an indispensable technology partner serving global clientele. Our engineers are experts in developing mobile solutions, functional web applications, intelligent AI software and Blockchain solutions. Our technology services are in par excellence to digitally transform your business, mobile-ready, powered by technology to run your business efficiently and make profits. Clients enjoy the best of all worlds through working with an offshore technology partner supplying best talent, great quality and cost-effective measures. We understand the complexity of technology projects and can provide customized solutions pertaining to your business. Following agile methodology that proves efficiency, timely delivery and quality assurance to live upto world standards.

Job ID: 130607681

Similar Jobs

Chennai, Mumbai, Coimbatore

Skills:

Azure SqlAzure Data LakeAzure DatabricksAzure Data Factory

Chennai, India

Skills:

T-sqlAzure Data BricksPerformance TuningAzure Synapse AnalyticsAzure Data LakeSpark SQLSqlAzure Data FactoryAzure Machine LearningDaxAzure Key VaultManaged IdentitySecurityAzure Analysis ServiceEvent HubsADLSGEN2Azure SQL DWHdeployment Automationtabular modelsAzure BlobAzure Data StackAzure Stream AnalyticsAzure environment PaaS IaaSrbacSizing

Chennai, India

Skills:

Azure Data FactorySqlDatabricksPythonAzure SynapsePysparkAzure Services

Chennai, India

Skills:

Azure DatabricksSpark SQLAzure Data FactoryPythonGitPysparkAzure Data Lake Storage Gen2Delta Lake

Bengaluru, Chennai, Hyderabad

Skills:

AzureSqlPythonPysparkdata bricks

Beware of Scammers

We don’t charge money for job offers