Search by job, company or skills

Lead Data Engineer

Lead Data Engineer

Randomtrees
8-13 Years
Not Disclosed
Quick Apply
  • Posted a month ago
  • Over 100 applicants have applied

Job Description

Key Responsibilities:

Data Pipeline Development & Optimization

  • Design, develop, and optimize data pipelines using Azure and Databricks.
  • Build and manage scalable ELT/ETL workflows with Databricks, Delta Lake, ADF, and other Azure services.
  • Optimize Spark, SQL, and Python code for performance and scalability.

CI/CD & Environment Management

  • Hands-on work with Databricks Asset Bundles (DABs) for CI/CD and environment management.
  • Continuously adopt and apply new Databricks features to accelerate delivery.
  • Apply software engineering best practices (TDD, CI/CD, modular design, code reviews) to ensure robust and maintainable pipelines.

Team Mentorship & Collaboration

  • Mentor team members to improve technical capability and overall productivity.
  • Deliver high-quality solutions in a fast-paced environment with quick turnaround expectations.

More Info

Job Type:
Industry:
Function:
Employment Type:

Key Skills

About Company

RandomTrees is a global provider of Data and AI services. Led by distinguished Industry experts with a passion to innovate with data. We help reinvent your enterprise with cutting edge Generative AI backed by our niche expertise to handle any type of data. We bring a unique set of data & and AI accelerators that help jumpstart & accelerate AI journey in the enterprise. Our differentiators are backed by the leading enterprise cloud partners and clients span across life sciences, Automotive and other manufacturing Industries Our portfolio comprises of: (1) Cloud data engineering services & Accelerators (2) Gen AI Products, Accelerators & services.

Similar Jobs

12-14 yrs
Hyderabad, India
Skills:
Python, Apache Flink, Scala, Cosmos, Sql, Devops, Apache Kafka, Databricks, MongoDB, Azure, Data integration tools, ETL processes, DataOps, Data security patterns
10-12 yrs
Hyderabad, India
Skills:
Github, Data Factory, Power Bi, Pyspark, Data Warehouse, Sql, ELT, Python, Azure DevOps, Etl, Data Governance, OneLake, Real-Time Intelligence, Lakehouse, Microsoft Fabric, Spark optimization
10-12 yrs
Hyderabad, India
Skills:
Data Factory (ADF), Devops, Microsoft Tfs, T-sql, Pyspark, Data Lake, Visual Studio, Azure Databricks, SSIS, Scala, Spark, Azure Sql, Delta Lake, Unity Catalog, ADLS Gen2, Azure cloud tools
10-12 yrs
Hyderabad, India
Skills:
snowflake , Git, Terraform, Openflow, PostgreSQL, Python, Sql, AWS, Apache Nifi, dbt
7-10 yrs
Hyderabad, India
Skills:
Data Factory (ADF), T-sql, Microsoft Tfs, Pyspark, Scala, Visual Studio, Azure Databricks, SSIS, Azure Sql, Devops, Spark, Data Lake, Advanced Python Programming, ADLS Gen2, Unity Catalog, Delta Lake