Job Title: Senior Data Engineer
Years of Experience: 5-8 Years
Job Summary: We are seeking a highly skilled Senior Data Engineer with extensive experience in Azure Databricks, Azure Data Factory, and Palantir to join our dynamic team in Hyderabad. The ideal candidate will have a strong background in data engineering, with a proven track record of designing and implementing data solutions that drive business insights and decision-making. This role requires a deep understanding of data architecture, ETL processes, and cloud-based data platforms.
Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure Databricks and Azure Data Factory.
- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions.
- Implement data integration processes to ingest, transform, and store data from various sources into data lakes and warehouses.
- Optimize data workflows and ensure data quality, reliability, and performance.
- Utilize Palantir for data analysis and visualization, providing actionable insights to the business.
- Monitor and troubleshoot data pipeline issues, ensuring timely resolution and minimal downtime.
- Stay updated with the latest industry trends and technologies related to data engineering and cloud computing.
- Mentor junior data engineers and contribute to team knowledge sharing.
Mandatory Skills
- Proven experience with Azure Databricks (deal breaker skill).
- Strong proficiency in Azure Data Factory for data integration and orchestration.
- Experience with Palantir for data analysis and visualization.
- Solid understanding of data modeling, ETL processes, and data warehousing concepts.
- Proficient in programming languages such as Python, SQL, or Scala.
Preferred Skills
- Experience with cloud platforms such as Azure, AWS, or Google Cloud.
- Familiarity with big data technologies like Apache Spark, Hadoop, or Kafka.
- Knowledge of data governance and data security best practices.
- Experience with CI/CD pipelines and DevOps practices in data engineering.
Qualifications
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- 5-8 years of experience in data engineering or a related role.
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration abilities.
If you are passionate about data engineering and want to work in a fast-paced environment, we encourage you to apply for this exciting opportunity.
Azure Databricks, Python, Azure Data Factory, PySpark
