Senior Data Engineer

About the Company

KPI Partners is seeking a highly skilled Databricks engineer with a strong background in data engineering and analytics. The successful candidate will be responsible for designing, implementing, and maintaining data solutions in the AWS cloud environment. The ideal candidate will have a minimum of 6 years of experience in data engineering, with expertise in SQL, Python, Airflow, Spark, and PySpark.

Who can join immediately

About the Role

The successful candidate will be responsible for designing, implementing, and maintaining data solutions in the Azure cloud environment.

Responsibilities

• Design and develop scalable data pipelines and infrastructure on AWS
• Optimize data storage and retrieval processes to ensure high performance and efficiency.
• Collaborate with data scientists, analysts, and business stakeholders to gather requirements and provide effective data solutions.
• Implement data integration and transformation processes using Databricks data services.
• Monitor and troubleshoot data systems to ensure accuracy and reliability.
• Stay updated with industry best practices and emerging technologies in data engineering and Azure services.

Qualifications

• Btech /Mtech or Any graduate

Required Skills

• Proficiency in SQL for data manipulation and querying.
• Strong programming skills in Python, with experience in data processing libraries.
• Familiarity with AWS cloud services, including Databricks, Airflow
• Experience with big data technologies, specifically Spark and PySpark, for data processing and analysis.
• Excellent problem-solving skills and ability to work collaboratively in a team environment.