Data Engineer

People Prime Worldwide

Chennai

5-10 Years

Save

Posted 6 days ago
Be among the first 30 applicants

Early Applicant

Quick Apply

Job Description

We are seeking a highly skilled Data Engineer with strong hands-on experience in Python, PySpark, Apache Spark, Kafka, and Databricks to design, build, and maintain scalable and high-performance data pipelines. The role focuses on developing robust batch and streaming data solutions for large-scale distributed data environments.

Key Responsibilities

Design, develop, and maintain batch and real-time data pipelines using PySpark and Apache Spark
Build and manage scalable data workflows and jobs on Databricks
Develop and optimize Python and SQL code to process large and complex datasets
Implement Kafka-based streaming solutions for real-time data ingestion and processing
Perform Spark job tuning, performance optimization, and resource management
Ensure data reliability, scalability, and efficiency across pipelines
Work closely with analytics, product, and engineering teams to support data requirements
Monitor, troubleshoot, and resolve data pipeline and performance issues
Maintain documentation for data pipelines, workflows, and processing logic

Required Experience & Qualifications

510 years of experience in Data Engineering roles
Strong hands-on coding experience in Python and PySpark
Extensive experience with Apache Spark and Databricks platforms
Hands-on experience with Kafka for streaming data processing
Strong SQL skills for data transformation and validation
Experience working with large-scale, distributed data systems

Preferred Skills

Experience with Delta Lake for data versioning and reliability
Exposure to cloud platforms such as AWS, Azure, or GCP
Experience with workflow orchestration tools such as Airflow

More Info

Job Type:

Permanent Job

Industry:

Consulting, Information Services

Role:

Software Engineer /Programmer

Function:

Employment Type:

Full time

Open to candidates from:

Indian

About Company

People Prime Worldwide

At People Prime Worldwide, we deliver innovative IT solutions that drive business success. Our comprehensive suite of services include IT Consulting & Implementation, Staff Augmentation (T&M), and Managed Teams & Managed Services and is designed to help organisations optimise their workforce, streamline operations, and enhance technological capabilities. With a focus on agility, efficiency, and cost-effectiveness, we provide expert consulting, skilled talent, and end-to-end IT management to ensure seamless project execution. Whether you need cutting-edge IT solutions, flexible workforce expansion, or fully managed IT services, we are your trusted partner in navigating the digital landscape. IT Consulting and Implementation Services - Transforming Businesses with Scalable IT Solutions: We provide expert IT Consulting and Implementation Services, enabling businesses to optimize their technology infrastructure and streamline operations. Our specialized services cover SAP, Salesforce, ServiceNow, and Cloud Migration, helping enterprises modernize their digital ecosystem. Staff Augmentation (T&M) - IT Resources As A Service: We offer Time & Materials (T&M) Staff Augmentation, ensuring businesses can access the right talent at the right time. Whether it's short-term project support or long-term IT staffing, we provide skilled professionals to meet your requirements. Managed Teams and Managed Services - End-to-End IT Operations Without the Hassle: We provide Managed Teams and Managed Services, allowing businesses to focus on their core objectives while we handle IT operations efficiently.

Job ID: 137573189

Jobs by Skill - IT