Senior Software Engineer

Experience: 5-7 Years

Location:Chennai

Job Overview

As a PySpark Data Engineer, you will be responsible for designing, developing, and supporting scalable, reliable, and high-performance data solutions. You will work closely with business and technology teams to build robust data pipelines, develop analytical data models, and deliver data-driven solutions that support strategic business objectives.

Key Responsibilities

• Design, develop, and maintain scalable, extensible, and highly available data solutions.
• Build and optimize large-scale data processing applications using PySpark and SQL.
• Deliver critical business requirements while ensuring alignment with enterprise architecture and data strategy.
• Design and develop analytical data models to support reporting, analytics, and business intelligence initiatives.
• Develop, automate, test, and deploy high-performance data pipelines.
• Analyze, transform, and manipulate large datasets using PySpark and SQL.
• Identify and mitigate potential risks across the data supply chain.
• Collaborate with business stakeholders, architects, and engineering teams to deliver scalable data solutions.
• Troubleshoot and resolve complex data, performance, and application-related issues.
• Mentor team members and contribute to technical best practices and engineering standards.

Required Skills & Experience

• 5-7 years of experience in Data Engineering.
• Strong hands-on experience with PySpark and Spark for large-scale data processing.
• Advanced proficiency in SQL for data querying, transformation, optimization, and analysis.
• Strong understanding of relational databases and data warehousing concepts.
• Experience in designing and implementing analytical and dimensional data models.
• Hands-on experience in building, automating, and monitoring ETL/ELT processes and data pipelines.
• Experience working with cloud platforms such as AWS, Azure, or GCP.
• Strong understanding of cloud-native technologies and modern data engineering patterns.
• Excellent analytical, troubleshooting, and problem-solving skills.
• Strong communication skills with the ability to collaborate effectively across teams.

AI & Innovation Focus

• Leverage AI-powered tools such as GitHub Copilot to enhance development productivity and accelerate delivery.
• Stay informed about emerging AI technologies and recommend innovative approaches to improve products and solutions.
• Utilize AI-assisted engineering practices to improve software quality and operational efficiency.

Preferred Qualifications

• Experience with big data ecosystems and distributed data processing frameworks.
• Strong ownership mindset with the ability to independently drive project deliverables.
• Exposure to Agile development methodologies.
• Passion for continuous learning and adopting new technologies.