Job Title: Data Engineer
Location: Hyderabad
Experience: 5+ Years
Job Overview
We are looking for a skilled Data Engineer with strong experience in Hadoop, SQL, Python, and Kafka, along with mandatory expertise in AWS. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines and data infrastructure to support business analytics and data-driven decision-making.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL processes.
- Work with Hadoop ecosystem tools to process large-scale structured and unstructured data.
- Develop data solutions using Python and SQL for data transformation and analysis.
- Implement and manage real-time data streaming using Kafka.
- Build and maintain data infrastructure on AWS services.
- Optimize data workflows and improve system performance and reliability.
- Collaborate with data scientists, analysts, and engineering teams to support data requirements.
- Ensure data quality, security, and governance standards are maintained.
Required Skills
- Strong experience with Hadoop ecosystem.
- Proficiency in SQL and Python for data processing and automation.
- Hands-on experience with Apache Kafka for real-time data streaming.
- Working knowledge of Scala (basic level).
- Mandatory experience with AWS services such as S3, EMR, Glue, or similar.
- Experience in building ETL/ELT pipelines and data integration solutions.
- Strong understanding of data warehousing concepts and big data processing.
Preferred Skills
- Experience with GCP (Google Cloud Platform) is a plus.
- Knowledge of data lake architecture and distributed systems.
- Experience with workflow orchestration tools like Airflow is an advantage.
Education
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
