Senior Data Engineer

Job Role: Senior Data Engineer

Experience Level: 3.5 to 10 years

Work location: Bangalore

Joining timeline: 0 to 45 days

Job Description:

Must have skills:

3+ years of hands-on experience in building large-scale data pipelines.

Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads

Proficiency in Pyspark/Python, including building Kafka producers for data ingestion

Experience working with Confluent Kafka/spark streaming for streaming ingestion from on-premise sources.

Solid understanding of AWS services including S3, Redshift, Glue, CloudWatch, and Secrets Manager

Experience working with Medallion architecture and hybrid data destinations (e.g., Redshift and on-premise Oracle)

Ability to implement monitoring dashboards and observability for pipelines using tools like CloudWatch or Datadog

Strong SQL skills for validating data accuracy and developing job-level metrics

Experience with building alerting mechanisms for pipeline failures and performance issues

Strong collaboration and communication skills

Proven ability to take ownership of technical deliverables, driving execution from design to deployment

Experience in guiding junior developers or data engineers, providing technical direction and code reviews to uphold best practices

AWS Data Engineering - Associate Certified.

Good to have skills :

Experience with orchestration tools like Apache Airflow or AWS Step Functions

Exposure to Bigdata stacks like Sqoop,HDFS,Hive,Nifi

Exposure to Terraform for infrastructure automation

Familiarity with CI/CD pipelines for data workflows