We are seeking a skilled Big Data Engineer to join our team in India. The ideal candidate will have extensive experience in designing and implementing robust big data solutions that support our data-driven decision-making processes.

Job description

Design, build, and maintain scalable ETL/ELT pipelines using AWS Glue, Lambda, Step Functions, and Apache Spark.
Develop and optimize data lake and data warehouse architectures using AWS S3, Redshift, and Athena.
Implement data ingestion, transformation, and storage solutions using AWS Glue, Kinesis, and Kafka.
Work with structured and unstructured data to develop efficient data pipelines.
Optimize query performance in SQL-based and NoSQL databases (Redshift, DynamoDB, Aurora, PostgreSQL, etc.).
Ensure data security, compliance, and governance using IAM roles, KMS encryption, and AWS Lake Formation.
Automate data workflows using Terraform, CloudFormation, or CDK.
Monitor and troubleshoot data pipelines, ensuring high availability and performance.
Collaborate with Data Scientists, Analysts, and Engineers to provide robust data solutions.

Required Skills:

Strong experience with AWS Data Services (Glue, Redshift, S3, Lambda, Kinesis, Athena).
Proficiency in Python, SQL, and Scala
Experience with ETL/ELT pipeline design and data warehousing concepts.
Strong knowledge of relational and NoSQL databases (PostgreSQL, MySQL, DynamoDB).
Familiarity with DevOps practices and CI/CD tools (Jenkins, GitHub Actions).
Experience with Kafka or Kinesis for real-time data streaming.
Knowledge of Terraform, CloudFormation, or CDK for infrastructure automation.
Hands-on experience with data security and governance.
Strong analytical and problem-solving skills