S
Data Engineer
S
- Posted 19 days ago
- Over 100 applicants have applied
Job Description
Job Description:
We are looking for an experienced Data Engineer to work on a contract basis, supporting data migration and engineering initiatives on the AWS platform.
Key Requirements:
- 6–12 years of overall experience, with strong hands-on expertise in AWS services including S3, Lambda, PySpark, Airflow, Glue, Athena, Lake Formation, Step Functions, Kinesis, and Streaming Pipelines
- Strong programming experience in Python, along with Functional Testing (FT) and Unit Testing (UT)
- Experience with CI/CD concepts, including Jenkins and SQL DB scripting
- Proven experience delivering 2–3 data migration projects on AWS platforms (data engineering focus, not data science)
Good to Have:
- Experience with Big Data technologies such as Teradata, Snowflake, Spark, Redshift, Kafka, etc.
- Experience with data management processes on AWS
- Hands-on experience implementing complex ETL transformations using AWS Glue
- Familiarity with relational database environments (Oracle, Teradata, etc.), including databases, tables/views, stored procedures, and agent jobs
More Info
Key Skills
About Company
Skills:
Power Bi, Apache Spark, Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Powershell Scripting, Python, Azure DevOps, Lakehouse Architecture, SQL Performance Optimization, Azure SQL Database, Azure Data Lake Storage, Delta Lake, Microsoft Entra ID Azure AD
Skills:
Apache Airflow, Aws Redshift, Pyspark, Python, Sql, AWS, AWS EMR
Skills:
Apache Spark, Kafka, Apache Airflow, Pandas, Elasticsearch, MongoDB, pgvector, Polars, Qdrant, Vector databases, NoSQL databases, Data models and storage architectures, Distributed data systems, Data processing performance and optimisation, Data pipelines, Milvus
Skills:
Gcp, Sql Server My Sql, Data Engineer
Skills:
Azure Functions, Adf, Pyspark, Databricks, Azure, Sql, Synapse, Azure Purview, Delta Lake, Unity Catalog