Job Description
Equired Skills & Qualifications
812 years of hands-on software development and data engineering experience with a strong focus on coding and building systems from scratch.Expert-level proficiency in Python for data engineering, scripting, automation, and API development.
Deep hands-on experience with PySpark/Spark, including writing transformations, aggregations, and joins, working with DataFrames and Spark SQL.Strong SQL development skills with cloud-native relational databases such as Snowflake, Athena, and PostgreSQL.Strong understanding of APIs and experience with building API on AWS. Design and implement APIs from scratch to accelerate the integration.Strong AWS cloud expertise, including:Core data services: S3, Glue (Catalog, ETL, Crawlers), EMR, Athena, Redshift, Lake FormationCompute & orchestration: Lambda, Step Functions,
CI/CD, IAM, Terraform, CloudWatch, and operational best practicesSolid understanding of lakehouse architectures and table formats (Iceberg, Delta Lake, Hudi), metadata management (Glue Catalog, Hive Metastore), and data governance patterns.
Experience designing and implementing orchestration workflows (Step Functions, Airflow, or similar); ability to articulate state machine design, error handling, retries, and parallel execution patterns.Hands-on experience with CI/CD pipelines, version control (Git), and infrastructure-as-code (Terraform).Problem-Solving & AdaptabilityDemonstrated ability to reason through complex technical problems without relying solely on documentation or prepared notes.
Comfortable working in ambiguous, fast-paced environments where requirements evolve rapidly.
Strong debugging, troubleshooting, and root-cause analysis skills across distributed systems.
Excellent verbal and written communication skills; ability to clearly explain technical concepts, trade-offs, and design decisions to both technical and non-technical stakeholders.
