Senior Data Engineer

As a Senior Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems and infrastructure. You will contribute to multiple projects from both operational and technical standpoints, drive architectural discussions, and guide teams on data engineering best practices. This role requires strong knowledge of data architecture, cloud platforms, data modeling, ETL/ELT, and modern data processing patterns across batch and real-time workflows. 

• Design, store, process, and maintain large-scale data systems and related infrastructure.
• Drive multiple projects from both operational and technical standpoints.
• Ideate and build PoVs/PoCs for new products and solutions that help drive business growth.
• Define, design, and implement data engineering best practices, strategies, and solutions.
• Act as a data architect guiding customers, teams, and the organization on tools, technologies, and best practices.
• Lead architecture discussions and align solutions with business needs, security requirements, and engineering best practices.
• Execute multiple end-to-end data engineering projects as a data architect.
• Demonstrate strong conceptual understanding of data warehousing, ETL, data governance, security, cloud computing, and batch & real-time data processing.
• Apply strong execution knowledge of data modeling, SQL/NoSQL databases, SDLC practices, unit testing, and functional programming.
• Understand and work with Medallion architecture patterns.
• Manage conversations with client stakeholders, understand requirements, and translate them into technical outcomes.

Required Skills and Experience  

• 4-6 years of relevant work experience.
• Strong proficiency in SQL and experience with query and code optimization.
• Experience working with at least one major cloud platform (AWS, Azure, or GCP).
• Working knowledge of ETL and orchestration tools such as IICS, Talend, Matillion, Airflow, ADF, AWS Glue, or GCP Composer.
• Working knowledge of OLTP databases (Postgres, MySQL, SQL Server, etc.).
• Working knowledge of data warehouses such as Snowflake, Redshift, Azure Synapse, Hive, or BigQuery.
• Proficiency in at least one data engineering programming language such as Python (or Scala / Java / Rust).
• Strong execution knowledge of data modeling concepts including star schema, snowflake schema, and fact/dimension tables.
• Proficiency in Spark and distributed data platforms such as Databricks, EMR, AWS Glue, or GCP Dataproc.
• Experience working with Kafka and real-time streaming systems.
• Strong understanding of data architecture patterns such as Lambda, Kappa, data harmonization, and CDP patterns.
• Strong knowledge of Git and CI/CD pipeline design for continuous delivery.
• Experience with data and networking security including RBAC, secret management, key vaults, certificates, and VNet/subnet configurations.