Senior Data Engineer

Job Requirements

• Support the design, implementation, and maintenance of enterprise ETL processes for data platforms, for a global client base.
• Develop scalable and efficient code to process data, ensuring availability and accessibility in a timely manner.
• Leverage big data processing frameworks such as Apache Spark and Hadoop to build and optimize data pipelines.
• Collaborate with senior engineers to address data challenges, contributing to solutions that maintain high data quality.
• Assist in the data delivery process, working alongside Data Engineers and Analysts to support accurate, high-value data solutions across various clients and industries.
• Build strong working relationships with team members and clients, contributing to both local and global projects.
• Learn and apply industry best practices, including version control, code reviews, and data validation, to ensure quality in data processes.
• Use SQL and other database technologies to help optimize data processing and reduce the time required to handle large data sets.
• Design, implement, and maintain data pipelines using ETL frameworks, orchestration tools, and distributed data processing engines.
• Participate in efforts to automate routine data tasks and streamline processes.
  
  

Work Experience

Years of Experience: 4-6 Years

Strong hands-on expertise in Spark, Airflow, and SQL.

Deep understanding and experience with modern Data Platforms – Cloudera Data Platform (CDP)

• Strong hands-on experience with PySpark.
• Experience with Cloudera Data Platform (CDE, CDW, Ozone, Airflow, SDX), Apache Ranger
• Deep understanding of distributed data systems and Hive Metastore
• Experience and understanding of cataloging, lineage, and governance
• Experience / understanding Open Data Contract Standard (ODCS) and its implementation
• Experience working with SQL, file formats (Iceberg/Parquet), and partitioning/bucketing strategies.
  
  

Nice to have

• Experience modernizing enterprise finance systems or regulated environments
• Knowledge of CI/CD, data engineering best practices
• Understanding of financial data structures, accounting processes, or reconciliation workflows.