Senior Data Engineer Job Profile
About the Job
Are you passionate about working with massive datasets and eager to leverage the latest technologies This opportunity could be the perfect fit for you.
Company is seeking a highly skilled Senior Data Engineer or Datawarehouse developer to join our Enterprise Data Warehousing team. We are actively building a new data warehouse solution aimed at expanding our analytical capabilities, addressing increasing data volumes, and supporting the growing need for robust reporting.
The role is based within the Platform and Frameworks team at our Gurugram office. This team is responsible for building and maintaining our cloud-based data warehouse platform, as well as developing essential frameworks that empower our data onboarding teams. The goal is to enable efficient delivery of analytics solutions by applying the Data Vault 2.0 methodology.
Responsibilities
- Play a critical role in supporting the teams primary objective of establishing a robust and scalable data warehouse platform.
- Design, author, and maintain data pipelines using Airflow.
- Develop .NET services to consume data from Kafka and persist data to Azure Data Lake Storage. Build and operate .NET services for publishing data to Kafka.
- Ensure reliability, performance, and security across data pipelines and services.
- Learn and explore new technologies to support new feature requirements on the platform.
Your Profile
To be successful in your role, we expect that you
- Have 5+ years of experience as a Data Engineer or Data Warehouse Developer.
- Strong proficiency in Python for data processing (e.g., Pandas, PySpark when applicable).
- Strong proficiency in .NET (C#, .NET 6+/.NET Core) for services, APIs, and data pipeline components.
- Working knowledge of any relational databases (Microsoft SQL, Oracle, PostgreSQL, Azure SQL or snowflake etc.)
- Always design and build solutions with platform stability, resiliency, and security in mind.
- You work well within a team and communicate efficiently with colleagues and stakeholders.
Nice to Have Skills
- Working experience with a cloud platform (preferably Azure)
- Knowledge of deploying and managing Airflow in a containerized environment.
- Knowledge of Data Vault modelling.
- Previous experience working in the financial industry.
