Business Analyst, Data Engineer

Inviting applications for the role of Business Analyst, Data Engineer!

The Data Engineer will be responsible for designing, developing and maintaining robust data pipelines and ETL processes using Amazon Web Services (AWS) tools. This role requires expertise in SQL, Python and AWS technologies, along with experience in integrating APIs to enhance data accessibility and usability.

Responsibilities:

• Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs.

• Design and implement scalable ETL processes using AWS tools such as AWS Glue, Lambda, StepFunctions etc.

• Design Data Transformation pipelines leveraging Snowflake warehouse as a source or as a Target.

• Integrate external APIs to enrich data sets.

• Optimize SQL queries for performance and efficiency in large-scale data environments.

• Monitor and troubleshoot data pipeline issues to ensure seamless operation.

Minimum Qualification/Skills:

• Good experience in data engineering role.

• Good years of experience working with AWS based ETL and flow management tools.

• Proficiency in SQL and Python for data manipulation and analysis.

• Extensive experience with AWS tools for ETL processes, including AWS Glue, Lambda, Step Functions.

• Very good understanding of GIT and CI/CD methodologies.

• Working knowledge of REST and SOPA APIs both for development and data enrichment.

• Experience with deploying containerized applications (Docker/Kubernetes).

• Good understanding of data architecture and warehousing concepts.

Preferred Skills:

• Any Experience or knowledge of Snowflake and Snowpark.

• Any Experience or knowledge of Linux.

• Any Experience of PYSPARK