Python - Senior Engineer

Job Description

Key Responsibilities

. Design, develop, and maintain Python-based data processing pipelines for on-prem and file-based systems.

. Work with large structured and unstructured datasets for data ingestion, transformation, and validation.

. Implement and optimize ETL/ELT processes, ensuring data quality, consistency, and performance.

. Collaborate with cross-functional teams to design end-to-end data solutions, including integration with cloud and on-premise environments.

. Leverage AWS services (such as S3, Lambda, Glue, Step Functions, or EC2) for data processing and automation.

. Participate in solution design discussions and contribute to architectural decisions.

. Monitor, troubleshoot, and improve existing data workflows for efficiency and scalability.

. Ensure compliance with data governance, security, and performance standards.

________________________________________

Required Skills & Qualifications

. Bachelor's degree in Computer Science, Information Technology, or related field.

. 5+ years of hands-on experience with Python for data processing and automation.

. Proven experience with on-premise data workflows and file-based data processing (CSV, JSON, XML, etc.).

. Strong understanding of ETL/ELT design principles and data integration tools.

. Basic to intermediate knowledge of AWS cloud data processing services (e.g., S3, Glue, Lambda, Redshift, or EMR).

. Experience in solution design and data architecture planning.

. Familiarity with SQL and relational databases (MySQL, PostgreSQL, etc.).

. Excellent problem-solving, debugging, and analytical skills.

________________________________________

Good to Have

. Experience with workflow orchestration tools (e.g., Airflow, Prefect, Step Functions).

. Exposure to DevOps practices, CI/CD pipelines, or infrastructure automation.

. Understanding of data security and compliance best practices.

Mandatory Competencies