Sr. Python Developer (5+ years, Pyspark, Ahmedabad)

Key Responsibilities:

Azure Cloud s Databricks:

oDesign and build efficient data pipelines using Azure Databricks (PySpark).

oImplement business logic for data transformation and enrichment at scale.

oManage and optimize Delta Lake storage solutions.

API Development:

oDevelop REST APIs using FastAPI to expose processed data.

oDeploy APIs on Azure Functions for scalable and serverless data access.

Data Orchestration s ETL:

oDevelop and manage Airflow DAGs to orchestrate ETL processes.

oIngest and process data from various internal and external sources on a scheduled basis.

Database Management:

oHandle data storage and access using PostgreSQL and MongoDB.

oWrite optimized SQL queries to support downstream applications and analytics.

Collaboration:

oWork cross-functionally with teams to deliver reliable, high-performance data solutions.

oFollow best practices in code quality, version control, and documentation.

Required Skills s Experience:

5+ years of hands-on experience as a Data Engineer.

Strong experience with Azure Cloud services.

Proficient in Azure Databricks, PySpark, and Delta Lake.

Solid experience with Python and FastAPI for API development.

Experience with Azure Functions for serverless API deployments.

Skilled in managing ETL pipelines using Apache Airflow.

Hands-on experience with PostgreSQL and MongoDB.

Strong SQL skills and experience handling large datasets.