Build and optimize scalable
ETL/ELT pipelines integrating Oracle EBS/Fusion, enterprise applications,
supply chain systems, and cloud analytics platforms using Databricks, PySpark,
ADF, SQL, and BigQuery. Support ERP modernization and enterprise reporting
initiatives.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark and Databricks.
- Build and optimize ETL/ELT workflows for processing large volumes of structured and unstructured data.
- Develop and manage data integration solutions using Azure Data Factory (ADF).
- Collaborate with data analysts, business stakeholders, and cross-functional teams to understand data requirements.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Ensure data quality, governance, and security across data platforms.
- Troubleshoot and resolve data pipeline issues while ensuring high availability.
Required Skills
- Strong hands-on experience with PySpark.
- Experience working with Databricks for data engineering and analytics.
- Hands-on experience with Azure Data Factory (ADF).
- Strong knowledge of SQL and data warehousing concepts.
- Good understanding of Azure cloud services and ETL processes.
- Experience with performance tuning and optimization of Spark jobs.
- Excellent analytical, problem-solving, and communication skills.
Good to Have
- Experience with Power BI for reporting and dashboard development.
- Knowledge of Azure Data Lake, Delta Lake, or Synapse Analytics.
- Familiarity with CI/CD pipelines and version control tools such as Git.
