Responsibilities
- Develop, optimize, and maintain scalable data pipelines using Databricks (PySpark/SQL).
- Collaborate with data scientists to build, train, and deploy machine learning models for use cases such as recommendation engines and time series forecasting.
- Work with large data sets to prepare and process structured and unstructured data for analytics and ML.
- Implement and maintain CI/CD pipelines for model deployment and versioning.
- Collaborate with cross-functional teams to gather requirements and deliver data-driven solutions.
- Optimize query performance and manage data storage efficiently on the Lakehouse architecture.
Qualifications
Preferably BE/BTech/BIT/MCA/BCA
8-12 years of experience
