We are looking for a skilled Big Data Engineer to support the onsite Lilly team in delivering high-quality data solutions. The ideal candidate will have strong experience in big data technologies, cloud platforms, and CI/CD practices.
Key Responsibilities:
- Collaborate closely with the on-ground Lilly team to complete project deliverables on time
- Design, develop, and maintain big data pipelines and workflows using Python and PySpark
- Leverage cloud platforms (AWS, Azure, or GCP) to deploy and manage big data solutions
- Implement Continuous Integration and Continuous Deployment (CI/CD) pipelines for automated deployments
- Troubleshoot and optimize data processing jobs for performance and reliability
- Participate in code reviews, testing, and documentation
Required Skills:
- Strong programming skills in Python and PySpark
- Experience with at least one major cloud platform: AWS, Azure, or GCP
- Knowledge of CI/CD tools and best practices
- Experience in supporting and delivering big data projects in a collaborative team environment
- Good communication and problem-solving skills
