As a Senior Data Engineer at Medtronic's Global Diabetes Capability Center in Pune, you'll be a pivotal member of our Digital Technology team, dedicated to transforming diabetes management through innovative data solutions. You'll primarily focus on designing, developing, and maintaining scalable and robust data pipelines, ideally utilizing PySpark, to deliver cost-effective solutions for evolving business needs. Your expertise in Big Data technologies and cloud platforms will directly contribute to expanding our global reach and improving the lives of people with diabetes.
A Day in the Life
You will drive data engineering excellence and innovation by:
- Translating conceptual needs and business requirements into finalized architectural designs for data solutions.
- Managing large data projects or processes that span across collaborative teams both within and beyond Digital Technology.
- Operating autonomously to define, describe, diagram, and document the role and interaction of high-level technological and human components that provide cost-effective and innovative data solutions.
- Promoting, guiding, and governing good architectural practice through well-defined, proven technology and architecture mentorship.
- Designing, developing, and maintaining scalable data pipelines, with a strong preference for PySpark.
- Working with structured and unstructured data from various sources to build comprehensive data solutions.
- Optimizing and tuning PySpark applications for peak performance and scalability.
- Supporting the full lifecycle management of IT portfolio components related to data, including selection, appropriate usage, enhancement, and replacement of applications and infrastructure.
- Implementing robust data quality checks and ensuring the integrity of data across all pipelines.
- Monitoring and troubleshooting data pipeline issues, ensuring timely resolution to maintain operational stability.
- Documenting technical specifications and maintaining comprehensive documentation for all data pipelines.
- Staying abreast of the fast-paced world of Big Data technology and building ETL/ELT solutions using new and emerging technologies while maintaining platform stability.
Required Knowledge and Experience
- 4-8 years of experience in data engineering, with a significant focus on PySpark.
- Strong programming knowledge in Python and Spark, with excellent coding and debugging skills.
- Proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL, SQL Server).
- Experience in designing and building Enterprise Data solutions on AWS Cloud, Azure, or Google Cloud Platform (GCP).
- Experience with big data technologies such as Hadoop, Hive, and Kafka.
- Experience with data warehousing solutions like Redshift, Snowflake, Databricks, or Google Big Query.
- Familiarity with data lake architectures and various data storage solutions.
- Knowledge of CI/CD pipelines and version control systems (e.g., Git).
- Excellent problem-solving skills and the ability to troubleshoot complex issues.
- Strong communication and collaboration skills, with the ability to work effectively in a team environment.
Benefits & Compensation
This position is eligible for a short-term incentive called the Medtronic Incentive Plan (MIP).
