Nextpathway | The Automated Cloud Migration Company
Next Pathway is a technology services company providing clients a pathway from existing to emerging technologies. Our automation technology helps our customers accelerate the migration of complex applications and workloads to the cloud.
About the Role:
Next Pathway is looking for a highly skilled and experienced Senior Engineer to join our team, specializing in large-scale cloud migration projects. The ideal candidate will have a minimum of 10 years of experience in data engineering and ETL, with a strong background in migrating on-premise data systems to the cloud. This role requires a hands-on professional who can design, develop, and implement complex data pipelines, working independently and as a key contributor within a project team.
Key Responsibilities:
- Development and Implementation: Design, develop, and deploy robust and scalable data pipelines for migrating data from various on-premise sources to cloud platforms.
- Technical Expertise: Serve as a subject matter expert in the project's core technologies, including Datastage, Azure Data Factory (ADF), Databricks, and PySpark.
- Solution Design: Collaborate with architects and technical leads to design efficient and effective cloud migration strategies and data transformation solutions.
- Code Quality: Ensure the delivery of high-quality, well-documented, and maintainable code. Participate in code reviews and provide constructive feedback to peers.
- Performance Optimization: Analyze and optimize the performance of data pipelines and applications to ensure efficiency and scalability.
- Problem-Solving: Troubleshoot and resolve complex technical issues related to data migration, data quality, and system performance.
- Collaboration: Work closely with cross-functional teams, including business analysts, data scientists, and other engineers, to understand requirements and deliver successful outcomes.
- Mentorship: Provide guidance and technical support to junior engineers, helping them grow their skills and contribute effectively to the team.
Required Skills and Qualifications:
- Minimum of 10 years of experience in the IT industry, with a focus on data engineering and ETL development.
- Extensive hands-on experience with Datastage for building and managing ETL jobs.
- Proven experience in Cloud Migration projects, with a strong understanding of the challenges and best practices involved in moving data to the cloud.
- In-depth knowledge and hands-on experience with Azure Data Factory (ADF) for building and orchestrating data workflows.
- Strong proficiency in Databricks and its ecosystem, including notebooks, jobs, and Delta Lake.
- Expertise in PySpark for developing data processing applications on a distributed computing framework.
- Solid understanding of cloud platforms, preferably Microsoft Azure.
- Experience with SQL and database concepts.
- Excellent analytical, problem-solving, and communication skills.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
