About The Client: An Indian multinational information technology (IT) consulting company headquartered in Noida, The company has offices in 52 countries and over 225,944 employees. The Client is a global IT services and consulting company that offers a wide range of services and products across various industries like IT Infrastructure Services, Cybersecurity Services, Cloud Services, Big Data and Analytics, Internet of Things (IoT) Solutions, Semiconductor Services and Enterprise Software Products.
About The Job:
We are seeking a skilled Data Engineer with strong experience in PySpark, Azure Databricks, and Azure Data Factory (ADF) to design, build, and optimize scalable data pipelines.
The ideal candidate will work closely with data architects, analysts, and business stakeholders to deliver high quality data solutions in a fast-paced environment.
Essential Job Functions:
Design, develop, and maintain ETL/ELT pipelines using PySpark and Databricks.
Build and orchestrate complex workflows using Azure Data Factory (ADF).
Develop optimized Spark jobs for batch and streaming workloads.
Implement Delta Lake solutions for data ingestion, curation, and quality enforcement.
Work with various data sources (SQL, NoSQL, APIs, Blob, ADLS).
Optimize data processing performance and manage cluster resources in Databricks.
Implement data quality, data validation, and monitoring frameworks.
Collaborate with cross-functional teams for data modeling and solution design.
Ensure adherence to security, governance, and compliance standards.
Troubleshoot data issues, optimize existing jobs, and improve reliability.
Qualifications :
Min years of experience 4 - 8+
Technical Skills
Strong hands-on expertise in PySpark and Spark optimization.
Advanced experience with Azure Databricks (Delta Lake, notebooks, workflows, clusters).
Solid experience in Azure Data Factory (pipelines, mapping data flows, triggers).
Experience with Git-based CI/CD (Azure DevOps preferred).
Understanding of data warehousing concepts, SCD, star/snowflake schema.
Soft Skills
Strong problem-solving and analytical skills.
Ability to work in an Agile environment.
Excellent communication and collaboration abilities.
Preferred Qualifications
Microsoft Certified: Azure Data Engineer Associate.
Experience with streaming technologies (Event Hub, Kafka, Spark Streaming).
Exposure to DataBricks Workflows, Unity Catalog, or ML pipelines.
How to Apply: Interested candidates are encouraged to respond/submit their updated resumes, and for additional job opportunities, please visit Jobs In India - VARITE.
Unlock Rewards: Refer Candidates and Earn. If you're not available or interested in this opportunity, please pass this along to anyone in your network who might be a good fit and interested in our open positions. VARITE offers a Candidate Referral program, where you'll receive a one-time referral bonus based on the following scale if the referred candidate completes a three-month assignment with VARITE.
Experience Level Bonus Referral:0-2 yearsINR 5,0002-6 yearsINR 7,5006+ yearsINR 10,000 About VARITE: VARITE is a global staffing and IT consulting company providing technical consulting and team augmentation services to Fortune 500 Companies in USA, UK, CANADA and INDIA. VARITE is currently a primary and direct vendor to the leading corporations in the verticals of Networking, Cloud Infrastructure, Hardware and Software, Digital Marketing and Media Solutions, Clinical Diagnostics, Utilities, Gaming and Entertainment, and Financial Services.
Equal Opportunity Employer: VARITE is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, marital status, veteran status, or disability status.
