Data Science

Your Profile

We are looking for experienced Data Engineers to support the design, development and deployment of scalable data pipelines. The ideal candidate will have strong hands-on experience with Azure Data Services and Databricks.They will be expected to:

• Design and implement scalable data pipelines using Azure Data Factory, Data Lake, and Databricks ensuring they are optimized for data processing and analytics.

• Develop andoptimizePySparkand SQL workflows for data transformation and orchestration.

• Create and refine processes for data modelling, mining and production to support operational and analytical needs.
• Establish and enforce data quality checks and validation routines to maintain high standards of data accuracy and reliability.
• Collaborate with AI/ML teams to support RAG-based LLM applications.

• Ensure data security and governance using Azure Key Vault and related services.

Your Role

• Over 6 years of hands-on experience with Azure Data Services including Data Lake, Data Factory, Key Vault and Cognitive Search.

• Proficient in Databricks ecosystem including cluster optimization, performance tuning withexpertisein Delta Lake,PySpark and orchestrating workflows.
• Experiencewith anyrelational SQL (SQLServer/Oracle) and NoSQL (MongoDB/DynamoDB) databases including Snowflake along with strong expertise in Python/PySpark for largescale data processing.
• Experience with real-time systems like Event Hubs, Apache Kafka, Spark-Streaming, etc.
• Experience withanyBigData frameworks like Spark/Kafka/ Hive/ Hadoop etc.
• Strong programming skills in Python and SQL for data engineering and analytics.
• Basic understanding of GenAI concepts including RAG and related AI/ML technologies and experience in generating embeddings for both structured and unstructured data sources would be preferred.
• Familiarity with DevOps practices including CI/CD pipelines, automation strategies and experience in technologies like GitHub and Bitbucket will be good to have.

• Working knowledge of BI tools (Tableau, Power BI) and data engineering platforms (Microsoft Fabric, Apache Storm, Apache NiFi) for reporting and pipeline setup will be beneficial.

What you will love about working here

• We recognize the significance of flexible work arrangements to provide support. Be it remote work, or flexible work hours, you will get an environment to maintain healthy work life balance.
• At the heart of our mission is your career growth. Our array of career growth programs and diverse professions are crafted to support you in exploring a world of opportunities.
• Equip yourself with valuable certifications in the latest technologies such as Generative AI.