Senior Data Engineer

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.

If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Role: Senior Data Engineer - Azure

Experience Level: 4 to 7 Years

Work location: Mumbai, Bangalore & Trivandrum

Notice Period: 0- 15 days

Role & Responsibilities:

. Support the planning and implementation of data flow design, provide sizing and configuration assistance.

.Assisting the Solution Architects on programs for transformations and modernizations of data solutionsusingAzure cloud data technologies.

. Write production quality code to deploy ML workflows in a distributed environment.

. Design and BuildAzure Data Pipelines and Data StreamsDesign and build a hybrid data solution using acombination ofon-premise and in-cloud services.

. Diagnose and troubleshoot complex distributed systems problems and develop solutions with a significantimpact on our massive scale.

. Build tools to ingest and process terabyte-scale data on a daily basis.

. Communicate with a wide set of teams, including product owners, business analysts, data and enterprisearchitects, platform engineers, and cloud vendors, etc.

. Compilation of detailed design documents, technical specification documents, and test plans as required.

.Independently manage development tasks as per technical specification documents, prod deployment,post-prod support, etc

. Managing support to Unit Testing, SIT & UAT.

. Coordinate with business and support teams on testing and migration activities.

Skills expectation:

. A strong hands-on knowledge of Spark/Pyspark.

. Expertise in writing SQL queries and performance optimization.

. Strong hands-on experience with a programming language like Python(preferred), java, or scala.

.Prior experience with working on cloud platforms like Azure(preferred) , AWS or GCP.

. Priorexperience of ADF is a plus.

. DataBricks experience is good to have.

. Data Warehouse: Snowflake or Synapse.

. Design Dimensional Models for Data Warehouse and Data Marts.

. Knowledge of version control systems and build automation tools like Azure DevOps and/or MLOps.

.Experience with operating system internals, file systems, disk/storage technologies, and storage Protocol.