Search by job, company or skills

Virtusa

Data Engineering ETL with SQL

6-9 Years
new job description bg glownew job description bg glownew job description bg svg
  • Posted 2 days ago
  • Over 50 applicants
Quick Apply

Job Description

  • Develop and maintain a metadata driven generic ETL framework for automating ETL code
  • Design, build, and optimize ETL/ELT pipelines using Databricks (PySpark/SQL) on AWS .
  • Ingest data from a variety of structured and unstructured sources (APIs, RDBMS, flat files, streaming).
  • Develop and maintain robust data pipelines for batch and streaming data using Delta Lake and Spark Structured Streaming.
  • Implement data quality checks, validations, and logging mechanisms.
  • Optimize pipeline performance, cost, and reliability.
  • Collaborate with data analysts, BI, and business teams to deliver fit for purpose datasets.
  • Support data modeling efforts (star, snowflake schemas) de norm tables approach and assist with data warehousing initiatives.
  • Work with orchestration tools Databricks Workflows to schedule and monitor pipelines.
  • Follow best practices for version control, CI/CD, and collaborative development
  • Skills
  • Hands-on experience in ETL/Data Engineering roles.
  • Strong expertise in Databricks (PySpark, SQL, Delta Lake), Databricks Data Engineer Certification preferred
  • Experience with Spark optimization, partitioning, caching, and handling large-scale datasets.
  • Proficiency in SQL and scripting in Python or Scala.
  • Solid understanding of data lakehouse/medallion architectures and modern data platforms.
  • Experience working with cloud storage systems like AWS S3
  • Familiarity with DevOps practices Git, CI/CD, Terraform, etc.
  • Strong debugging, troubleshooting, and performance-tuning skills.

More Info

Job Type:
Function:
Employment Type:
Open to candidates from:
Indian

About Company

Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.
Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.
Virtusa is an Equal Opportunity Employer. All applicants will receive fair and impartial treatment without regard to race, color, religion, sex, national origin, ancestry, age, legally protected physical or mental disability, protected veteran status, status in the U.S. uniformed services, sexual orientation, gender identity or expression, marital status, genetic information or on any other basis which is protected under applicable federal, state or local law.
Applicants may be required to attend interviews in person or by video conference. In addition, candidates may be required to present their current state or government-issued ID during each interview. All candidates must be authorized to work in the USA.

Job ID: 118070829