V
Data Engineer
V
Data Engineer
VirtusaThis job is no longer accepting applications
Job Description
About The Role
We are seeking an experienced Senior Software Engineer with deep Python expertise to design, develop, and maintain backend services and data platform components for an AI-driven data intelligence platform. This is an individual contributor role with significant technical influence, requiring architectural leadership, mentorship, and the delivery of scalable, production-grade backend solutions.
Key Responsibilities
Backend Development and Architecture
Design and develop high-performance backend services and RESTful APIs using Python frameworks such as FastAPI, Django, or Flask following clean architecture and domain-driven design principles.
Build and maintain asynchronous processing pipelines using Celery, Redis, and event-driven microservices.
Architect scalable microservices for high-throughput data ingestion, transformation, and analytics.
Own end-to-end backend feature delivery from design through deployment, performance optimization, and production support.
Data Platform and Integration
Develop and optimize data pipelines integrating Databricks, SQL databases, and cloud-native storage services.
Design caching strategies, data access layers, and API contracts for internal and external consumers.
Collaborate with Data Engineering and Data Science teams to operationalize machine learning models and analytics workflows.
Integrate third-party APIs and enterprise systems with a focus on reliability, fault tolerance, and scalability.
Quality and Engineering Excellence
Lead code reviews and establish coding standards and engineering best practices.
Develop comprehensive unit, integration, and contract testing strategies.
We are seeking an experienced Senior Software Engineer with deep Python expertise to design, develop, and maintain backend services and data platform components for an AI-driven data intelligence platform. This is an individual contributor role with significant technical influence, requiring architectural leadership, mentorship, and the delivery of scalable, production-grade backend solutions.
Key Responsibilities
Backend Development and Architecture
Design and develop high-performance backend services and RESTful APIs using Python frameworks such as FastAPI, Django, or Flask following clean architecture and domain-driven design principles.
Build and maintain asynchronous processing pipelines using Celery, Redis, and event-driven microservices.
Architect scalable microservices for high-throughput data ingestion, transformation, and analytics.
Own end-to-end backend feature delivery from design through deployment, performance optimization, and production support.
Data Platform and Integration
Develop and optimize data pipelines integrating Databricks, SQL databases, and cloud-native storage services.
Design caching strategies, data access layers, and API contracts for internal and external consumers.
Collaborate with Data Engineering and Data Science teams to operationalize machine learning models and analytics workflows.
Integrate third-party APIs and enterprise systems with a focus on reliability, fault tolerance, and scalability.
Quality and Engineering Excellence
Lead code reviews and establish coding standards and engineering best practices.
Develop comprehensive unit, integration, and contract testing strategies.
More Info
About Company
Skills:
Aws Lambda, Data Modeling, Aws Redshift, Pyspark, AWS Glue, Apache Spark, AWS Athena, Sql, ELT, Git, AWS IAM, Data Warehousing, Python, Aws S3, Etl, Airflow, AWS EMR, AWS Step Functions, CI CD
Skills:
data engineering , Spark, Databricks, Emr, Airflow, AWS data services, Micro-services architecture, Streaming pipelines, Athena
Skills:
Power Bi, Power Automate, Power Query, Sql, Microsoft 365, Dax, Data Analytics, Python, Root Cause Analysis, Process Automation, Dashboard Development, Sharepoint, KPI Reporting, ETL Data Pipelines
Skills:
snowflake , Data Warehousing, Kafka, Automated Testing, ELT, Docker, Python, Azure DevOps, AWS, Data Modelling, Sql, Devops, Data Quality, Git, Databricks, Kubernetes, Etl, data lakes, Security, MDM, Event Hubs, metadata lineage, production reliability, GitHub Enterprise, DataOps, dbt, CI CD pipelines