Lead Data Engineer - AWS, Databricks & Graph Technologies

GeekSoft Consulting

10-12 Years

Not Disclosed

Bengaluru, India

This job is no longer accepting applications

Job Description

Help design, build and continuously improve the clients online platform.
Research, suggest and implement new technology solutions following best practices/standards.
Take responsibility for the resiliency and availability of different products.
Be a productive member of the team.

Requirements

Minimum 10 years of experience in Data Engineering.
Experienced Lead Data Engineer to design, develop, and implement modern cloud-native data platforms that enable analytics, artificial intelligence, and data-driven decision-making.
Work closely with architects, engineers, and business stakeholders to solve complex data challenges across diverse industries, including government, financial services, infrastructure, energy, and manufacturing.
As a technical leader, play a key role in defining data architecture standards, mentoring Data Engineers, and driving best practices within the Data & Analytics team.
Design and implement scalable Data Lakes, Data Warehouses, and Lakehouse architectures.
Develop and maintain robust ETL/ELT pipelines for both batch and real-time data processing.
Design and optimize data models while ensuring data quality, governance, and consistency.
Build and support cloud-native data solutions on AWS.
Develop event-driven and serverless architectures to support modern data platforms.
Enable analytics, business intelligence, artificial intelligence, and machine learning initiatives through efficient data solutions.
Provide technical leadership, mentor Data Engineers, and contribute to architecture and design decisions.
Collaborate within Agile teams to deliver innovative and scalable client solutions.
Strong experience designing and implementing cloud-native data platforms.
Deep understanding of Data Engineering principles, ETL/ELT processes, Data Modeling, and Data Warehousing concepts.
AWS Cloud Technologies: Amazon S3, Amazon Redshift, AWS Glue, Amazon Athena, AWS Lambda, Amazon DynamoDB, Amazon ECS/Fargate, AWS Step Functions, Amazon SNS/SQS
Data Engineering & Big Data Technologies: Apache Spark, Apache Kafka, SQL, Python
Relational databases such as PostgreSQL, Oracle, SQL Server, or equivalent.
NoSQL databases including MongoDB, DynamoDB, Cassandra, or similar platforms.
Engineering & DevOps: Git version control, Docker, CI/CD pipelines, Infrastructure as Code (IaC), Agile/Scrum methodologies
Databricks & Modern Data Platforms: Databricks, Delta Lake, Unity Catalog, Lakehouse Architecture
Experience with Snowflake, Microsoft Fabric, or similar modern data platforms
Graph Technologies: Neo4j, Amazon Neptune, GraphDB, or similar graph databases, Graph Data Modeling, Graph Analytics, Knowledge Graphs, Data Lineage and Dependency Mapping
Additional Preferred Skills: Event-Driven Architecture, Microservices Architecture, Machine Learning and Advanced Analytics
Experience with Microsoft Azure Data Platform technologies
Bachelor's or Master's degree in Computer Science, Informatics, Data Science, Engineering, or a related field.