Lead Data Engineer - R01566080

Brillio

Bengaluru, India

8-10 Years

Save

Posted 5 hours ago
Be among the first 10 applicants

Early Applicant

Job Description

Lead Data Engineer

Primary Skills

ETL Fundamentals, SQL, BigQuery, Dataproc, SQL (Basic + Advanced), Python, Data Catalog, Data Warehousing, Composer, Dataflow, Cloud Trace, Cloud Logging, Cloud Storage, Datafusion, Modern Data Platform Fundamentals, Data Modelling Fundamentals, PLSQL, T-SQL, Stored Procedures

Specialization

AWS Data Engineering Basic: Senior Data Engineer

Job requirements

We are seeking a highly skilled and motivated Senior Data Engineer with 8-10 years of experience in designing, building, and maintaining scalable cloud-based data platforms. The ideal candidate will have strong expertise in AWS data services, modern data engineering practices, and data warehousing solutions. This role requires a hands-on engineer who can collaborate with cross-functional teams, translate business requirements into technical solutions, and drive the development of robust, high-performance data pipelines that support analytics, reporting, and AI/ML initiatives. Key Responsibilities Design, develop, and maintain scalable and reliable data pipelines, ETL/ELT frameworks, and data integration solutions. Build cloud-native data solutions using AWS services including Glue, Lambda, Redshift, Aurora, OpenSearch, Step Functions, SNS, and S3. Develop and optimize data processing workflows using Python, PySpark, SQL, and PL/SQL. Design and implement data warehouse solutions, data marts, and data models to support enterprise reporting and analytics. Work closely with business stakeholders, product teams, data scientists, and analysts to understand requirements and deliver high-quality data solutions. Ensure data quality, integrity, governance, security, and compliance across the data ecosystem. Optimize database performance, query execution, and large-scale data processing workloads. Implement monitoring, alerting, and troubleshooting mechanisms to ensure platform reliability and operational excellence. Participate in solution design discussions, architecture reviews, and cloud modernization initiatives. Mentor junior team members and promote engineering best practices, code quality standards, and knowledge sharing. Support production deployments, issue resolution, and continuous improvement activities. Required Skills & Qualifications Experience
8-10 years of experience in Data Engineering, Data Warehousing, and Cloud Data Platform development.
Proven experience delivering enterprise-scale data engineering solutions in cloud environments.
AWS Cloud Technologies Strong hands-on experience with: Cloud Technologies: AWS Glue, Lambda, Redshift, Aurora, OpenSearch, Step Functions, SNS, S3 Programming & Data Engineering Skills: Python, PySpark, SQL, PL/SQL, Data Warehousing Programming & Data Engineering
Advanced proficiency in Python and PySpark for large-scale data processing.
Strong expertise in SQL and PL/SQL development.
Experience building and optimizing ETL/ELT pipelines and data ingestion frameworks.
Solid understanding of data warehousing concepts, dimensional modeling, and database design.
Experience working with structured, semi-structured, and unstructured data.
Data Architecture & Platform Engineering
Strong understanding of modern data architectures, including Data Lakes and Data Warehouses.
Experience designing scalable, secure, and high-performance data solutions.
Knowledge of data quality frameworks, governance practices, and data lifecycle management.
Experience with workflow orchestration, event-driven architectures, and distributed data processing. Professional Skills
Strong analytical and problem-solving capabilities.
Excellent communication and stakeholder management skills.
Ability to work independently and collaboratively in a fast-paced environment.
Strong ownership mindset with a focus on quality, reliability, and continuous improvement. Preferred Qualifications
Experience supporting Data Science, Machine Learning, AI, or Advanced Analytics initiatives.
Exposure to modern Data Lakehouse architectures.
Experience with CI/CD pipelines, DevOps practices, and Infrastructure as Code (IaC).
Experience working in Agile/Scrum delivery environments.
AWS Certifications such as AWS Certified Data Engineer, Solutions Architect, or equivalent cloud certifications.
Success Criteria The successful candidate will be able to:
Deliver scalable and reliable cloud-base