Lead Data Engineer - Big Data Technologies

5-7 Years

Save

Early Applicant

Job Description

Description

Key Responsibilities :

Extract and process data from multiple databases, APIs, and cloud platforms by establishing strong source connections.
Design, develop, and maintain ETL pipelines for efficient data transformation and ingestion.
Write and optimize complex SQL queries for data extraction, validation, and reporting.
Utilize PySpark and BigData technologies (Hadoop, Spark, Databricks, etc.) to process and analyze large datasets.
Perform data analysis using Python (Pandas, NumPy, etc.) to uncover trends, patterns, and insights.
Develop and maintain interactive dashboards and reports in Power BI (good to have).
Work with cross-functional teams to understand business requirements and translate them into data-driven solutions.
Experience in Data warehouse schema creation and optimization
Ensure data accuracy, security, and compliance with company policies.
Present analytical findings and recommendations to stakeholders in a clear and actionable manner.

Required Skills & Qualifications

Minimum 5+ Years of experience in data analytics, data engineering, or business intelligence.
Strong expertise in SQL (writing, optimizing, and managing complex queries).
Hands-on experience in ETL processes and building/maintaining data pipelines.
Experience working with Big Data technologies (Hadoop, Spark, PySpark, Databricks, etc.).
Proficiency in Python for data analysis (Pandas, NumPy, etc.).
Experience of working on Fabric
Experience of working on Databrick
Experience in connecting to and processing data from multiple data sources (databases, APIs, cloud platforms).
Experience of working on Cloud platforms ( Azure /AWS / GCP)
Experience of delivering the Projects
Experience of requirement understanding
Experience of preparing estimations and creating POCs
Experience of working with different Project domains.
Proficiency in delivering High quality codes to clients
Proficiency in Data warehouse schema creation.
Experience of client communication.
Strong analytical and problem-solving skills.
Experience of mentoring and managing the team.

(ref:hirist.tech)