I
Data Engineer
Job Description
Databricks Certification Azure Data Engineer Associate Certification AWS Data Analytics Certification Experience with Delta Lake, Iceberg, or Lakehouse Architecture Knowledge of GenAI/LLM data pipelines
Design, develop, and optimize scalable batch and real-time data pipelines. Build and maintain robust ETL/ELT processes for ingesting, transforming, and loading data from multiple sources. Develop and manage data lakes, data warehouses, and metadata frameworks. Collaborate with cross-functional teams to understand data requirements and deliver data solutions. Ensure data quality, governance, lineage, and security standards are maintained. Monitor, troubleshoot, and optimize data workflows and pipeline performance.
Design, develop, and optimize scalable batch and real-time data pipelines. Build and maintain robust ETL/ELT processes for ingesting, transforming, and loading data from multiple sources. Develop and manage data lakes, data warehouses, and metadata frameworks. Collaborate with cross-functional teams to understand data requirements and deliver data solutions. Ensure data quality, governance, lineage, and security standards are maintained. Monitor, troubleshoot, and optimize data workflows and pipeline performance.
More Info
Key Skills
About Company
Skills:
Api Development, AWS or any public cloud platform, SQL proficiency, Apache Spark or Databricks, CI CD pipelines, Core Java or Core Python
Skills:
BigQuery, Google Cloud Platform, Sql, Apache Airflow, Cloud Storage, DataFlow, Data Modelling, Python, Workflow orchestration tools, Cloud Functions, powerdesigner, Kafka streaming, Teradata, Cloud Composer, Python SDK integrations, ELT pipelines, Pub/Sub
Skills:
Hadoop, Power Bi, Ssrs, Msbi, SSIS, JIRA, Sql, Apache Airflow, Ssas, Spark, Microsoft BI, Oracle, Dax, GenAI tools, Airflow, MDX, Claude, GitHub Copilot, Microsoft Fabric
Skills:
Apache Airflow, Aws Lambda, Amazon Redshift, Amazon S3, AWS Glue, Python, Sql, Amazon Athena, AWS Step Functions, ETL ELT, AWS MWAA
Skills:
containerization , Jenkins, Sap Data Services, Gitlab, Databricks, Sql, Microsoft Azure PaaS, DevOps methodologies, ADLS, Azure ADF