Infrastructure as Code (IaC), Github, Data Modeling, Kafka, Apache Airflow, Terraform, Docker, Python, Java, Streaming, BigQuery, Scala, Dataproc, Sql, Jenkins, Spark, Data Warehousing, Apache Beam, DataFlow, Kubernetes, Etl, Cloud Spanner, Cloud Composer, dbt, AI Platform, Batch streaming, Data Catalog, data fusion, CI/CD, Pub/Sub

Azure Data Lake Storage (ADLS), Github, Data Factory, Power Bi, Pyspark, Data Governance, Data Warehouse, Sql, ELT, Spark, Databricks, Python, Azure DevOps, Etl, OneLake, Real-Time Intelligence, Azure AI Services, Dimensional data modeling, Microsoft Purview, CI/CD pipelines, Lakehouse, Synapse Analytics, Microsoft Fabric

RDS, Pyspark, AWS Glue, Dynamodb, Emr, Sql, Lambda, Cloudwatch, Sqs, Iam, Sns, Python, AWS, Lake Formation, CloudTrail

Unit Testing, Hadoop, Pyspark, Apache Spark, User Acceptance Testing, Hive, System Testing, Shell scripting, Python Programming, Data historical load and overall Framework concepts, AWS ecosystem, Spark related performance tuning, Git repository, CDC operations, UNIX operating system concepts, Google Cloud BigQuery, Jenkins or equivalent CICD tool, Agile delivery model, Cloudera Hortonworks Data Platform, AWS S3 Filesystem operations

Pyspark, Databricks, Python, Sql, Medallion Architecture

Pyspark, Python, Sql, Data Transformation Integration, ETL Data Pipeline Development, Azure Data Engineering, Azure Databricks ADB, Azure Data Factory ADF

Azure Data Factory, Data Modeling, Data Integration, Python, Sql, Business Intelligence systems, Relational Database Design, workflow orchestration, event processing, ETL pipelines

snowflake , Azure Data Factory, Power Bi, Sql, Python, DataOps

Pandas, Python, Aws S3, pyarrow, identity-based grouping strategies, HuggingFace Datasets, tokenizer-specific formatting, Content hashing, batch processing workflows, JSONL processing, Multi-turn conversation chat data structures, Arrow-based storage