Infrastructure as Code (IaC), Github, Data Modeling, Kafka, Apache Airflow, Terraform, Docker, Python, Java, Streaming, BigQuery, Scala, Dataproc, Sql, Jenkins, Spark, Data Warehousing, Apache Beam, DataFlow, Kubernetes, Etl, Cloud Spanner, Cloud Composer, dbt, AI Platform, Batch streaming, Data Catalog, data fusion, CI/CD, Pub/Sub

RDS, Pyspark, AWS Glue, Dynamodb, Emr, Sql, Lambda, Cloudwatch, Sqs, Iam, Sns, Python, AWS, Lake Formation, CloudTrail

Unit Testing, Hadoop, Pyspark, Apache Spark, User Acceptance Testing, Hive, System Testing, Shell scripting, Python Programming, Data historical load and overall Framework concepts, AWS ecosystem, Spark related performance tuning, Git repository, CDC operations, UNIX operating system concepts, Google Cloud BigQuery, Jenkins or equivalent CICD tool, Agile delivery model, Cloudera Hortonworks Data Platform, AWS S3 Filesystem operations

Pyspark, Databricks, Python, Sql, Medallion Architecture

Pyspark, Python, Sql, Data Transformation Integration, ETL Data Pipeline Development, Azure Data Engineering, Azure Databricks ADB, Azure Data Factory ADF

Apis, Pyspark, Apache Spark, Azure Databricks, Database Development, Sql, Azure Sql, Azure Data Factory, System Design, Cosmos DB, Python, NoSQL databases, cloud-native architecture, Synapse, ADLS

Azure Data Factory, Data Modeling, Data Integration, Python, Sql, Business Intelligence systems, Relational Database Design, workflow orchestration, event processing, ETL pipelines

Pandas, Python, Aws S3, pyarrow, identity-based grouping strategies, HuggingFace Datasets, tokenizer-specific formatting, Content hashing, batch processing workflows, JSONL processing, Multi-turn conversation chat data structures, Arrow-based storage