Key Skills: Snowflake, Data Engineer, Data Engineering, MySQL, Python
Roles and Responsibilities:
- Design, develop, and optimize data pipelines using snowflake, including multi-layered data models.
- Optimize queries and manage cost efficiency of snowflake environments.
- Design, deploy, and manage scalable data solutions using aws cloud services.
- Build big data solutions using apache spark for real-time and batch processing.
- Develop and manage end-to-end etl/elt pipelines using apache airflow, apache nifi, or dbt.
- Build and optimize stream processing solutions using kafka for real-time ingestion and processing.
- Write complex sql queries and design scalable database architectures for operational and analytical workloads.
- Continuously improve performance, scalability, and reliability of the data platform.
- Collaborate with cross-functional teams and coordinate with offshore development teams.
Skills Required:
- Strong experience in Snowflake data modeling, performance tuning, and query optimization.
- Solid understanding of AWS cloud services such as S3, EMR, EC2, Redshift, Athena, Lambda, and Glue.
- Hands-on experience with big data technologies including Apache Spark and Kafka.
- Experience with data orchestration tools such as Apache Airflow, Apache NiFi, or dbt.
- Proficiency in programming languages such as Python and Scala for data engineering tasks.
- Strong expertise in SQL and relational databases including MySQL, PostgreSQL, and SQL Server.
- Experience in building scalable ETL/ELT pipelines and real-time data processing systems.
- Familiarity with CI/CD tools like Jenkins and infrastructure as code tools such as Terraform.
- Understanding of cloud platforms such as Azure and Google Cloud is an added advantage.
- Strong analytical, problem-solving, and communication skills to work across teams.
Education:
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
- Relevant certifications in Snowflake, AWS, or data engineering technologies are an added advantage.
