Data Engineer
Data Engineer
Falabella IndiaThis job is no longer accepting applications
Job Description
Requirements
- Should have a total of 3-5 years of experience in Data Engineering.
- Should have hands-on experience in distributed computing.
- Should have working experience in Data Architecture design.
- Strong in programming languages like Python and Java.
- Should be aware of storage and compute options, and when to choose what.
- Must have one cloud hands-on experience.
- Proficient in either Apache Spark, Apache Beam or Apache Flink.
- Advanced SQL knowledge is mandatory.
- Must be aware of streaming concepts like Windowing, Late arrival, Triggers, etc.
- Should have exposure to GCP tools to develop an end-to-end data pipeline for various scenarios (including ingesting data from traditional databases as well as integration of API based data sources).
- Should have a business mindset to understand data and how it will be used for BI and Analytics purposes.
- Should have working experience on CI/CD pipelines, Deployment methodologies, and Infrastructure as Code.
- Should have advance understanding of different file formats, e. g., AVRO, Parquet, ORC, CSV and when to choose what.
- Good to have Hands-on experience on Kubernetes.
- Good to have a vector-based database like Qdrant.
- Should have a good understanding of Cluster Optimisation/ Pipeline Optimisation strategies.
More Info
Key Skills
GCP Tools
Parquet
Infrastructure as Code
Data Architecture Design
Qdrant
About Company
Skills:
Pyspark, AWS Glue, Data Modeling, Emr, Avro, Autosys, Sql, Cloudwatch, Terraform, Etl Tools, Databricks, Python, AWS, Airflow, Parquet, AWS Lake Formation, Iceberg
Skills:
Power Bi, Pyspark, Agile Scrum, Tableau, Databases, Git, Data Warehousing, Informatica Powercenter, Python, Airflow, Data Lake concepts, Advanced SQL and performance tuning, IICS, dbt, CI/CD
Skills:
Kms, Pyspark, Iam, Sql, Python, AWS Lake Formation
Skills:
data engineering , Data Governance, Data integration techniques and tools, Data Analysis Interpretation, Data Architecture Principles, Cloud-based data platforms, Distributed computing environments
Skills:
data engineering , AWS Glue, Data Modeling, Sql, Etl, Step Functions, Airflow, IAM roles and permissions