T
Data Engineer
Search Jobs
Search by job, company or skills
Search by job, company or skills
Key responsibilities
1) Should be able to design strategies and programs to collect, store, analyse and visualize data from various sources.
2) Should be able to develop big data solution recommendations and ensure implementation of the chosen big data solution.
3) Needs to be able to program, preferably in different programming/scripting languages such as Scala, Python, Java, Pig or SQL.
4) Proficient knowledge in Big data frameworks Spark, Map Reduce,
5) Should have an understanding of Hadoop, Hive, HBase, MongoDB and/or MapReduce.
6) Should also have experience with one of the large cloud-computing infrastructure solutions like Amazon Web Services or Elastic MapReduce.
7) Tuning the Spark Engine for high volume of data ( approx billion records) processing using BDM.
8) Troubleshoot data issues, deep dive into root cause analysis of any performance issue
Job ID: 153330897
Skills:
Java, Unix, Bash Scripting, Autosys, Sql, Python, Tivoli
Skills:
Hadoop, Scala, Data Modeling, Emr, Sparksql, Informatica, Bodi, SSIS, Sql, Hive, Odi, Sql Pl, Datastage, Hiveql, Spark, Python, MDX, SQL DDL, KornShell, ETL pipelines
Skills:
Pyspark, Sql, Git, Azure Data Lake, Flask, FastAPI, Python, Azure DevOps, Azure Web Apps, Streamlit, Azure Monitor, Application Insights
Skills:
Terraform, Docker, Scala, Pyspark, Spark, Python 3, AWS, ETL fundamentals, Java JVM memory fundamentals
Skills:
bigtable , Github, BigQuery, DataFlow, Python, Pub Sub, GitHub Actions, Cloud Composer
