T
Pyspark Data Engineer
T
- Posted 2 days ago
- Be among the first 40 applicants
Job Description
Greetings from TATA CONSULTANCY SERVICES!
TCS is hiring for Pyspark
Location: - Chennai, Hyderabad, Bangalore, Kolkata
Experience Range: - 4+ years
- 4 – 7 years of total experience with good knowledge of Knowledge of Pyspark, Big Data experience, AWS EMR, S3, IAM, Lambda, SNS, SQS, Python and AWS services
- Good python, Pyspark scripting skills
- Spark UI/Optimization/debugging techniques
- Intermediate SQL exposure – Subquery, Joins, CTE's
- Good understanding of Database technologies
- Excellent communication skills to liaise with Business & IT stakeholders.
- Expertise in planning execution of a project and efforts estimation.
- Understanding of Data Vault, data mesh and data fabric architecture patterns.
- Exposure to working in Agile ways of working.
More Info
About Company
Data Engineer (PySpark, Informatica BDM, ETL, Banking)Insights Techhub International Private Limited
Skills:
Informatica BDM (Big Data Management), Pyspark, Etl, Banking or Financial Services domain
Skills:
BigQuery, Pyspark, Apache Spark, Dataproc, Sql, ELT, Cloud Storage, Git, Gcp, DataFlow, Python, Etl, Airflow, cloud security best practices, CI CD, Pub Sub, Cloud Composer, data quality monitoring
Skills:
Spark SQL, S3, Pyspark, Scala, Apache Spark, Emr, Redshift, Apache Airflow, Lambda, Hadoop Ecosystem, Hive, Cloudwatch, AWS, Step Functions, HDFS, DAG design and optimization, Workflow orchestration, Glue, Athena, Distributed Data Processing
Skills:
Hive, Hadoop, Linux, Pyspark, Apache Oozie, Kafka, Sql, Etl, Airflow, Cloudera Data Platform