Senior Engineer, Data Engineering
Senior Engineer, Data Engineering
News CorpThis job is no longer accepting applications
Job Description
Equal Opportunity Employer
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, national origin, protected veteran status, or disability status. EEO/Disabled/Vets
Job Description :
We are looking for a Senior Data Engineer with experience in excellent analytical abilities in GCP BigQuery, ETL/BI, Python, Composer/Airfl ow, SQL, GCP Services - with the ability to operate in an agile environment and keen to detail while understanding the larger data warehouse.
Skills & Competencies
A minimum of 4.5+ years of working experience as a hands-on data engineering is a must
Advanced Degree in the fi eld of Computer Science, Data Engineering, Data Analytics or equivalent discipline
Strong programming experience with Python, SQL, ETL and effi ciently querying big datasets using SQL
Prior experience with CircleCI CI/CD, Terraform and Github version control is a must
Solid understanding of Data Collection, Data Ingestion, Data Integration, and Data Operations & Maintenance
Implement data governance practices, including data quality, security, and compliance, in line with organizational standards.
Domain knowledge on News, Media & Communications is an added advantage.
R E S P O N S I B I L I T I E S
Design, build, and maintain scalable ETL/ELT pipelines in Google Cloud, using tools like Cloud Composer (Airfl ow), Dataproc and BigQuery.
Collaborate with cross-functional teams to defi ne data architecture and data warehouse requirements
Automate data pipelines, ETL/ELT processes, and data integration workfl ows using infrastructure-as-code (IaC) and confi guration management tools
Deep understanding of BigQuery, including SQL, optimization techniques, partitioning, clustering, and performance tuning.
Deliver production quality, modular, reproducible code in Python, SQL
Implement continuous integration and continuous deployment (CI/CD) pipelines for data pipelines and data-related applications
Keep abreast of the latest developments in the fi eld through continuous learning and proactively champion promising new methods relevant to the problems at hand.
Our Tech Stack
Job Category:
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, national origin, protected veteran status, or disability status. EEO/Disabled/Vets
Job Description :
We are looking for a Senior Data Engineer with experience in excellent analytical abilities in GCP BigQuery, ETL/BI, Python, Composer/Airfl ow, SQL, GCP Services - with the ability to operate in an agile environment and keen to detail while understanding the larger data warehouse.
Skills & Competencies
A minimum of 4.5+ years of working experience as a hands-on data engineering is a must
Advanced Degree in the fi eld of Computer Science, Data Engineering, Data Analytics or equivalent discipline
Strong programming experience with Python, SQL, ETL and effi ciently querying big datasets using SQL
Prior experience with CircleCI CI/CD, Terraform and Github version control is a must
Solid understanding of Data Collection, Data Ingestion, Data Integration, and Data Operations & Maintenance
Implement data governance practices, including data quality, security, and compliance, in line with organizational standards.
Domain knowledge on News, Media & Communications is an added advantage.
R E S P O N S I B I L I T I E S
Design, build, and maintain scalable ETL/ELT pipelines in Google Cloud, using tools like Cloud Composer (Airfl ow), Dataproc and BigQuery.
Collaborate with cross-functional teams to defi ne data architecture and data warehouse requirements
Automate data pipelines, ETL/ELT processes, and data integration workfl ows using infrastructure-as-code (IaC) and confi guration management tools
Deep understanding of BigQuery, including SQL, optimization techniques, partitioning, clustering, and performance tuning.
Deliver production quality, modular, reproducible code in Python, SQL
Implement continuous integration and continuous deployment (CI/CD) pipelines for data pipelines and data-related applications
Keep abreast of the latest developments in the fi eld through continuous learning and proactively champion promising new methods relevant to the problems at hand.
Our Tech Stack
- Google Cloud Platform is IaaS
- BigQuery, DataProc, Python, Pyspark, SQL
- Composer / Airfl ow - orchestration tools
- Terraform, CircleCI, CI/CD, GIthub, Jira, Confl uence
Job Category:
More Info
About Company
Skills:
Nosql, Quicksight, Tableau, Redshift, Oracle, Python, Sql, Etl
Skills:
Pytorch, Python, Hugging Face, ESPnet, Model training, ASR fine-tuning, nemo, Evaluation frameworks, MLflow logging
Skills:
Hadoop, Pyspark, Scala, Apache Spark, Aws Redshift, Python, Sql, Airflow, AWS EMR
Skills:
Pyspark, Spark, Databricks, Python, Azure Data Engineering
Skills:
Tensorflow, Pytorch, Python, MLops, machine learning engineering, generative AI, Ai Development