We are looking for a highly motivated Sr Data Engineer to build a state-of-the-art data platform to solve various data-driven use cases across the organization. This platform will host multiple data products, including but not limited to, Subscription, Content, and Product Analytics, Personalization and Recommendation, Marketing & Ad-Sales enablement. You will be building a solution in the cloud - handles both streaming and batch data processing, capable of solving any big data initiatives in scope now or in the future.This individual will bring in his/her expertise in a wide variety of big data processing frameworks (both open source and proprietary), large scale database systems (OLAP and OLTP), Batch & stream data processing.
Build software across our entire cutting-edge data platform, including event driven data processing, storage, and serving through scalable and highly available APIs, with cutting-edge technologies.
Change how we think, act, and utilize our data by performing exploratory and quantitative analytics, data mining, and discovery.
Work closely with data analysts and business stake holders to make data easily accessible and understandable to them.
Ensure data quality by implementing re-usable data quality frameworks.
Develop and enforce data engineering, security, data quality standards through automation.
Participate in supporting platform 24X7.
Be responsible for cloud cost and improving efficiency.
Contributing back to engineering community by writing/publishing blogs, articles, or papers in respected engineering conferences
Mentor and provide technical guidance to junior data engineers in the team.
What To Bring
Masters or Bachelors degree in computer science or similar discipline
5+ years of experience in software engineering and/or data engineering
Ability and willingness to learn new technologies and apply them at work to stay ahead of the curve.
Expertise in at least a few programming languages - Scala, Python, Java or similar.
Expertise in distributed data processing frameworks such as Apache Spark, Flink or similar.
Experience in working with Data platforms such as Databricks, Cosmos DB, EMR (Databricks is huge plus).
Expertise in building and managing large volume data processing (both streaming and batch) platform is a must.
Expertise in stream processing systems such as Kafka, Kinesis, Pulsar or Similar
Expertise in SQL and No-SQL Apache Cassandra, DynamoDB, MySQL
Expertise in OLAP databases such as Snowflake or Redshift.
Experience with variety of data Tools & frameworks (example Apache Airflow, Druid) will be a huge plus.
Expert with CI/CD pipelines, preferably GitHub Actions Workflows.
Experience with Analytics Tools such as Looker, Tableau is preferred.
Cloud (AWS) experience is preferred
Direct to consumer digital business experience is preferred
Strong interpersonal, communication and presentation skills.
