Data Engineer II

Location: Bangalore/Mumbai

About Media.net :

Media.net is a leading, global ad tech company that focuses on creating the most transparent and efficient path for advertiser budgets to become publisher revenue. Our proprietary contextual technology is at the forefront of enhancing Programmatic buying, the latest industry standard in ad buying for digital platforms.

The Media.net platform powers major global publishers and ad-tech businesses at scale across ad formats like display, video, mobile, native, as well as search. Media.net's U.S. HQ is based in New York, and the Global HQ is in Dubai. With office locations and consultant partners across the world, Media.net takes pride in the value-add it offers to its 50+ demand and 21K+ publisher partners, in terms of both products and services.

What does the team do

• Every single web page view hits one or more services and hosts high scale service to handle this large volume of requests, across 5 million unique topics.

• Some of the platforms built and managed by the team.

• To achieve this we use cutting edge Machine Learning and AI technologies on a large Hadoop cluster.

• Tech Stack – Java, Elastic Search/Solr, Kafka, Spark, Machine Learning, NLP, Deep Learning, Redis, Big Data technologies like Hadoop, HBase, YARN etc.

Roles and Responsibilities:

• Design, execution and management of a large and complex distributed data systems.

• Monitoring of performance and optimizing existing projects.

• Researching on and integrating any Big Data tools and frameworks required to provide requested capabilities.

• Understanding Business/Data requirements and implementing scalable solutions.

• Creating reusable components and data tools which help all the teams in the company to integrate with our data platform.

Who should apply for this role

• 2 to 4 years of experience in big data technologies (Apache Hadoop) and relational databases (ms sql server/oracle/MySQL/postgres).

• Proficiency in at least one of the following programming languages - Java, Python or Scala.

• Expertise in SQL (T-SQL/PL-SQL/SPARK-SQL/HIVE-QL).

• Proficiency in Apache Spark. Hands on knowledge of working with Data Frames, Data Sets, RDDs, Spark SQL/PySpark/Scala APIs with deep understanding of Performance Optimizations.

• Good Understanding of Distributed Storage (HDFS/S3).

• Strong analytical/quantitative skills and comfortable working with very large sets of data.

• Experience with integration of data across multiple data sources.

• Good understanding of distributed computing principles.

Good to have skills:

• Experience with Message Queues (e.g., Apache Kafka).

• Experience with MPP systems (e.g., Redshift/Snowflake).

• Experience with NoSQL storage (e.g., MongoDB).