Senior Data Engineer

Essential Responsibilities:

As a Senior Software Engineer, your responsibilities will include:

• Building, refining, tuning, and maintaining our real-time and batch data infrastructure
• Daily use technologies such as Python, Spark, Airflow, Snowflake, Hive, FastAPI, etc.
• Maintaining data quality and accuracy across production data systems
• Working with Data Analysts to develop ETL processes for analysis and reporting
• Working with Product Managers to design and build data products
• Working with our DevOps team to scale and optimize our data infrastructure
• Participate in architecture discussions, influence the road map, take ownership and responsibility over new projects
• Participating in on-call rotation in their respective time zones (be available by phone or email in case something goes wrong)

Desired Characteristics:

• Minimum 8 years of software engineering experience.
• An undergraduate degree in Computer Science (or a related field) from a university where the primary language of instruction is English is strongly desired.
• 2+ Years of Experience/Fluency in Python
• Proficient with relational databases and Advanced SQL
• Expert in usage of services like Spark and Hive.
• Experience working with container-based solutions is a plus.
• Experience in adequate usage of any scheduler such as Apache Airflow, Apache Luigi, Chronos etc.
• Experience in adequate usage of cloud services (AWS) at scale
• Proven long term experience and enthusiasm for distributed data processing at scale, eagerness to learn new things.
• Expertise in designing and architecting distributed low latency and scalable solutions in either cloud and on-premises environment.
• Exposure to the whole software development lifecycle from inception to production and monitoring.
• Experience in Advertising Attribution domain is a plus
• Experience in agile software development processes
• Excellent interpersonal and communication skills