Insight Global is hiring a Sr. Data Engineer for our client, a major telecommunications firm providing connectivity services to millions of customers. The Senior Data Engineer is part of the Network Analytics team, responsible for building and maintaining the data infrastructure supporting enterprise networks.
Key Responsibilities:
- Design, develop, and maintain scalable ETL pipelines (primarily using Spark) to ensure data quality and availability.
- Implement monitoring and alerting solutions for data pipeline reliability and performance.
- Manage deployment pipelines for continuous integration and delivery.
- Optimize and manage data storage solutions (distributed file systems, relational databases, flat files, external APIs).
- Collaborate with cross-functional teams to align data solutions with business requirements.
- Provide technical leadership on data engineering best practices.
- Document and communicate data engineering processes to a diverse audience.
- Continuously evaluate and improve data engineering tools and methodologies.
Qualifications:
- Bachelor's degree (or relevant experience) in computer science, mathematics, statistics, physics, operations research, or related quantitative fields.
- Expertise in data engineering languages (Scala preferred, Java, Python).
- Experience with Big Data tools, especially Spark, and familiarity with Hadoop ecosystems.
- Proficiency with cloud platforms (AWS: S3, Lambda, Redshift, EMR).
- Strong understanding of relational and NoSQL databases, data architecture, warehousing, and marts.
- Experience implementing monitoring solutions and deployment pipelines for data applications.
- Strong communication and collaboration skills.
- Experience with Linux-based operating systems.
- Demonstrated ability and desire to continually expand skill set and share knowledge with others.
