Role: Team Lead – Data Engineer (Python, PySpark, SQL)Experience: 10–17 Years
Responsibilities:- Align with key client initiatives and interface daily with customers across leading Fortune 500 companies to understand strategic requirements
- Ability to understand business requirements and tie them to data engineering solutions
- Open to work from client location as per the demand of the project / customer
- Facilitate technical aspects and develop highly scalable and fault-tolerant distributed data pipelines using Python and PySpark
- Excellent experience in data application development, support, integration development, and quality assurance
- Provide technical leadership and manage the team on a day-to-day basis
- Stay up-to-date on the latest technology to ensure the greatest ROI for customers
- Hands-on coder with good understanding of enterprise-level code
- Design and implement data pipelines, APIs, abstractions, and integration patterns to solve distributed data processing problems
- Experience in defining technical requirements, data extraction, data transformation, automating jobs, productionizing pipelines, and exploring new big data technologies within a parallel processing environment
Qualifications:- 10–17 years track record of relevant work experience and a Computer Science or related technical discipline is required
- Experience in development of enterprise-scale data applications; capable of developing frameworks and design patterns
- Experience with functional and object-oriented programming; Python is a must
- Strong hands-on experience in PySpark and SQL
- Hands-on knowledge in Hadoop ecosystem, MapReduce, HBase, and Elasticsearch
- Development and support experience in Big Data domain
- Experience with database modeling, data mining, and data warehousing
- Experience with data extraction, transformation, and large-scale data processing
- Unit, Integration, and User Acceptance Testing
- Effective communication skills (both written and verbal)
- Ability to collaborate with a diverse set of engineers, data scientists, and product managers
- Comfort in a fast-paced environment
Preferred Qualification:- Experience in Agile methodology
- Proficient with SQL and its variations across popular databases
- Experience working with large, complex datasets from multiple sources
Culture:- Must be a strategic thinker with the ability to think unconventional / out-of-the-box
- Analytical and solution-driven orientation
- Raw intellect, talent, and energy are critical
- Entrepreneurial and Agile mindset; understands the demands of a high-growth environment
- Ability to be both a leader and a hands-on doer
Python,Pyspark,Big Data Technologies
