BASIC FUNCTIONS: This position is a subject matter expert on broad technologies and sets the architectural standard for the workgroup. The position executes corporate tactical technology strategy on large multifunctional development activities, solves complex technical problems, writes complex code for computer systems, and serves as a senior source of expertise. This position is expected to be seen as a go to person across the division for a set of technologies.
Qualifications
- 12+ years of Software Engineering experience
- BS Engineering/Computer Science or equivalent experience required; advanced degree preferred
Requirements
- Should have 12+ years of experience. 8+ years of relevant experience.
- Knowledge of SDLC process like requirement gathering, design, implementation (Coding), testing, deployment, and maintenance. Solid understanding of software development principles, design patterns, and best practices.
- Proficient in requirement gathering and documentation.
- Excellent communication skills verbal and written.
- Strong problem-solving skills.
- Proven experience as a Senior Analytics Engineer, working on complex projects and delivering high-quality software solutions.
- Strong proficiency in Python and PySpark programming language and its libraries
- In-depth knowledge and experience in SQL, including designing and optimizing complex queries, and working with relational databases such as MySQL, PostgreSQL, or Oracle.
- Proficiency in working with version control systems like Git.
- Proficiency in Unity Catalog and various AI Agents like Copilot, Databricks Genie, etc.
- Strong problem-solving and analytical skills, with the ability to quickly identify and resolve issues.
- Excellent communication and collaboration skills, with the ability to work effectively in a team environment.
- Experience with Agile development methodologies.
- Must have a positive attitude and desire to learn new processes and technologies.
Key Responsibilities:
- Design and Develop Scalable Data Pipelines: Architect and implement robust, scalable, and high-performance data pipelines using Azure Data Factory (ADF) and Azure Databricks to ingest, transform, and load data from various sources.
- Data Integration and Transformation: Build complex data workflows and transformation logic using PySpark and SQL, ensuring efficient processing of structured and unstructured data.
- Data Modeling and Architecture: Design and maintain data models (star/snowflake schemas), data lakes, and data warehouses to support analytics, reporting, and machine learning use cases.
- Performance Optimization: Monitor and optimize data pipeline performance, troubleshoot bottlenecks, and implement best practices for data partitioning, caching, and parallel processing.
- Data Quality and Governance: Implement data validation, cleansing, and quality checks. Ensure compliance with data governance policies and maintain data lineage and metadata.
- Collaboration and Stakeholder Engagement: Work closely with data scientists, analysts, product managers, and business stakeholders to gather requirements, deliver solutions, and support data-driven decision-making.
- Documentation and Knowledge Sharing: Create and maintain comprehensive documentation for data pipelines, architecture, and processes. Mentor junior engineers and contribute to team knowledge sharing.
- Innovation and Continuous Improvement: Stay current with emerging technologies in data engineering and cloud computing. Evaluate and recommend new tools and frameworks to improve efficiency and scalability.
- AI Integration: Working knowledge of AIassisted tools like Databricks Assistant, Genie, Copilot, etc. (e.g., for data analysis, code generation, automation, and productivity) and the ability to effectively leverage them to improve solution delivery, efficiency, and client outcomes.
We know your well-being and happiness are key to a long and successful career. We are delighted to offer country specific benefits. Click here to access benefits specific to your location.
