Tech Lead - AWS SageMaker Job

Experience

8+ Years

Job Description

Key Responsibilities:

Data Engineering:

. Data LakeHouse:

o Simplify analytics and AI with unified data management across Amazon S3 data lakes, Amazon Redshift data warehouses, operational databases, and enterprise applications.

. Data Processing:

o Use Visual ETL jobs and unified notebooks to perform data analysis, preparation, and integration with data sources like Amazon S3, Redshift, and AWS RDS.

. Orchestration and Automation:

o Create and manage SageMaker Pipelines and Workflows to automate the ETL Pipelines, ML model continuous integration and continuous delivery (CI/CD) lifecycle.

. Data Governance and Collaboration:

o Work within collaborative projects in SageMaker Unified Studio, leveraging the SageMaker Catalog (built on Amazon DataZone) to securely share data and AI artifacts.

o Implement various features in the project like Lineage, Data Quality, Metadata forms, Business Glossaries etc.

AI/ML:

. End-to-End MLOps:

o Build, deploy, execute, and monitor complete data and AI workflows using SageMaker Unified Studio's integrated environment.

. Model Development:

o Develop, train, and evaluate ML and generative AI models using SageMaker's purpose-built tools and algorithms.

o Utilize advanced features such as HyperPod for distributed training, JumpStart for pre-trained models, and SageMaker Experiments for tracking model versions.

. Generative AI Development:

o Rapidly build and customize generative AI applications within the integrated Amazon Bedrock environment in Unified Studio.

o Implement advanced customization capabilities like Knowledge Bases, Guardrails, Agents, and Flows.

. Security and Compliance:

o Implement and maintain security best practices, including fine-grained access controls and data protection, within the SageMaker environment.

. Performance Optimization:

o Optimize the performance and cost of ML workloads by right-sizing compute resources and monitoring job metrics.

. Troubleshooting:

o Utilize Amazon Q Developer's generative AI-powered assistance for code generation and resolving issues within the notebooks and jobs.

Required Skills

. Experience with data processing and analytics using services integrated with Sagemaker Unified Studio (e.g., AWS Glue, Amazon EMR, Amazon Athena, AWS Redshift).

. Demonstrable expertise with Amazon SageMaker Unified Studio and its core components (Domain Units, Projects, Data Assets, Jobs, Data Products).

. Proficiency in PySpark, Python and relevant ML/AI libraries such as TensorFlow, PyTorch, and Scikit-learn.

. Experience with generative AI application development using Amazon Bedrock services within Unified Studio.

. Proven experience developing machine learning models and data pipelines on AWS.

. Solid understanding of MLOps principles and experience building CI/CD pipelines for ML models.

. Familiarity with AWS security best practices, including IAM roles, policies, and data governance, AWS Lake Formation

. Knowledge of SQL and experience with the built-in SQL editor in Unified Studio for data querying

. AWS Certified Data Engineer Associate / AWS Certified Machine Learning - Specialty or other relevant AWS certifications.

Required Technical/ Functional Competencies

Domain/ Industry Knowledge:

• Basic knowledge of customer's business processes- relevant technology platform or product.
• Able to prepare process maps, workflows, business cases and simple business models in line with customer requirements with assistance from SME and apply industry standards/ practices in implementation with guidance from experienced team members.

Requirement Gathering and Analysis:

• Working knowledge of requirement management processes and requirement analysis processes, tools and methodologies.
• Able to analyse the impact of change requested / enhancement / defect fix and identify dependencies or interrelationships among requirements and transition requirements for the engagement.

Product/ Technology Knowledge:

• Working knowledge of technology product/platform standards and specifications.
• Able to implement code or configure/customize products and provide inputs in design and architecture adhering to industry standards/ practices in implementation.
• Analyse various frameworks/tools, review the code and provide feedback on improvement opportunities.

Architecture tools and frameworks:

• Basic knowledge of architecture Industry tools & frameworks
• Able to analyse available tools and frameworks for review by the SME and plan for tool configurations and development.

Architecture concepts and principles:

• Basic knowledge of architectural elements, SDLC, methodologies.
• Able to apply various architectural constructs in the projects and identify various architectural patterns and implement.

Analytics Solution Design:

• High-level awareness of a wide range of core data science/analytics techniques, their advantages, disadvantages, and areas of application.

Tools & Platform Knowledge:

• Familiar with wide range of mainstream commercial and open-source data science/analytics software tools, their constraints, advantages, disadvantages, and areas of application.

Required Behavioral Competencies

Accountability:

• Takes responsibility for and ensures accuracy of own work, as well as the work and deadlines of the team.

Collaboration:

• Participates in team activities and reaches out to others in team to achieve common goals.

Agility:

• Demonstrates a willingness to accept and embrace differing ideas or perceptions which are beneficial to the organization.

Customer Focus:

• Displays awareness of customers stated needs and gives priority to meeting and exceeding customer expectations at or above expected quality within stipulated time.

Communication:

• Targets communications for the appropriate audience, clearly articulating and presenting his/her position or decision.

Drives Results:

• Sets realistic stretch goals for self & others to achieve and exceed defined goals/targets.

Certifications