Tech Lead - AWS Job

Experience

8+ Years

Job Description

Key Responsibilities:

Data Engineering:

.Data LakeHouse:

o Simplify analytics and AI with unified data management across Amazon S3 data lakes, Amazon Redshift data warehouses, operational databases, and enterprise applications.

.Data Processing:

o Use Visual ETL jobs and unified notebooks to perform data analysis, preparation, and integration with data sources like Amazon S3, Redshift, and AWS RDS.

.Orchestration and Automation:

o Create and manage SageMaker Pipelines and Workflows to automate the ETL Pipelines, ML model continuous integration and continuous delivery (CI/CD) lifecycle.

.Data Governance and Collaboration:

o Work within collaborative projects in SageMaker Unified Studio, leveraging the SageMaker Catalog (built on Amazon DataZone) to securely share data and AI artifacts.

o Implement various features in the project like Lineage, Data Quality, Metadata forms, Business Glossaries etc.

AI/ML:

.End-to-End MLOps:

o Build, deploy, execute, and monitor complete data and AI workflows using SageMaker Unified Studio's integrated environment.

.Model Development:

o Develop, train, and evaluate ML and generative AI models using SageMaker's purpose-built tools and algorithms.

o Utilize advanced features such as HyperPod for distributed training, JumpStart for pre-trained models, and SageMaker Experiments for tracking model versions.

.Generative AI Development:

o Rapidly build and customize generative AI applications within the integrated Amazon Bedrock environment in Unified Studio.

o Implement advanced customization capabilities like Knowledge Bases, Guardrails, Agents, and Flows.

.Security and Compliance:

o Implement and maintain security best practices, including fine-grained access controls and data protection, within the SageMaker environment.

.Performance Optimization:

o Optimize the performance and cost of ML workloads by right-sizing compute resources and monitoring job metrics.

.Troubleshooting:

o Utilize Amazon Q Developer's generative AI-powered assistance for code generation and resolving issues within the notebooks and jobs.

Required Skills:

• Experience with data processing and analytics using services integrated with Sagemaker Unified Studio (e.g., AWS Glue, Amazon EMR, Amazon Athena, AWS Redshift).
• Demonstrable expertise with Amazon SageMaker Unified Studio and its core components (Domain Units, Projects, Data Assets, Jobs, Data Products).
• Proficiency in PySpark, Python and relevant ML/AI libraries such as TensorFlow, PyTorch, and Scikit-learn.
• Experience with generative AI application development using Amazon Bedrock services within Unified Studio.
• Proven experience developing machine learning models and data pipelines on AWS.
• Solid understanding of MLOps principles and experience building CI/CD pipelines for ML models.
• Familiarity with AWS security best practices, including IAM roles, policies, and data governance, AWS Lake Formation
• Knowledge of SQL and experience with the built-in SQL editor in Unified Studio for data querying
• AWS Certified Data Engineer Associate / AWS Certified Machine Learning - Specialty or other relevant AWS certifications.

Required Technical/ Functional Competencies

Domain/ Industry Knowledge:

• Basic knowledge of customer's business processes- relevant technology platform or product.
• Able to prepare process maps, workflows, business cases and simple business models in line with customer requirements with assistance from SME and apply industry standards/ practices in implementation with guidance from experienced team members.

Requirement Gathering and Analysis:

• Working knowledge of requirement management processes and requirement analysis processes, tools & methodologies.
• Able to analyse the impact of change requested/ enhancement/ defect fix and identify dependencies or interrelationships among requirements & transition requirements for engagement.

Product/ Technology Knowledge:

• Working knowledge of technology product/platform standards and specifications.
• Able to implement code or configure/customize products and provide inputs in design and architecture adhering to industry standards/ practices in implementation.
• Analyze various frameworks/tools, review the code and provide feedback on improvement opportunities.

Architecture tools and frameworks:

• Working knowledge of architecture Industry tools & frameworks.
• Able to identify pros/ cons of available tools & frameworks in market and use those as per Customer requirement and explore new tools/ framework for implementation.

Architecture concepts and principles:

• Working knowledge of architectural elements, SDLC, methodologies.
• Able to provides architectural design/ documentation at an application or function capability level and implement architectural patterns in solution & engagements and communicates architecture direction to the business.

Analytics Solution Design:

• Knowledge of statistical & machine learning techniques like classification, linear regression modelling, clustering & decision trees.
• Able to identify the cause of errors and their potential solutions.

Tools & Platform Knowledge:

• Familiar with wide range of mainstream commercial & open-source data science/analytics software tools, their constraints, advantages, disadvantages, and areas of application.

Required Behavioral Competencies

Accountability:

• Takes responsibility for and ensures accuracy of own work, as well as the work and deadlines of the team.

Collaboration:

• Shares information within team, participates in team activities, asks questions to understand other points of view.

Agility:

• Demonstrates readiness for change, asking questions and determining how changes could impact own work.

Customer Focus:

• Identifies trends and patterns emerging from customer preferences and works towards customizing/ refining existing services to exceed customer needs and expectations.

Communication:

• Targets communications for the appropriate audience, clearly articulating and presenting his/her position or decision.

Drives Results:

• Sets realistic stretch goals for self & others to achieve and exceed defined goals/targets.

Resolves Conflict:

• Displays sensitivity in interactions and strives to understand others views and concerns.

Certifications