Data Catalogue

Role & responsibilities

• Lead the end-to-end implementation of a data cataloging solution within AWS (preferably AWS Glue Data Catalog or third-party tools like Apache Atlas, Alation, Collibra, etc.).
• Establish and manage metadata frameworks for structured and unstructured data assets in the data lake and data warehouse environments.
• Integrate the data catalog with AWS-based storage solutions such as S3, Redshift, Athena, Glue, and EMR.
• Collaborate with data Governance/BPRG/IT projects teams to define metadata standards, data classifications, and stewardship processes.
• Develop automation scripts for catalog ingestion, lineage tracking, and metadata updates using Python, Lambda, Pyspark or Glue/EMR customs jobs.
• Work closely with data engineers, data architects, and analysts to ensure metadata is accurate, relevant, and up to date.
• Implement role-based access controls and ensure compliance with data privacy and regulatory standards.
• Create detailed documentation and deliver training/workshops for internal stakeholders on using the data catalog.

Preferred candidate profile

• AWS Certifications (e.g., AWS Certified Data Analytics, AWS Solutions Architect).
• Experience with data catalog tools like Alation, Collibra, or Informatica EDC. Or open sources tools hand-on experience.
• Exposure to data quality frameworks and stewardship practices.
• Knowledge of data migration with data catalog and data-mart is plus.
• 4 to 8+ years of experience in data engineering or metadata management roles.
• Proven expertise in implementing and managing data catalog solutions wiithin AWS environments.
• Strong knowledge of AWS Glue, S3, Athena, Redshift, EMR, Data Catalog and Lake Formation.
• Hands-on experience with metadata ingestion, data lineage, and classification processes.
• Proficiency in Python, SQL, and automation scripting for metadata pipelines.
• Familiarity with data governance and compliance standards (e.g., GDPR, RBI guidelines).
• Experience integrating with BI tools (e.g., Tableau, Power BI) and third-party catalog tools is a plus.
• Strong communication,
• Problem-solving, and stakeholder management skills.