Search by job, company or skills

Lead Data Engineer

Foundation AI

    Highlights

    Job Description

    More Info

    Recruiter Info

5-7 Years
25 days ago
48 Viewed
0 Applied

Job Description

Responsibilities

  • Architect and Design Data Systems: Design and develop scalable data pipelines and systems to manage large volumes of structured and unstructured data.
  • Data Pipeline Development: Develop scalable end-to-end ETL (Extract, Transform, Load) data pipelines to ingest data from various diverse sources, perform transformations, and load data into target systems to meet both functional and non-functional requirements.
  • Data Modeling and Optimization: Develop and optimize data models (including relational Data) for efficient storage and retrieval, ensuring data quality and integrity.
  • Technical Oversight: Manage technical scope, ensuring alignment with business objectives and delivering high-quality solutions. Provided required technical guidance and oversight to various cross-functional teams and team members on databases like PostgreSQL/ Vector DB and various queries, functions, and datasets.
  • Configuration Management: Manage the configurations for various tools and databases like postgresDB, vectorDB, etc. for both on-premises and cloud platforms for optimized performance and redundancy
  • Performance Tuning: Monitor and optimize data pipelines, SQL/queries, functions, procedures, and triggers for performance and scalability.
  • Stakeholder Collaboration: Work closely with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and deliver solutions ensuring alignment with organizational goals.
  • Promoting Best Practices: Define and promote reusable, scalable, and maintainable solutions, emphasizing software engineering best practices and continuous improvement.
  • Communication: Communicate effectively at all levels about the importance of solution design, conducting technical walk-throughs to ensure a clear understanding of system architecture.
  • Continuous Improvement: Work with data and analytics experts to enhance functionality in data systems and products, driving improvements and growth within the data team.
  • Cloud Experience: Experience of working with AWS RDS databases is preferred

Required Experience, Skills & Qualifications

  • Experience: 5-7 years in High-Performance Data Products or Data Systems as a Data Architect/Engineer.
  • Technical Skills: Proficiency in orchestration tools like Airflow.
  • Proficiency in SQL and relational databases (e.g., PostgreSQL, Aurora) and NoSQL databases (e.g., MongoDB, Cassandra).
  • Good knowledge of Vector database like pgvector etc.
  • Hands-on experience with cloud platforms (e.g., AWS, Azure, GCP) and containerization technologies (e.g., Docker, Kubernetes).
  • Experience with agile development methodologies (e.g., Scrum, Kanban).
  • Software Engineering: Proficient in software engineering best practices, unit testing, integration testing, and tools like Git, Maven, and Docker.
  • Security and Compliance: Familiarity with security compliances and design practices.
  • Communication: Exceptional interpersonal, analytical, and communication skills. Ability to explain and discuss concepts with colleagues and teams effectively.
  • CI/CD Pipeline: Fully adhere to and promote an entire CI/CD pipeline.
  • API Development: Familiarity with API development and data formats like JSON/XML.

Desired Experience, Skills & Qualifications

  • Domain Experience: Experience in the Insurance/Legal domain is a plus.
  • Regulatory Knowledge: Familiarity with security and privacy regulations such as GDPR and HIPAA.
  • Collaboration: Proven ability to collaborate effectively with cross-functional teams in a fast-paced environment.
  • Problem Solving: Demonstrated ability to conduct root cause analyses and drive continuous improvement initiatives.

Interview Process

  • Application Review: Your application will be thoroughly reviewed by our hiring team to assess your qualifications and experience.
  • Technical Interview: If shortlisted, you'll have an initial interview with our interviewing partner Bar raiser.This is an opportunity for us to get to know you better and for you to learn more about the role and the company. The panel will also ask you technical questions to evaluate your technical skills and problem-solving abilities
  • Technical Interview 2: You will have a technical discussion with the internal panel to evaluate your technical skills and problem-solving abilities.
  • On-Site Interview*: Depending on the circumstances, an on-site interview might be conducted further to assess your fit for the role and team culture. (For Candidates based in Hyderabad)
  • Final Interview: A final interview with senior leadership to discuss your background, accomplishments, and alignment with Foundation AI's goals and values.
  • Offer: Successful candidates will receive a formal proposal outlining compensation, benefits, and other details.
Follow
Save
Report

Similar Jobs

LEAD ENGINEER KENVUE DATA PLATFORMS

Company Name Confidential

People also considered

DelhiBengaluru / BangaloreNoidaMumbaiHyderabad / Secunderabad Telangana
Last Updated: 20-06-2024 11:43:04 AM
Beware of Scammers

We don’t charge money for job offers