Work as a member of a Data Platform Engineering team that uses Cloud and Big Data technologies to design, develop, implement and maintain solutions to support various functional areas like Manufacturing, Commercial, Research and Development.
Collaborate with lead architect, Business SMEs, and Data Scientists to architect data solutions
Serve as lead engineer for technical implementation of projects including planning, architecture, design, development, testing, and deployment in agile setting
Overall management of the Enterprise Data Lake on AWS environment to ensure that the service delivery is cost effective and business SLAs around uptime, performance and capacity are met
Help define guidelines, standards, strategies, security policies and change management policies to support the Data Lake
Proactively work on challenging data integration problems by implementing optimal ETL patterns, frameworks for structured and unstructured data.
Automate and Optimize data pipeline and framework for easier and cost-effective development process.
Advice and support project teams (project managers, architects, business analysts, and developers) on cloud platforms (AWS, Databricks preferred), tools, technology, and methodology related to the design, build scalable, efficient and maintain Data Lake and other Big Data solutions
Experience developing in an Agile development environment, and comfortable with Agile terminology and ceremonies.
Familiarity with code versioning using GITLAB, and code migration tools.
Provide mentorship and technical guidance to junior engineers and team members.
Stay up to date with the latest data technologies and trends

Basic Qualifications and Experience:

Master's degree in computer science or engineering field and 1 to 3 years of relevant experience OR
Bachelor's degree in computer science or engineering field and 3 to 5 years of relevant experience OR
Diploma and Minimum of 8+ years of relevant work experience

Must-Have Skills:

Experience with Databricks (or Snowflake), including cluster setup, execution, and tuning
Experience with common data processing librariesPandas, PySpark, SQLAlchemy.
Experience in UI frameworks (Angular.js or React.js)
Experience with data lake, data fabric and data mesh concepts
Experience with data modeling, performance tuning, and experience on relational databases
Experience building ETL or ELT pipelines; Hands-on experience with SQL/NoSQL
Program skills in one or more computer languages – SQL, Python, Java
Experienced with software engineering best-practices, including but not limited to version control (Git, GitLab.), CI/CD (GitLab, Jenkins etc.), automated unit testing, and Dev Ops
Exposure to Jira or Jira Align.

Data Platform Engineer

Data Platform Engineer