Data Platform Engineer

• Work as a member of a Data Platform Engineering team that uses Cloud and Big Data technologies to design, develop, implement and maintain solutions to support various functional areas like Manufacturing, Commercial, Research and Development.
• Collaborate with lead architect, Business SMEs, and Data Scientists to architect data solutions
• Serve as lead engineer for technical implementation of projects including planning, architecture, design, development, testing, and deployment in agile setting
• Overall management of the Enterprise Data Lake on AWS environment to ensure that the service delivery is cost effective and business SLAs around uptime, performance and capacity are met
• Help define guidelines, standards, strategies, security policies and change management policies to support the Data Lake
• Proactively work on challenging data integration problems by implementing optimal ETL patterns, frameworks for structured and unstructured data.
• Automate and Optimize data pipeline and framework for easier and cost-effective development process.
• Advice and support project teams (project managers, architects, business analysts, and developers) on cloud platforms (AWS, Databricks preferred), tools, technology, and methodology related to the design, build scalable, efficient and maintain Data Lake and other Big Data solutions
• Experience developing in an Agile development environment, and comfortable with Agile terminology and ceremonies.
• Familiarity with code versioning using GITLAB, and code migration tools.
• Provide mentorship and technical guidance to junior engineers and team members.
• Stay up to date with the latest data technologies and trends

Basic Qualifications and Experience: 

• Master's degree in computer science or engineering field and 1 to 3 years of relevant experience OR
• Bachelor's degree in computer science or engineering field and 3 to 5 years of relevant experience OR
• Diploma and Minimum of 8+ years of relevant work experience

Must-Have Skills:

• Experience with Databricks (or Snowflake), including cluster setup, execution, and tuning
• Experience with common data processing librariesPandas, PySpark, SQLAlchemy.
• Experience in UI frameworks (Angular.js or React.js)
• Experience with data lake, data fabric and data mesh concepts
• Experience with data modeling, performance tuning, and experience on relational databases
• Experience building ETL or ELT pipelines; Hands-on experience with SQL/NoSQL
• Program skills in one or more computer languages – SQL, Python, Java
• Experienced with software engineering best-practices, including but not limited to version control (Git, GitLab.), CI/CD (GitLab, Jenkins etc.), automated unit testing, and Dev Ops
• Exposure to Jira or Jira Align.