Lead Data Engineer

Key Responsibilities:

Solution Architecture & Development:

• Architect and design large enterprise-level analytics solutions, leveraging big data platforms such as Databricks, Snowflake, Hive, Hadoop, and PostgreSQL.
• Develop data pipelines for big data ingestion using tools like Spark, Informatica PowerCenter & BDM, and Apache NiFi/Airflow for continuous data flow automation.
• Work with enterprise systems such as ERPs (SAP, Oracle), RDBMS, CRMs, and logistics tools to create analytics solutions that improve business functions across the organization.
• Ensure scalability, optimization, and automation for all data solutions to meet high-performance and high-availability requirements.
• Develop APIs, microservices, and platform-based solutions for data integration, enabling seamless data interaction across systems and platforms.

Collaboration & Leadership:

• Lead a cross-functional team of data engineers, data architects, data scientists, DevOps, scrum masters, and product owners to build and deploy end-to-end solutions.
• Take ownership of data engineering design and data management best practices, including dimensional modeling, data warehousing, and data mining.
• Collaborate with leadership to understand business needs and propose optimal data-driven solutions.
• Ensure continuous improvement of data processes, focusing on efficiency, automation, and reducing manual work.

Big Data & Cloud Infrastructure:

• Lead the deployment of big data ingestion jobs using Spark and other industry-standard tools on Hadoop, Databricks, and NoSQL platforms.
• Ensure the integration of cloud-based solutions using Azure and AWS platforms, enabling seamless scaling and flexibility.
• Work with IoT data and ensure the integration of data sources for analytics solutions.

Advanced Analytics & Data Science:

• Leverage your expertise to integrate advanced analytics and machine learning capabilities into the platform for predictive and prescriptive analytics.
• Build solutions that harness big data for actionable insights, forecasting, predictive modeling, and other analytics needs.

Process Improvement & Automation:

• Identify opportunities for process optimization and automation, ensuring that all data operations are efficient and reduce the manual effort involved.
• Work towards building a data-driven culture within Honeywell Connected Enterprise, fostering an environment that embraces automation and continuous improvement.

Platform Development:

• Contribute to the creation of a new platform by working on both frontend and backend aspects, including data storage, APIs, microservices, and data orchestration.
• Enable seamless integration between existing and new platforms, ensuring a robust and scalable architecture.

Qualifications:

Must Have:

• Bachelor's/Master's degree in Computer Science, Engineering, Applied Mathematics, or a related field.
• 12+ years of experience in data engineering, data design, and enterprise data management.
• Strong expertise in architecting large-scale analytics projects and big data platforms (Databricks, Snowflake, Hadoop, etc.).
• 4+ years of hands-on experience with tools like Spark, Informatica PowerCenter & BDM, Apache NiFi/Airflow, and cloud platforms like Azure and AWS.
• Experience in dimensional modeling, data warehousing, and data mining.
• Strong knowledge of big data ingestion, processing, and optimization, with a focus on scalability and performance.
• Expertise in cloud-based data solutions (Azure, AWS), with experience in developing, deploying, and managing cloud data pipelines.
• Proven experience in data integration, building APIs, and working with microservices architectures.

Nice to Have:

• Experience with industrial data (IoT, SAP, and other operational systems).
• Proficiency in data science techniques, including predictive modeling, machine learning, and AI-based analytics.
• Knowledge of advanced data analytics platforms such as Power BI or Tableau.
• Ability to work with non-relational databases (Cassandra, NoSQL, etc.) and experience in advanced query optimization.