Responsibilities
- Analyze the business requirements and needs of the organization.
- Architect and design robust, scalable, and secure enterprise data platform leveraging Databricks and various cloud services on Microsoft Azure.
- Lead implementation of advanced data analytics solutions utilizing Databricks and Delta Lake architectures to extract, transform, and load (ETL) data.
- Leverage Databricks to perform data engineering, artificial intelligence and machine learning tasks.
- Collect, process, and analyze large datasets from various sources within the aviation domain (e.g., flight data, baggage information, weather data, operational logs etc.).
- Perform data quality checks and ensure data accuracy and consistency.
- Identify trends, patterns, and anomalies in data to provide valuable insights and support business decision-making.
- Contribute to the development of data models and database designs.
- Document data sources, data flows, and analytical processes.
- Stay up to date with the latest trends and technologies in data analysis and the aviation industry.
- Create insightful and interactive data visualizations and reports using tools like Power BI to communicate findings to non-technical audiences.
- Define and govern enterprise-wide cloud architecture standards, security protocols, and operational best practices.
- Developing and implementing strategies to connect different systems, ensuring they can share data and functionality
- Defining and managing API that allow different software applications to communicate with each other
- Defining and managing asynchronous data exchange between applications and as a part of data pipelines
- Designing real-time streaming pipelines for IoT time-series data
- Diving deep into the details to solve complex technical challenges
- Ensure the platform is secure and adheres to best practices for data protection and privacy.
- Ensuring high data quality and integrity to avoid operational failures and improve system accuracy
- Perform regular audits and maintenance of the platform to ensure its continued performance and stability.
- Support project management activities, resource monitoring, technical risk identification & mitigations
- Mentor and provide technical leadership to platform engineering and data engineering teams.
- Ensure continuous platform optimization, cost management, and high availability of critical analytics infrastructure.
- Partner with business stakeholders to translate airport operational requirements into technical blueprints and scalable solutions.
Qualifications
Required Qualifications & Experience
- Microsoft Certified: Azure Solutions Architect Expert or equivalent advanced architectural certification.
- Databricks Certified Data Engineer Professional or similar advanced credential in big data ecosystems.
- Proven track record of designing large-scale digital platforms in complex operational environments, preferably within aviation or large infrastructure.
Educational Background
- Bachelor's or Master's degree in Computer Science, Information Technology, or a related technical discipline.
- Advanced certifications in Enterprise Architecture frameworks such as TOGAF are highly preferred.
Years & Type of Experience
- 15 to 20 years of progressive experience in IT, with at least 5+ years in a senior architectural capacity.
- Demonstrated experience leading large-scale data engineering and analytics transformation programs utilizing Microsoft Azure and Databricks.
Domain Expertise
- Deep functional knowledge of cloud computing paradigms (IaaS, PaaS, SaaS) and distributed data systems.
- Strong understanding of big data ecosystems, data warehousing, and modern data mesh architectures.
- Familiarity with the aviation or large-scale infrastructure domain, including passenger processing, non-aero revenue analytics, and airport operations.
Digital & Operational Tools
- Cloud Platforms: Advanced proficiency in Microsoft Azure services (Azure Data Factory, Azure Event Hub, Azure Kubernetes Service, Azure IoT Hub etc).
- Data & Analytics: Expertise in Databricks, Apache Spark, Delta Lake, and enterprise Business Intelligence platforms.
- Proficiency in one programming languages such as JavaScript/Node.js, Java, Python, etc.
- Expertise in databases like PostgreSQL, MS SQL, and MongoDB.
- Expertise in APIs, ESB (Enterprise Service Bus), and Kafka.
- Integration with IoT platforms (Azure IoT Hub), device protocols (MQTT, AMQP, HTTP)
- Experience of data analytics, business intelligence, and AI/ML technologies
- Use of AI tools for code generation, pipeline automation, testing
- Knowledge of Azure DevOps, Docker, Kubernetes, and enterprise source control systems.
- Understanding of machine learning fundamentals (supervised, unsupervised, feature engineering)
- Experience supporting ML model lifecycle (training data, inference pipelines)
- Exposure to LLMs, GenAI pipelines, and vector databases
- Knowledge of feature engineering and model-ready datasets
- Knowledge of MLOps tools (MLflow, Kubeflow, Airflow, etc.)
- Building CI/CD pipelines for data + ML workflows
- Experience of Agile methodologies, Jira, and Confluence.
- Knowledge of cybersecurity principles and practices.
Leadership Capabilities
- Ability to drive technical consensus and influence technology strategy across diverse stakeholder groups and executive leadership.
- Strategic thinker with a demonstrated capability to align IT and platform roadmaps with long-term business objectives.
- Proven track record of mentoring senior engineers and fostering a culture of technical excellence and innovation.
- Capable of managing complex, multi-million dollar technology initiatives and strategic vendor relationships.
