Job Title: Data Engineering Architect (Azure)
Location: India
About Data Management Competency: Focused on Data Governance and Quality Management, establishing and enforcing policies, processes, and practices to ensure the integrity, availability, and reliability of data across the organization.
Note: looking for 30-45 days notice period only. Immediate joiner would be preferred.
Job Responsibilities:
Acting as a Senior Consultant/Solution Advisor :The role involves designing the project architecture. Setup, monitoring Internal Developer Platform. UC migration. Building Databricks Asset Bundles. PR reviews. Data Model / Semantic Layer setup. Model QA automation.
Key Task:
- Collaborate with stakeholders to understand business requirements and translate them into data engineering solutions.
- Design and oversee the overall data architecture and infrastructure, ensuring scalability, performance, security, maintainability, and adherence to industry best practices.
- Define data models and data schemas to meet business needs, considering factors such as data volume, velocity, variety, and veracity.
- Select and integrate appropriate data technologies and tools, such as databases, data lakes, data warehouses, and big data frameworks, to support data processing and analysis.
- Create scalable and efficient data processing frameworks, including ETL (Extract, Transform, Load) processes, data pipelines, and data integration solutions.
- Ensure that data engineering solutions align with the organization's long-term data strategy and goals.
- Evaluate and recommend data governance strategies and practices, including data privacy, security, and compliance measures.
- Collaborate with data scientists, analysts, and other stakeholders to define data requirements and enable effective data analysis and reporting.
- Provide technical guidance and expertise to data engineering teams, promoting best practices and ensuring high-quality deliverables. Support to team throughout the implementation process, answering questions and addressing issues as they arise.
- Oversee the implementation of the solution, ensuring that it is implemented according to the design documents and technical specifications.
- Stay updated with emerging trends and technologies in data engineering, recommending and implementing innovative solutions as appropriate.
- Conduct performance analysis and optimization of data engineering systems, identifying and resolving bottlenecks and inefficiencies.
- Ensure data quality and integrity throughout the data engineering processes, implementing appropriate validation and monitoring mechanisms.
- Collaborate with cross-functional teams to integrate data engineering solutions with other systems and applications.
- Participate in project planning and estimation, providing technical insights and recommendations.
- Document data architecture, infrastructure, and design decisions, ensuring clear and up-to-date documentation for implementation, reference and knowledge sharing.
Must have:
- 10+ years experience in the Data & Analytics area. 4+ years experience into Data Engineering Architecture.
- Strong expertise in Azure cloud services such as: Databricks including Unity Catalog, Asset Bundles and Data Factory, Microsoft Fabric. Logic App, Function App.
- Strong Experience data warehousing, big data technologies, and data lakes. Especially Azure, DataBricks
- Expertise in data modeling, data integration, and ETL processes.
- Programming skills for data analysis (especially Python PySpark, SparkSQL, SQL)
- Experience with data governance, data security, and compliance standards.
- Experience with agile methodologies and the ability to apply DevOps and DataOps principles to data pipelines.
- Ability to use ARM templates
- Certifications preferred.
- Experience in selecting and integrating appropriate technologies to meet business requirements and long-term data strategy..
- Strong analytical and problem-solving skills, with the ability to identify and address complex data engineering challenges.
- Experience in designing and implementing data solutions in a cloud environment.
- Excellent communication and collaboration skills, with the ability to effectively communicate technical concepts to non-technical stakeholders.
- Experience in leading and mentoring data engineering teams, providing guidance and technical expertise.
- Familiarity with agile methodologies and experience in working in agile development environments.
- Continuous learning mindset, staying updated with the latest advancements and trends in data engineering and related technologies.
- Strong project management skills, with the ability to prioritize tasks, manage timelines, and deliver high-quality results within designated deadlines.
- Strong understanding of distributed computing principles, including parallel processing, data partitioning, and fault-tolerance.
We offer:
- Stable employment. On the market since 2008, 1500+ talents currently on board in 7 global sites.
- Office as an option model. You can choose to work remotely or in the office, depending on your location.
- Flexibility regarding working hours and your preferred form of contract.
- Comprehensive online onboarding program with a Buddy from day 1.
- Cooperation with top-tier engineers and experts.
- Unlimited access to the Udemylearning platform from day 1.
- Certificate training programs. Lingarians earn 500+ technology certificates yearly.
- Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.
- Internal Gallup Certified Strengths Coach to support your growth.
- Grow as we grow as a company. 76% of our managers are internal promotions.
- A diverse, inclusive, and values-driven community.
- Autonomy to choose the way you work. We trust your ideas.
- Create our community together. Refer your friends to receive bonuses.
- Activities to support your well-being and health.
- Plenty of opportunities to donate to charities and support the environment.
- Modern office equipment. Purchased for you or available to borrow, depending on your location.
