Senior Principal Consultant - Databricks Data Architect

0-2 years
2 months ago 11 Applied
Job Description

With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world&rsquos biggest brands&mdashand we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We&rsquore harnessing the power of technology and humanity to create meaningful transformation that moves us forward in our pursuit of a world that works better for people. Now, we&rsquore calling upon the thinkers and doers, those with a natural curiosity and a hunger to keep learning, keep growing., People who thrive on fearlessly experimenting, seizing opportunities, and pushing boundaries to turn our vision into reality. And as you help us create a better world, we will help you build your own intellectual firepower.

Welcome to the relentless pursuit of better.

Inviting applications for the role of Senior Principal Consultant-Databricks Data Architect

In this role,theDatabricks Architectis responsible forprovidingtechnical direction andleada group of one or more developer to address a goal.


  • Architect and design solutions to meet functional and non-functionalrequirements.

  • Create and review architecture and solution designartifacts.

  • Evangelize re-use through the implementation of sharedassets.

  • Enforce adherence to architectural standards/principles, global product-specific guidelines, usability design standards, etc.

  • Proactively guide engineering methodologies, standards, and leading practices.

  • Guidance of engineering staff and reviews of as-built configurations during the construction phase.

  • Provide insight and direction on roles and responsibilities required for solutionoperations.

  • Identify, communicate and mitigate Risks, Assumptions, Issues, and Decisions throughout the fulllifecycle.

  • Considers the art of the possible, compares various architectural options based on feasibility and impact, and proposes actionableplans.

  • Demonstratestrong analytical and technical problem-solvingskills.

  • Ability to analyze and operate at various levels ofabstraction.

  • Ability to balance what is strategically right with what is practicallyrealistic.

  • Growing the Data Engineering business by helping customersidentifyopportunities to deliver improved business outcomes, designing and driving the implementation of those solutions.

  • Growing &retainingthe Data Engineering team withappropriate skillsand experience to deliver high quality services to ourcustomers.

  • Supporting and developing our people, including learning & development, certification & career development plans

  • Providingtechnical governance and oversight for solution design and implementation

  • Should have technical foresight to understand new technology and advancement.

  • Leading team in the definition of best practices & repeatable methodologies in Cloud Data Engineering, including Data Storage, ETL, Data Integration & Migration, Data Warehousing and Data Governance

  • Should have Technical Experience inAzure,AWS & GCP Cloud Data Engineering services andsolutions.

  • Contributing to Sales & Pre-sales activities including proposals, pursuits,demonstrations,and proof of concept initiatives

  • Evangelizing the Data Engineering service offerings to both internal and external stakeholders

  • Development of Whitepapers, blogs, webinars and other though leadership material

  • Development of Go-to-Market and Service Offering definitions for Data Engineering

  • Working with Learning & Development teams toestablishappropriate learning & certificationpaths for theirdomain.

  • Expand the business within existing accounts and help clients, by building and sustaining strategic executive relationships, doubling up as their trusted business technologyadvisor.

  • Position differentiated and custom solutions to clients, based on the market trends, specific needs of the clients and the supporting businesscases.

  • Build new Data capabilities, solutions, assets,accelerators,and teamcompetencies.

  • Manage multiple opportunities through the entire business cycle simultaneously, working with cross-functionalteamsasnecessary.

Qualifications we seek in you!

Minimum qualifications

  • Excellent technical architecture skills, enabling the creation of future-proof, complex globalsolutions.

  • Excellent interpersonal communication and organizational skills are required to operate as a leading member of global, distributed teams that deliver quality services and solutions.

  • Ability to rapidly gain knowledge of the organizational structure of the firm to facilitate work with groups outside of the immediate technical team.

  • Knowledge and experience in IT methodologies and life cycles that will be used.

  • Familiar with solution implementation/management, service/operations management, etc.

  • Leadership skills can inspire others andpersuade.

  • Maintains close awareness of new and emerging technologies and their potential application for service offerings and products.

  • Bachelor&rsquos Degree or equivalency (CS, CE, CIS, IS, MIS, or engineering discipline) or equivalent workexperience

  • Experience in a solution architecture role using service and hosting solutions such as private/public cloud IaaS, PaaS, and SaaS platforms.

  • Experience in architecting and designing technical solutions for cloud-centric solutions based on industry standards using IaaS, PaaS, and SaaS capabilities.

  • Must have strong hands-on experience on various cloud services like ADF/Lambda, ADLS/S3, Security, Monitoring, Governance

  • Must have experience to design platform on Databricks.

  • Hands-on Experience to design and build Databricks based solution on any cloud platform.

  • Hands-on experience to design and build solution powered by DBT models and integrate with databricks.

  • Must be very good designing End-to-End solution on cloud platform.

  • Must have good knowledge of Data Engineering concept and related services of cloud.

  • Must have good experience in Python and Spark.

  • Must have good experience in setting up development best practices.

  • Intermediate level knowledge is required for Data Modelling.

  • Good to have knowledge of docker and Kubernetes.

  • Experience with claims-based authentication (SAML/OAuth/OIDC), MFA,RBAC, SSO etc.

  • Knowledge of cloud security controls including tenant isolation, encryption at rest, encryption in transit, key management, vulnerability assessments, application firewalls, SIEM, etc.

  • Experience building and supporting mission-critical technology components with DRcapabilities.

  • Experience with multi-tier system and service design and development for large enterprises

  • Extensive, real-world experience designing technology components for enterprise solutions and defining solution architectures and reference architectures with a focus on cloudtechnologies.

  • Exposure to infrastructure and application security technologies and approaches

  • Familiarity with requirements gathering techniques.

Preferred qualifications

  • Must have designed the E2E architecture of unified data platform covering all the aspect of data lifecycle starting from Data Ingestion, Transformation, Serve and consumption.

  • Must have excellent coding skills either Python or Scala, preferably Python.

  • Must have experience in Data Engineering domain.

  • Must have designed and implemented at least 2-3 project end-to-end in Databricks.

  • Must have experience on databricks which consists of various components as below

    • Delta lake

    • dbConnect

    • db API 2.0

    • SQL Endpoint - Photon engine

    • Unity Catalog

    • Databricks workflows orchestration

    • Security management

    • Platform governance

    • Data Security

  • Must have knowledge of new features available in Databricks and its implications along with various possible use-case.

  • Must have followed various architectural principles to design best suited per problem.

  • Must be well versed with Databricks Lakehouse concept and its implementation in enterprise environments.

  • Must have strong understanding of Data warehousing and various governance and security standards around Databricks.

  • Must have knowledge of cluster optimization and its integration with various cloud services.

  • Must have good understanding to create complex data pipeline.

  • Must be strong in SQL and sprak-sql.

  • Must have strong performance optimization skills to improve efficiency and reduce cost.

  • Must have worked on designing both Batch and streaming data pipeline.

  • Must have extensive knowledge of Spark and Hive data processing framework.

  • Must have worked on any cloud (Azure, AWS, GCP) and most common services like ADLS/S3, ADF/Lambda, CosmosDB/DynamoDB, ASB/SQS, Cloud databases.

  • Must be strong in writing unit test case and integration test.

  • Must have strong communication skills and have worked with cross platform team.

  • Must have great attitude towards learning new skills and upskilling the existing skills.

  • Responsible to set best practices around Databricks CI/CD.

  • Must understand composable architecture to take fullest advantage of Databricks capabilities.

  • Good to have Rest API knowledge.

  • Good to have understanding around cost distribution.

  • Good to have if worked on migration project to build Unified data platform.

  • Good to have knowledge of DBT.

  • Experience around DevSecOps including docker and Kubernetes.

  • Software development full lifecycle methodologies, patterns, frameworks, libraries, and tools

  • Knowledge of programming and scripting languages such as JavaScript, PowerShell, Bash, SQL,Java, Python, etc.

  • Experience with data ingestion technologies such as Azure Data Factory, SSIS, Pentaho, Alteryx

  • Experience with visualization tools such as Tableau, Power BI

  • Experience with machine learning tools such as mlFlow, Databricks AI/ML, Azure ML, AWS sagemaker, etc.

  • Experience in distilling complex technical challenges to actionable decisions for stakeholders and guiding project teams by building consensus and mediating compromises when necessary.

  • Experience coordinating the intersection of complex system dependencies andinteractions

  • Experience in solution delivery using common methodologies especiallySAFeAgile but also Waterfall, Iterative, etc.

  • Demonstrated knowledge of relevant industry trends andstandards

Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values diversity and inclusion, respect and integrity, customer focus, and innovation. For more information, visit . Follow us on , , , and .

Furthermore, please do note that Genpact does not charge fees to process job applications and applicants are not required to pay to participate in our hiring process in any other way. Examples of such scams include purchasing a %27starter kit,%27 paying to apply, or purchasing equipment or training.




Databricks AI/ML
AWS sagemaker
Delta lake
db API 2.0
Photon engine
Unity Catalog
Job Source:

Genpact is an American professional services firm legally domiciled in Bermuda with its headquarters in New York City, New York. The company currently employs more than 100,000 people and provides services to clients in over 30 countries worldwide

People Also Considered

Career Advice to Find Better