About Us
Website: https://www.cognitioanalytics.com/
Cognitio Analytics, founded in 2013, aims to be the preferred provider of AI / ML driven productivity solutions for large enterprises. The company has received awards for its Smart Operations and Total Rewards Analytics Solutions and is dedicated to innovation, R&D, and creating sustained value for clients. Cognitio Analytics has been recognized as a Great Place to Work for its commitment to fostering an innovative work environment and employee satisfaction.
Our solutions include Total Rewards Analytics powered by Cognitio's Total Rewards Data Factory, The Total Rewards Analytics solutions helps our clients achieve better outcomes and higher ROI on investments in all kinds of Total Rewards programs.
Our smart operations solutions drive productivity in complex operations, such as claims processing, commercial underwriting etc. These solutions, based on proprietary capabilities based on AI, advanced process and task mining, and deep understanding of operations drive effective digital transformation for our clients.
Relevant Experience: 8-12 years
Location: Gurgaon
Job Description
- Lead the data engineering team by driving the design, build, test, and launch of new data pipelines and models on the production data platform
- Define and implement the processes needed to achieve operational excellence in all areas, including agile development, and data solutions
- Oversee the design, development and maintenance of data infrastructure, including data warehouses, data lakes, data integration components supporting optimal extraction, transformation, and loading of data
- Leading the communication with vendors, clients, the leadership team, and other stakeholders to facilitate effective project management and provide ongoing support
- Collaborate with business owners for roadmap planning and prioritization, to deliver robust cloud-based data solutions for our customers
- Work closely with cross-functional teams to propel and execute necessary solution enhancements and provide support for existing solutions
- Define and enforce best practices for data engineering, data modeling, and data quality to ensure accuracy, reliability, reusability, security and consistency of data
- Evaluate and implement new technologies, tools, and frameworks to improve the performance and efficiency of the data engineering team
- Lead the planning, prioritization, and execution and tracking of data engineering projects, ensuring timely delivery and alignment with business objectives
- Establish and monitor key performance indicators (KPIs) to measure the performance and effectiveness of the data engineering team
- Stay current with industry trends and advancements in data engineering, Azure Cloud, and provide strategic guidance for adopting new technologies or methodologies
Qualifications
- 8-12 years of demonstrable experience in data engineering, analytics, data warehousing, data management, Data governance and Compliance Requirements.
- Experience managing data engineers and guiding a team of engineers through project planning, execution, tracking and monitoring, and quality control stages
- Solid experience with cloud-based data tools and platforms
- Proficient in implementing efficient cost management strategies, particularly about storage and computational expenses.
- Experience building processes supporting data transformation, data structures, metadata, security, governance, and workload management.
- Experience supporting and working with cross-functional teams in a dynamic environment
- Expertise in designing and optimizing data models, RDBMS, NoSQL DBs, and data warehousing solutions
- Excellent leadership, communication, and interpersonal skills, with the ability to effectively collaborate with cross-functional teams
- Proven ability to drive innovation, make strategic decisions and lead complex data engineering initiatives to successful completion
Skill Required
Must Haves:
- SQL, Python, Py Spark, Spark SQL, Spark, Distributed Systems
- Databricks, ADLS Gen 2 Blob Storage, Azure DevOps, Azure Data Factory
- ETL, Building Data Pipelines, Data Warehousing, Datamart, Data Modelling and Governance
- Agile Practices, SDLC, DevOps practices, and CI/CD pipelines for data engineering workflows
- Solid understanding of the Microsoft Azure stack for large-scale data engineering developments and deployments is highly preferred
- Hands-on experience with Azure Databricks, including data ingestion, data transformation, workflow management and optimization, monitoring and troubleshooting Spark Jobs
- Ability to set up and manage Azure Data Lake Storage (ADLS) Gen 2 accounts, and familiarity with data lake architecture and best practices
- Familiarity with big data frameworks (e.g., Apache Spark).
- Knowledge of Azure Key Vaults for securely storing and managing cryptographic keys, secrets, and certificates
Good To Have
- Event Hubs for log management
- Workflow Orchestration
- Cosmos DB
- Power BI
- Professional Services Background
- Scala, Java
