Introduction
A career in IBM Consulting is built on long-term client relationships and close collaboration worldwide. You'll work with leading companies across industries, helping them shape their hybrid cloud and AI journeys. With support from our strategic partners, robust IBM technology, and Red Hat, you'll have the tools to drive meaningful change and accelerate client impact. At IBM Consulting, curiosity fuels success. You'll be encouraged to challenge the norm, explore new ideas, and create innovative solutions that deliver real results. Our culture of growth and empathy focuses on your long-term career development while valuing your unique skills and experiences.
Your Role And Responsibilities
As an Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing.
Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets.
In This Role, Your Responsibilities May Include
- Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporating a variety of statistical and machine learning techniques
- Designing and implementing various enterprise seach applications such as Elasticsearch and Splunk for client requirements
- Work in an Agile, collaborative environment, partnering with other scientists, engineers, consultants and database administrators of all backgrounds and disciplines to bring analytical rigor and statistical methods to the challenges of predicting behaviors.
- Build teams or writing programs to cleanse and integrate data in an efficient and reusable manner, developing predictive or prescriptive models, and evaluating modeling results
Preferred Education
Master's Degree
Required Technical And Professional Expertise
- 6 - 8 Years of total experience in IT/Data Engineering projects & 4 years of relevant experience.
- Strong coding skills in - Python, SQL, ADF, Databricks (PySpark, Delta live table, Live/structural data streaming, Unity Catalog, Performance Tuning of DLT pipelines with high volume of data processing , Cost Optimization of Databricks when DLT runs in live streaming on Photon, Serverless, non serverless)Azure Functions, Logic Apps
- The DE role are expected to have AI, Gen AI to be efficient and innovation in developement.
- Using Gen AI in developing automated data pipelines for faster delivery , Can a data engineer use agents or LLM to build data pipelines
- The candidate should be responsible, take ownership, self driven, dedication to achieve timeline.
- Should have great problem solving and analytical/reasoning capability, technical skill with good mindset and behavior is important good hands-on programming / scripting lnowledge in Python,
- Very strong skilled resources only with good communication-knowledge in Power BI
- Strong MS SQL, Azure Databricks experience
- Implement and manage data models in DBT, data transformation and alignment with business requirements.
- Ingest raw, unstructured data into structured datasets to cloud object store.
- Utilize DBT to convert raw, unstructured data into structured datasets, enabling efficient analysis and reporting.
- Write and optimize SQL queries within DBT to enhance data transformation processes and improve overall performance
- We are seeking a skilled Azure Data Engineer with 5+ years of experience
- Including 3+ years of hands-on experience with ADF/Databricks
- The ideal candidate Data bricks,Data Lake, Phyton programming skills.
- The candidate will also have experience for deploying to data bricks.
- Familiarity with Azure Data Factory
Preferred Technical And Professional Experience
- Good communication skills.
- 3+ years of experience with ADF/DB/DataLake.
- Ability to communicate results to technical and non-technical audiences
