Designation: Senior Data Engineer
Preferred Experience: 7+ years
Responsibilities:
- Design, develop, and maintain data pipelines for ingesting, processing, and transforming data from various sources into actionable insights.
- Integrate data from disparate sources (databases, APIs, and files) into a unified data platform using ETL processes and data integration techniques.
- Design and implement data models, schemas, and data structures to support analytical queries, reporting, and business intelligence needs.
- Optimize database performance, query execution, and data processing workflows for efficiency, scalability, and reliability.
- Ensure data quality, integrity, and consistency through data validation, cleansing, deduplication, and error-handling mechanisms.
- Architect and implement data solutions on Azure cloud platforms, leveraging Azure services for data storage, processing, and analytics.
- Implement data security measures, encryption techniques, and access controls to protect sensitive data and ensure compliance with regulations (e.g., GDPR, HIPAA).
- Work closely with cross-functional teams, data scientists, analysts, and stakeholders to understand data requirements, provide data solutions, and communicate insights effectively.
- Document data processes, workflows, and best practices, and promote data governance standards, data lineage, and metadata management.
- Stay updated with emerging technologies, industry trends, and best practices in data engineering, cloud computing, and data analytics to drive innovation and continuous improvement.
Required Skills:
- Proficiency in writing complex SQL queries, stored procedures, and functions for data extraction, transformation, and analysis.
- Experience in database design, optimization, and management using SQL Server/Azure SQL Database.
- Knowledge of data modeling techniques, including entity-relationship diagrams, dimensional modeling, and data normalization, is needed to design efficient data structures.
- Familiarity with ETL (Extract, Transform, Load) processes and tools such as Azure Data Factory, SSIS (SQL Server Integration Services), or other data integration platforms.
- Hands-on experience with Azure cloud services, including Azure SQL Database, Azure Data Factory and Azure Storage.
- Experience handling large-scale data processing and analytics with strong analytical skills and attention to detail.
- Strong statistical knowledge.
- Excellent communication skills.
Good To Have Skills:
- Experience working with C#, .Net Framework
- Experience with other Azure cloud services and sharepoint
- Experience with Python as programming language
- Experience manipulating unstructured data with regular expressions.
- Basic understanding of machine learning concepts and algorithms for data mining, predictive modeling, and statistical analysis.
- Knowledge of data warehousing concepts, methodologies, and tools for building and maintaining data warehouses or data marts.
Company Overview:
Aventior is a leading provider of innovative technology solutions for businesses across a wide range of industries. At Aventior, we leverage cutting-edge technologies like AI, ML Ops, DevOps, and many more to help our clients solve complex business problems and drive growth.
We also provide a full range of data development and management services, including Cloud Data Architecture, Universal Data Models, Data Transformation & and ETL, Data Lakes, User Management, Analytics and visualization, and automated data capture (for scanned documents and unstructured/semi-structured data sources). Our team of experienced professionals combines deep industry knowledge with expertise in the latest technologies to deliver customized solutions that meet the unique needs of each of our clients. Whether you are looking to streamline your operations, enhance your customer experience, or improve your decision-making process, Aventior has the skills and resources to help you achieve your goals.
We bring a well-rounded, cross-industry, and multi-client perspective to our client engagements. Our strategy is grounded in design, implementation, innovation, migration, and support. We have a global delivery model, a multi-country presence, and a team well-equipped with professionals and experts in the field.
