Azure Cloud/Databricks Engineer

Azure Cloud/Databricks Engineer

Requirements:

Education:Bachelor's or Masters degree in Computer Science, Information Technology, or a related field (Engineering or Math Preferred)

Technical Skills:

• 5+ years of experience in SQL, Python,.Net
• 2+ years of experience in Azure cloud services esp - SQL Server, ADF, Azure Databricks, ADLS, Key vault, Azure Functions, Logic Apps Etc. Highlighting Databricks.
• 2+ years of experience in using GIT and deploying code using CI/CD approach.

Soft Skills:

• Strong analytical and problem-solving skills.
• Excellent communication and interpersonal skills.
• Ability to work independently and as part of a team.
• Attention to detail and a commitment to quality.

Responsibilities

Create and manage scalable data pipelines to collect, process, and store large volumes of data from various sources

Data Integration: Integrate data from multiple data sources, ensuring data consistency, quality, and reliability.

Database Management: Design, implement, and optimize database schemas and structures to support data storage and retrieval.

ETL Processes: Develop and maintain ETL (Extract, Transform, Load) processes to ensure data is accurately and efficiently moved between systems.

Data Warehousing: Build and maintain data warehouses to support business intelligence and analytics needs.

Performance Optimization: Optimize data processing and storage performance, ensuring efficient use of resources and quick data retrieval.

Documentation: Create and maintain comprehensive documentation for data pipelines, ETL processes, and database schemas.

Monitoring and Troubleshooting: Monitor data pipelines and systems for performance and reliability, troubleshooting and resolving any issues that arise.

Technology Evaluation: Stay up-to-date with emerging technologies and best practices in data engineering, evaluating and recommending new tools and technologies as appropriate.