Sr. Developer

Roles and Responsibilities:

• Develop and optimize data solutions using Delta Sharing and Databricks Unity Catalog Administration.
• Implement and manage Databricks CLI and Delta Live Pipelines for efficient data processing.
• Utilize Structured Streaming for real-time data processing and analysis.
• Apply risk management strategies to ensure data integrity, security, and compliance.
• Integrate Apache Airflow for workflow orchestration and automation.
• Leverage Amazon S3 and Amazon Redshift for scalable data storage and retrieval.
• Develop and maintain Python scripts to automate data processing tasks.
• Utilize Databricks SQL and Delta Lake for data querying, transformation, and storage optimization.
• Design and implement Databricks Workflows to automate data pipelines.
• Develop PySpark applications to process and transform large datasets.
• Collaborate with cross-functional teams to understand requirements and deliver effective data solutions.
• Monitor and troubleshoot data workflows to ensure smooth operations.
• Stay updated with the latest industry trends and technologies to improve data solutions.

Qualifications:

• Strong experience in Delta Sharing and Databricks Unity Catalog Administration.
• Proficiency in Databricks CLI and Delta Live Pipelines.
• Good understanding of Structured Streaming and Risk Management.
• Experience with Apache Airflow, Amazon S3, and Amazon Redshift.
• Proficiency in Python and PySpark.
• Strong problem-solving and analytical skills.