A
Data Engineer (Databricks)
A
Data Engineer (Databricks)
aryan solutions pte. ltd.Early Applicant
- Posted 16 days ago
- Be among the first 10 applicants
Job Description
Required Technical Skills - Databricks
- Unity Catalog - catalogue/schema design, access control, and data lineage
- Lakeflow / Delta Live Tables for pipeline orchestration Delta Lake table format
- Databricks SQL and cluster/workspace administration (compute policies, pools, cost management)
- Databricks Asset Bundles (DABs) and Databricks Repos for CI/CD
- PySpark / Spark SQL for large-scale data transformation
- Working knowledge of Databricks system tables (audit logs, billing/usage, query history) for observability
- Minimum 5 years of hands-on experience in Data Engineering, Data Platform Engineering, or related disciplines.
- Minimum 3 years of hands-on experience with Databricks involving data pipeline development, platform administration, governance, and optimization.
Required Technical Skills - Common
- Strong SQL and Python (PySpark or general-purpose) for data engineering
- Data modeling - dimensional design, star/snowflake schemas, semantic layers
- CI/CD pipelines (e.g., Azure DevOps, GitHub Actions, GitLab CI) for data engineering workflows
- Infrastructure-as-code (Terraform preferred) for provisioning cloud data platform resources
- Hands-on experience on at least one hyperscaler - AWS, Azure, or Google Cloud
Understanding of data security and compliance frameworks applicable to government/public-sector environments
More Info
Key Skills
Databricks SQL
Infrastructure-as-code
Unity Catalog
Databricks Repos
GitLab CI
GitHub Actions
CI CD pipelines
Delta Lake
Lakeflow Delta Live Tables
About Company
Skills:
Spark SQL, Apache Flink, Adf, Pyspark, Sql, Git, Apache Kafka, Databricks, Devops Tools, Airflow, CI/CD, AWS Kinesis
Skills:
data masking , access control management , Data Lineage, Etl Tools, Databricks, Api, Data Governance, Data Quality Checks, Design-Build, Analytics Support, Design, pipeline development, Regulatory Compliance, requirements from stakeholders, Snowflake Cloud Data Warehouse, Monitoring
Skills:
data engineering , production deployment , Data Management, Pyspark, Apache Spark, AWS Glue, Sql, Cloud Storage, Data Pipeline, Data Lake, Batch Management, Databricks, AWS, Data Transformation, Enterprise Storage System, Analytical and Problem-Solving Skills, Data Quality Management, Technical Writing, Distributed Platforms, Error Handling
Skills:
Apis, Pyspark, Apache Spark, Data Quality, Tableau, Json, Sql, Xml, Databricks, Data Modelling, Performance optimization, Event or near-real-time ingestion, Delta Live Tables, Auto Loader, Unity Catalog, Git-based tools, Data warehouse integration, Delta Lake, CI/CD, Structured Streaming, Databricks Workflows
Skills:
Data Modelling, Pyspark, Databricks, Python, Etl, ELT, data pipeline development, Databricks Workspace, Delta Lake, Lakehouse architecture, AI Agent