Search by job, company or skills

Data Engineer | Databricks | GCP | PySpark | BigQuery | Remote India/WFH

Data Engineer | Databricks | GCP | PySpark | BigQuery | Remote India/WFH

remote_wfh_india
2-4 Years
Not Disclosed
Early Applicant
  • Posted 27 minutes ago
  • Be among the first 10 applicants

Job Description

Data Engineer | Databricks | GCP | PySpark | BigQuery

AgileEngine is looking for a strong, hands-on Senior Data Engineer to join an exciting engagement focused on building scalable data platforms, developing high-performance ETL/ELT pipelines, and productionizing AI-powered and agentic workflows for data engineering automation.

We are particularly interested in engineers with strong Data Engineering and Software Engineering fundamentals who can independently design, develop, optimize, and troubleshoot production-grade data solutions.

Location: Remote – India

Core Skills & Mandatory Requirements

- 2+ years of strong hands-on experience with Databricks and GCP.
- Strong hands-on experience with Python and PySpark.
- Advanced SQL and data-processing skills.
- Practical experience with GCP BigQuery.
- Hands-on experience with Delta Lake and modern data lake/lakehouse architectures.
- Strong understanding of ETL/ELT development and scalable data pipeline design.
- Experience with Spark performance optimization, partitioning, joins, and data processing at scale.
- Strong understanding of data modelling, incremental loads, and data quality frameworks.
- Experience with production troubleshooting, monitoring, and performance optimization.
- Upper-intermediate English proficiency (B2 or equivalent).

Added Advantages

- Experience building AI Agents and agentic workflows.
- Agentic AI and MCP (Model Context Protocol) experience.
- Experience productionizing AI-powered data engineering automation.
- Experience with Apache Airflow and dbt.
- Strong software engineering fundamentals, including clean coding, modular design, debugging, and optimization.
- Experience with REST APIs, backend development, and system design.
- GIS/Geospatial data experience.
- Go programming language.

What You'll Work On

- Design, develop, and optimize scalable ETL/ELT pipelines using Python, PySpark, SQL, and Databricks.
- Build production-grade data solutions on GCP, particularly BigQuery.
- Work with Delta Lake and modern lakehouse architectures.
- Optimize Spark workloads and handle large-scale data processing challenges.
- Implement data quality, validation, monitoring, and reconciliation frameworks.
- Productionize agentic workflows and AI-powered automation for data engineering.
- Develop clean, modular, reusable, and maintainable engineering components.
- Collaborate with engineering and business teams to deliver reliable and scalable data solutions.

Technical Interview Expectations

Important Hiring Consideration

This is a hands-on Data Engineering role. The mandatory screening criteria are 2+ years of hands-on experience with Databricks and GCP, advanced SQL, and practical experience with Delta Lake and lakehouse architectures.

Candidates must demonstrate real production experience rather than theoretical knowledge alone.

Experience building AI Agents and productionizing agentic workflows will be a strong advantage, but is not mandatory.

Interested Send me a direct message with:

1. Email ID
2. Updated resume
3. Total years of experience
4. Python experience
5. PySpark/Spark experience
6. Databricks experience (years)
7. GCP experience (years)
8. BigQuery experience
9. Delta Lake / Lakehouse experience
10. Advanced SQL experience
11. Airflow / dbt experience
12. AI Agents / Agentic AI experience (if any)
13. English proficiency level
14. Notice period / Last Working Day (LWD)
15. Current CTC / Expected CTC
16. Current location

More Info

Job Type:
Industry:
Employment Type:

Key Skills

About Company