Job Opportunity: Senior Big Data Engineer Databricks (AWS/Azure)
Position: Senior Big Data Engineer
Employment Type: Full-Time (FTE)
Experience: 8+ Years
Location: Bangalore (Nagavara, Near Manyata Tech Park)
Work Mode: Hybrid Client Location (2 3 days/week)
Notice Period: Immediate to 15 Days Preferred
Interview Mode: Face-to-Face (All interview rounds will be completed on the same day)
Mandatory Skills
- Apache Spark
- Python
- Databricks (AWS Databricks or Azure Databricks) Mandatory
- AWS or Azure Cloud
Key Responsibilities
- Design, develop, and implement scalable Big Data solutions.
- Build high-performance data pipelines using Apache Spark and Python.
- Develop and optimize ETL processes for large-scale data processing.
- Integrate data from multiple sources including RDBMS, ERP systems, and files.
- Optimize Spark jobs for performance and scalability.
- Work with cloud-based data platforms using AWS Databricks or Azure Databricks.
- Lead technical initiatives and mentor team members.
- Collaborate with cross-functional teams following Agile methodologies.
Required Skills & Experience
- 8+ years of experience in Big Data and Data Engineering technologies.
- Strong understanding of distributed computing concepts.
- Expert-level experience with Apache Spark.
- Hands-on programming experience in Python.
- Good knowledge of Hadoop ecosystem (Hadoop v2, HDFS, MapReduce, Sqoop).
- Experience with stream processing technologies such as Spark Streaming or Apache Storm.
- Experience with messaging platforms like Kafka or RabbitMQ.
- Good understanding of Hive and Impala.
- Experience integrating data from SQL Server, Oracle, ERP systems, and other enterprise data sources.
- Strong SQL skills including joins, stored procedures, and relational database concepts.
- Experience with NoSQL databases such as HBase, Cassandra, or MongoDB.
- Strong understanding of ETL frameworks and best practices.
- Experience in Spark performance tuning.
- Hands-on experience with Azure Databricks or AWS Databricks.
- Ability to lead technical teams and deliver end-to-end Big Data solutions.
- Experience working in Agile development environments.
What We Offer
- Opportunity to work on large-scale global projects.
- Continuous learning through online training, knowledge-sharing sessions, and external certifications.
- International collaboration and opportunities to contribute on global platforms.
- Sponsored Tech Talks and Hackathons.
- Opportunities for short-term and long-term relocation to global offices.
- Individual career development and growth plans.
- Comprehensive benefits including:
- Health & Medical Insurance
- Retirement Benefits
- Paid Time Off
- Flexible Benefits Program
- Employee engagement forums including CSR initiatives, photography, sports, painting, and other interest groups.
