- Posted 3 months ago
- Over 100 applicants have applied
Job Description
- Overall 10-18 yrs. of Data Engineering experience with Minimum 4+ years of hands on experience in Databricks. Ready to travel Onsite and work at client location.
- Proven hands-on experience as a Databricks Architect or similar role with a deep understanding of the Databricks platform and its capabilities.
- Analyze business requirements and translate them into technical specifications for data pipelines, data lakes, and analytical processes on the Databricks platform.
- Design and architect end-to-end data solutions, including data ingestion, storage, transformation, and presentation layers, to meet business needs and performance requirements.
- Lead the setup, configuration, and optimization of Databricks clusters, workspaces, and jobs to ensure the platform operates efficiently and meets performance benchmarks.
- Manage access controls and security configurations to ensure data privacy and compliance.
- Design and implement data integration processes, ETL workflows, and data pipelines to extract, transform, and load data from various sources into the Databricks platform.
- Optimize ETL processes to achieve high data quality and reduce latency.
- Monitor and optimize query performance and overall platform performance to ensure efficient execution of analytical queries and data processing jobs.
- Identify and resolve performance bottlenecks in the Databricks environment.
- Establish and enforce best practices, standards, and guidelines for Databricks development, ensuring data quality, consistency, and maintainability.
- Implement data governance and data lineage processes to ensure data accuracy and traceability.
- Mentor and train team members on Databricks best practices, features, and capabilities.
- Conduct knowledge-sharing sessions and workshops to foster a data-driven culture within the organization.
- Will be responsible for Databricks Practice Technical/Partnership initiatives.
- Build skills in technical areas which support the deployment and integration of Databricks-based solutions to complete customer projects.
Roles & Responsibilities
- Bachelor's or Masters degree in Computer Science, Information Technology, or related field.
- In depth hands-on implementation knowledge on Databricks. Delta Lake, Delta table - Managing Delta Tables, Databricks Cluster Configuration, Cluster policies.
- Experience handling structured and unstructured datasets
- Strong proficiency in programming languages like Python, Scala, or SQL.
- Experience with Cloud platforms like AWS, Azure, or Google Cloud, and understanding of cloud-based data storage and computing services.
- Familiarity with big data technologies like Apache Spark, Hadoop, and data lake architectures.
- Develop and maintain data pipelines, ETL workflows, and analytical processes on the Databricks platform.
- Should have good experience in Data Engineering in Databricks Batch process and Streaming
- Should have good experience in creating Workflows & Scheduling the pipelines.
- Should have good exposure on how to make packages or libraries available in DB.
- Familiarity in Databricks default runtimes
- Databricks Certified Data Engineer Associate/Professional Certification (Desirable).
- Should have experience working in Agile methodology
- Strong verbal and written communication skills.
- Strong analytical and problem-solving skills with a high attention to detail.
Role: Solution Architect
Industry Type: IT Services & Consulting
Department: Engineering - Software & QA
Employment Type: Full Time, Permanent
Role Category: Software Development
Education
UG: BCA in Any Specialization, B.Tech/B.E. in Any Specialization
PG: MCA in Any Specialization, M.Tech in Any Specialization
More Info
About Company
Impetus designs and develops state-of-the-art products, spanning all leading-edge development platforms, for a host of fast-paced and innovative software companies. We offer product-oriented services, across the entire product development lifecycle, to our clients.
We keep very successful company-all our clients are leaders in their spaces. They range from some of the finest startups in the Silicon Valley, such as Scalix, KnowNow, Navic, Sennari, to well established players including NeuStar, 3M, LiveCapital and PeopleSoft. Leading blue-chip VCs like Sequoia, Kleiner Perkins, Redrock, and Venrock recommend us as the preferred offshore service provider for companies they are incubating.
Our sister companies, Servigistics, ClearTrail and Intellicus, and the intellectual property we create, have been phenomenally successful, setting innovative trends and establishing new markets.
