Senior Data Engineer - Data Acquisition

About the Job

We are seeking a highly skilled Senior Data Engineer – Data Acquisition (ODS) to join our data engineering team. The ideal candidate will have extensive hands-on experience in building and optimizing data ingestion and transformation pipelines using Azure Data Platform, Databricks, Snowflake, and Kafka.

You will be responsible for designing, developing, and maintaining robust data acquisition frameworks that support large-scale analytics and operational data systems.

Job Title

Senior Data Engineer – Data Acquisition (ODS)

Skills

Kafka, Python, Spark, Databricks, SQL, Snowflake, Azure Data Factory (ADF), XML, JSON, Scala (Basic)

Experience Required : 5+ Years

Location: Remote

Notice Period: Immediate to 15 Days Preferred

Job Description

• Design, develop, and maintain data ingestion and transformation pipelines in an Azure-based environment.
• Work with Kafka for real-time data streaming and message queuing.
• Build scalable ETL/ELT pipelines using Databricks (PySpark/Scala) and integrate with Snowflake for analytics workloads.
• Develop and optimize SQL queries and stored procedures for performance and data accuracy.
• Utilize Azure Data Factory (ADF) to orchestrate workflows and integrate with various data sources.
• Work with structured and semi-structured data formats such as XML, JSON, Parquet, and CSV.
• Collaborate with data architects, analysts, and business teams to deliver high-quality, reliable data pipelines.
• Ensure code quality, version control, and data security best practices are followed.
• Contribute to data architecture design discussions and provide technical recommendations for system improvements.

Key Responsibilities

• End-to-end development of data acquisition solutions for operational and analytical systems.
• Implement data transformation logic using Spark and Python.
• Optimize performance of large-scale data pipelines in Databricks/Snowflake.
• Handle real-time and batch data ingestion via Kafka and ADF.
• Conduct data validation, profiling, and documentation of technical specifications.

Qualification & Certification

• Bachelor's degree in Computer Science, Information Technology, or related field.
• Microsoft Azure or Snowflake certification (preferred).
• Strong understanding of modern data engineering best practices.

Good to Have

• Experience with DevOps tools and CI/CD pipelines for data deployment.
• Exposure to data modeling and data warehouse design.
• Familiarity with other streaming platforms or data orchestration tools.

About InfoBeans

InfoBeans is a global digital transformation and product engineering company, enabling businesses to thrive through innovation, agility, and cutting-edge technology solutions. With over 1,700 team members across the globe, we specialize in custom software development, enterprise solutions, cloud, AI/ML, UX, automation, and digital transformation services.

At InfoBeans, we live by our core purpose of Creating WOW! — for our clients, team members, and the community. Our collaborative culture, growth opportunities, and people-first approach make us one of the most trusted and rewarding workplaces.

https://infobeans.ai/