Senior Software Engineer

Job Description

Required Skills:

• Bachelor s or Master s degree in Computer Science, Engineering, or related field.
• Minimum 8+ years of work experience
• Proven experience in building and managing large-scale distributed data processing systems.
• Strong experience with big data frameworks like Hadoop, Spark, Hive, Kafka, etc.
• Proficiency in data processing languages such as Python, Java, or Scala(Any one)
• Hands-on experience with cloud-based big data solutions (AWS, GCP, or Azure).
• Experience with data orchestration tools like Apache Airflow.
• Strong problem-solving skills and the ability to work in a fast-paced environment.

Key Responsibilities:

• Design and develop scalable and high-performance data pipelines for collecting, processing, and storing large datasets.
• Work with Hadoop, Spark, Kafka, and other big data technologies to manage and process structured and unstructured data.
• Optimize and troubleshoot data pipelines for speed and efficiency.
• Collaborate with data scientists and analysts to integrate data-driven insights into business decision-making processes.
• Implement data security measures and maintain data integrity across the systems.
• Analyze and improve performance and scalability of data pipelines and storage solutions.
• Monitor and maintain data workflows, ensuring consistency and reliability.
• Work with cross-functional teams, including product, engineering, and operations, to understand and implement data requirements.
• Continuously evolve the data architecture to meet the future needs of the company.
• Knowledge of containerization technologies like Docker and Kubernetes is a plus.