Role Overview

We are looking for an experienced Senior Software Engineer with strong expertise in Python, algorithm development, scientific computing, and data-driven solution development.

The ideal candidate will design, implement, validate, and productionize algorithms that transform complex scientific and operational data into actionable insights. The role will work closely with domain experts, data scientists, data engineers, and platform teams to develop robust solutions for data analysis, signal and image processing, predictive analytics, and machine learning.

As our data platform evolves from the current AWS-based environment toward Databricks, the candidate will also play an important role in migrating and modernizing algorithmic and analytical workloads. This includes adapting Python-based computational workflows for distributed processing, developing reusable features for analytics and machine learning, and helping establish scalable ML development and operational practices on Databricks.

This role does not own core data ingestion or ETL pipelines. It focuses on the analytical and algorithmic layer that consumes curated data provided by the data engineering platform.

Key Responsibilities

Design, develop, and maintain algorithms for processing, analyzing, and modeling scientific and operational data.
Develop robust Python-based computational workflows using appropriate scientific computing, data analysis, and machine learning libraries.
Analyze complex datasets to identify patterns, derive meaningful features, and develop predictive or diagnostic insights.
Develop and optimize algorithms for areas such as signal processing, image analysis, anomaly detection, predictive analytics, and other data-driven applications.
Evaluate algorithm performance through systematic testing, validation, benchmarking, and comparison against defined performance criteria.
Collaborate with domain experts and data scientists to translate scientific and business problems into practical analytical and algorithmic solutions.
Collaborate with data engineers to define data requirements and consume reliable, curated datasets without duplicating ingestion and ETL responsibilities.
Develop reusable, modular, testable, and maintainable Python components for algorithmic and analytical workloads.
Improve the performance and scalability of computational workloads, including the adoption of distributed processing where appropriate.
Support the migration of existing algorithmic and analytical workloads from the current AWS-based data platform to Databricks.
Refactor and adapt existing Python and analytical workloads to operate effectively with Databricks, Spark/PySpark, Delta-based datasets, and distributed compute where appropriate.
Develop reusable features and feature-generation logic that can be consistently used for experimentation, model training, batch inference, and production applications.
Contribute to the adoption of structured ML lifecycle practices including experiment tracking, reproducibility, model validation, versioning, deployment, and monitoring.
Work with platform and ML engineering teams to productionize algorithms and machine learning models through repeatable and automated workflows.
Participate in technical design discussions, solution reviews, estimation, and implementation planning for algorithm and ML solutions.
Apply software engineering practices including source control, automated testing, code review, CI/CD, documentation, and observability to algorithm and ML development.
Mentor less experienced engineers and contribute to engineering standards and reusable development practices.

Required Technical Skills

8-12 years of professional experience in software engineering, algorithm development, scientific computing, data science, machine learning, or related technical disciplines.
Strong hands-on programming experience with Python and experience developing production-quality Python software.
Strong experience with scientific computing and data analysis libraries such as NumPy, pandas, SciPy, or equivalent technologies.
Experience designing and implementing data-processing, analytical, mathematical, or predictive algorithms.
Experience working with complex datasets and developing data-driven models, features, or analytical workflows.
Experience in one or more areas such as machine learning, predictive analytics, signal processing, image processing, time-series analysis, optimization, or statistical modeling.
Good understanding of machine learning fundamentals including feature engineering, model training, validation, evaluation, and inference.
Experience translating exploratory or prototype algorithms into maintainable and production-ready implementations.
Good understanding of software engineering principles including modular design, object-oriented programming, testing, maintainability, and performance optimization.
Experience working with cloud-based data and compute environments, preferably AWS.
Working knowledge of SQL and experience accessing and analyzing structured datasets.
Understanding of scalable or distributed data-processing concepts and the considerations involved in processing large datasets.
Experience with source control systems such as Git.
Understanding of CI/CD, automated testing, DevOps practices, and modern software development workflows.
Ability to collaborate effectively with data engineers, data scientists, domain experts, platform engineers, and application-development teams.

Preferred Skills

Hands-on experience with Apache Spark or PySpark for distributed data processing.
Experience developing machine learning or analytical workloads using Databricks notebooks, jobs, or equivalent managed compute environments.
Familiarity with Delta Lake and lakehouse-based data architectures.
Experience with MLflow or equivalent tools for experiment tracking, model management, and ML lifecycle management.
Experience developing and managing reusable features for machine learning, including exposure to feature stores or feature engineering frameworks.
Familiarity with Unity Catalog or similar approaches to data, feature, and model governance.
Experience productionizing machine learning models for batch or real-time inference.
Understanding of MLOps practices including model versioning, reproducibility, automated testing, deployment, monitoring, and retraining workflows.
Experience optimizing Python or Spark workloads for performance and scalability.
Experience with containerization technologies such as Docker and deployment into cloud-based environments.
Experience integrating analytical algorithms or ML models into APIs, digital services, applications, or embedded systems.
Experience working with scientific instrumentation, engineering data, image data, sensor data, or other complex technical datasets.
Experience working in industrial R&D, scientific, engineering, or product-oriented development environments.

Behavioral Competencies

Strong analytical and problem-solving skills, with the ability to understand complex technical and scientific problems.
Ability to move between exploratory algorithm development and disciplined production software engineering.
Ability to learn new technologies and adapt existing solutions as the underlying data and analytics platform evolves.
Strong ownership and accountability for algorithm and ML deliverables from initial exploration through production use.
Ability to make pragmatic technical choices between conventional Python processing, distributed processing, and machine learning approaches.
Strong communication skills and the ability to explain analytical methods, assumptions, results, and technical trade-offs clearly.
Ability to collaborate effectively across multidisciplinary teams including software engineering, data engineering, data science, platform engineering, and domain specialists.
Ability to work independently while contributing effectively within cross-functional engineering teams.
Strong focus on software quality, reproducibility, performance, and maintainability.
Curiosity and willingness to develop new technical capabilities as the organization transitions toward a Databricks-based data and AI platform.

Education

Bachelor's or Master's degree in Computer Science, Data Science, Mathematics, Physics, Electronics, Engineering, or a related technical discipline.

Senior Software Engineer – Algorithm & ML Development