- Redesign our current BigQuery-based job orchestration system (jobs table with lease/complete state transitions, driven by frequent small DML updates) into a Pub/Sub-based work-queue architecture
- Rework existing Terraform-managed infrastructure to provision Pub/Sub topics, subscriptions, IAM, and dead-letter handling
- Update local worker client code (Python and MATLAB) to consume from Pub/Sub instead of polling BigQuery
- Design for idempotency and safe handling of redelivery/ack-deadline edge cases given long-running FEA jobs
- Address MATLAB's lack of native Pub/Sub SDK support (e.g., via REST/gRPC integration)
Vertex AI Migration (AI Training Pipeline)
- Migrate an existing containerized Python training pipeline from Cloud Run Jobs to Vertex AI (Custom Training Jobs) to resolve compute scaling limitations
- Set up Vertex AI infrastructure and Cloud Build CI/CD pipelines for containerized training jobs
- Port existing training codebase with minimal disruption, resolving environment/resource-spec differences between Cloud Run and Vertex AI
