Senior Engineer - Pretraining Research & Engineering

Key Responsibilities

Design and implement next-generation LLM/Transformer architectures

Conduct research on LLM scaling laws

Optimize large-scale model training and training stability

Develop and refine large-scale tokenization strategies

Train LLMs on large GPU/compute clusters

Manage distributed and multi-node training

Monitor training performance, convergence, and stability

Diagnose and resolve loss spikes, divergence, and training failures

Take technical ownership of Pathfinder/Scout training runs

Act as technical steward for Flagship model training runs

Ensure continuous training uptime and rapid issue resolution

Optimize training efficiency, throughput, and compute utilization

Required Skills

Strong experience in LLM Pretraining

Deep understanding of Transformer architectures

Hands-on experience with large-scale model training

Experience with GPU/compute clusters

Strong PyTorch or JAX expertise

Experience with Megatron-LM / DeepSpeed / FSDP

Knowledge of distributed training

Understanding of scaling laws and training optimization

Experience with tokenization/tokenizer development

Strong understanding of training stability and convergence

Education

B.E./B.Tech/M.Tech/M.S./Ph.D. in CS, EE, Maths or related