Sr MLOps Enigneer

Senior MLOps Engineer

We are seeking a Senior MLOps Engineer to steer the technical vision of our Training and Inference Optimization team. In this high-impact role, you will architect the infrastructure that powers our next-generation AI models. You will bridge the gap between systems programming and machine learning, optimizing large-scale LLM training via NVIDIA NeMo and building ultra-high-throughput serving systems using vLLM, TensorRT-LLM, and SGLang.

Your mission is to ensure our models are not only state-of-the-art but also production-hardened, cost-efficient, and performant at scale.

Key Responsibilities

• Training Infrastructure: Architect and maintain scalable distributed training pipelines using NVIDIA NeMo/Nemotron/Megatron-Bridge. You will optimize GPU utilization, manage complex checkpointing strategies, and implement automated fault tolerance for long-running jobs.

• Inference Orchestration: Lead the deployment of LLMs using vLLM, TensorRT-LLM, or SGLang. You will implement and tune cutting-edge techniques – including PagedAttention, continuous batching, and advanced quantization (AWQ/FP8) to maximize throughput and minimize TPOT (Time Per Output Token).

• Workload Orchestration: Utilize SLURM/Flyte/Ray/SkyPilot to manage and scale ML workloads across diverse cloud providers and on-prem clusters, ensuring seamless resource shifting and cost-effective execution.

• Lifecycle Management: Standardize model tracking, versioning, and transition workflows using MLflow (or similar tool), ensuring reproducible training runs and a clear path from research to production.

• Performance Engineering: Conduct deep-dive profiling and bottleneck analysis across the full stack – from CUDA kernels and NCCL collective communications to Python-level orchestration.

• Efficiency & Cost Governance: Monitor and optimize cloud and on-prem GPU expenditures through intelligent scaling policies and high-density resource packing.

• Technical Leadership: Set the bar for engineering excellence. You will drive the roadmap, perform rigorous code reviews, and mentor junior and mid-level engineers.

Required Qualifications

• Experience: 5+ years in MLOps, DevOps, or Software Engineering, with a minimum of 2 years dedicated to LLM infrastructure.

• Deep Learning Ecosystem: Expert-level proficiency with PyTorch and the NVIDIA stack (CUDA, NCCL, Triton).

• Specialized Tooling: Hands-on experience with NVIDIA NeMo (or Megatron-Bridge) for distributed training and at least two of the following for serving: vLLM, TensorRT-LLM, or SGLang.

• Orchestration & Lifecycle: Proven experience with SLURM/Flyte/Ray/SkyPilot for cluster management and MLflow (or similar tool) for experiment and model management.

• Infrastructure: Deep expertise in Kubernetes and K8s operators (e.g., KubeRay, MPI Operator, or Run:ai).

• Systems Programming: Mastery of Python and a functional understanding of C++ or Rust for performance-critical components.

• Next-Gen Hardware: Familiarity with high-performance networking (InfiniBand/RoCE) and NVIDIA H200/B200 (Blackwell) architectures.