Engineering Manager, Kubernetes Bare Metal Infrastructure
Together AISan FranciscoOn-Site
Needs a free Superjobs account. You'll finish on Together AI's Greenhouse page.
Skill score
Summary
The engineering manager leads a team that runs Kubernetes on bare-metal GPU infrastructure, owning provisioning, lifecycle, reliability and hiring for the fleet. It suits a senior infrastructure leader with deep Kubernetes, Linux and hardware experience who can both manage engineers and debug production systems.
- Function
- Development
- Estimated salary
- Est. $184k–275k/yr
- Estimated for a manager role, ~8 yrs experience, United States. Not published by the employer.
- Work mode and location
- On-Site, San Francisco
- Experience
- 8+ yrs
Job description
Overview
We are hiring an Engineering Manager/Tech Lead Manager to lead the team that runs Kubernetes on bare-metal GPU infrastructure at Together AI. You will own how thousands of GPUs across our data centers are provisioned, orchestrated and kept healthy for the customers training, fine-tuning and serving models on our platform.
About the team
This team sits at the base of that stack. Every training run, inference endpoint and customer cluster depends on the reliability, utilization and speed of the bare-metal Kubernetes layer you will lead.
Roles and responsibilities
Own the bare-metal Kubernetes platform
- Set the technical direction for how we run Kubernetes on bare metal across multiple data centers and regions.
- Drive automated, zero-touch provisioning of GPU nodes, from firmware and OS imaging through cluster join and validation.
- Own cluster lifecycle at scale: upgrades, capacity expansion, node health checks, automated remediation and decommissioning.
- Make GPU scheduling, topology-aware placement and high-performance networking (InfiniBand, RoCE, RDMA) first-class in the platform.
Drive reliability and efficiency
- Define and meet SLOs for cluster availability, node bring-up time and GPU utilization.
- Invest in observability across hardware, networking, Kubernetes and workloads so failures are found before customers see them.
- Cut the time from racked hardware to customer-ready capacity.
Lead and grow the team
- Hire, manage and develop a team of infrastructure and platform engineers, setting a high bar for technical depth and operational ownership.
- Run the team's planning, prioritization and delivery, balancing roadmap work against fleet growth and production support.
- Build a healthy on-call culture: sustainable rotations, blameless postmortems and follow-through on remediation.
- Work with data center operations, networking, hardware vendors, inference, training and product teams to plan capacity and launch new hardware generations.
- Represent the platform's needs and trade-offs to leadership, and turn customer requirements into a clear roadmap.
Skills required
Top skills
- Kubernetes
- Bare metal infrastructure
- Linux systems
- Networking
All skills (11)
- Hardware management
- PXE boot
- Fleet provisioning automation
- Cluster API
- Terraform
- Ansible
- SLO management
- Incident response
- NVIDIA GPU clusters
- InfiniBand
- Slurm
Qualifications
- 8+ years in infrastructure, SRE or platform engineering, including 2+ years managing engineers.
- Deep hands-on experience running Kubernetes in production at scale, ideally on bare metal rather than only managed services.
- Solid grounding in Linux systems, networking and hardware: kernels, drivers, BIOS/BMC, PXE boot and out-of-band management.
- Experience building automation for fleet provisioning and lifecycle with tools such as Cluster API, Metal3, Ironic, MAAS, Ansible or Terraform.
- A track record of owning production systems with on-call, SLOs and incident response.
- Strong technical judgment: you can review designs, debug alongside the team and make clear trade-offs.
- Clear communication with engineers, leadership and external partners.
Nice to have
- Experience operating large NVIDIA GPU clusters (H100, H200, B200, GB200) and the NVIDIA GPU and Network Operators, DCGM and NCCL.
- Familiarity with InfiniBand, RoCE, SR-IOV and high-performance storage for AI workloads.
- Experience with Slurm, Kueue, Volcano or other batch schedulers for training workloads.
- Contributions to Kubernetes or related CNCF projects.
- Experience at an AI lab, GPU cloud or hyperscaler.
Other details
We offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full-time position is: $300,000 -350,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.
Company info
About Together AI
Together AI provides a full-stack AI cloud platform offering inference, fine-tuning, and GPU cluster services. Its platform is used by developers and organizations that want to build and run AI models.