Skip to content

Inference Engineer

Adaption LabsSan FranciscoRemote

Estimated salary, not published by the employer: Est. $138k–208k/yr

Needs a free Superjobs account. You'll finish on Adaption Labs's Ashby page.

Skill score

Summary

The Inference Engineer owns the cost and performance of Adaption Labs' inference stack, improving throughput, latency, and efficiency through caching, batching, quantization, decoding, and kernel-level work. This suits engineers with hands-on production experience in model serving and GPU performance who prefer measurable output over credentials.

Function
Development
Estimated salary
Est. $138k–208k/yr
Estimated for a mid-level role, ~5 yrs experience, United States. Not published by the employer.
Work mode and location
Remote, San Francisco
Experience
5+ yrs
Employment type
Full-time

Job description

Overview

Our vision is AI systems that are flexible, adaptable, and accessible to everyone. We believe efficiency is what makes this possible. It’s how we expand access and ensure innovation benefits the many, not the few.

You'll own the cost and performance of our inference stack. Your work will determine how efficiently we serve models as workloads, traffic, and hardware change.

You'll work closely with the engineers operating the serving fleet while owning the core performance levers: caching, batching, quantization, decoding, and kernel-level optimization. Success means improving throughput and latency without compromising reliability or model quality.

Roles and responsibilities

  • Improve throughput, cost, and tail latency through KV-cache management, continuous batching, speculative decoding, and quantization.
  • Optimize long-context prefill and decode workloads based on real production traffic.
  • Tune routing between our infrastructure and external providers based on cost, capacity, and performance.
  • Work within serving engines such as vLLM, SGLang, and TensorRT-LLM, going below the framework when needed.
  • Build profiling and measurement systems that show where time, memory, and compute are being spent.

Skills required

Top skills

  • Quantization
  • Inference infrastructure
  • Performance engineering
  • Model serving

All skills (8)

  • vLLM
  • SGLang
  • TensorRT-LLM
  • Python
  • C++
  • ML systems
  • CUDA
  • Rust

Qualifications

We look beyond traditional markers of experience. What matters most to us is what you’ve built and the impact you’ve had. We value demonstrable output, curiosity and alignment with our core values:

  • 5+ years in ML systems, inference infrastructure, or performance engineering, with measurable improvements in cost or latency.
  • Deep understanding of model serving, including prefill and decode, memory bandwidth, batching, and concurrency.
  • Production experience with serving engines such as vLLM, SGLang, or TensorRT-LLM.
  • Strong Python skills and proficiency in C++, Rust, or another systems language.
  • Experience with GPU performance, including CUDA, NCCL, mixed precision, memory layout, kernels, or quantization.

Other details

Above all, we're looking for great teammates who make work feel lighter and aren't afraid to go out on a limb with bold ideas. You don't need to be perfect, but you do need to be adaptable. We encourage you to apply, even if you don't check every box.

Benefits

  • Innovative culture: You’ll be part of a generational opportunity to define the trajectory of AI, surrounded by a team pushing the boundaries of what’s possible.
  • Growth paths: A dynamic environment with opportunities to take on new challenges, expand your responsibilities, and make an impact beyond your immediate role.
  • Adaption Passport: An annual travel allowance to explore somewhere new, experience different cultures and keep expanding your horizons. It can be used to travel to any country you haven’t previously been to.
  • Lunch on us: A weekly allowance for takeout or grocery delivery, wherever you are in the world. Whether it's stocking your fridge or having dinner delivered, it’s one less thing to think about.
  • Comprehensive benefits: High-quality medical coverage and benefits designed to support your health and well-being.
  • Team offsites and visits to SF HQ: We're a global team, but believe there's real value in spending time together in person. We regularly bring teammates together at our SF HQ and for team offsites in locations around the world, including London, Mexico City, and New York.

Company info

Adaption Labs builds adaptive AI systems that continuously learn and evolve. Its AI is designed to be efficient and to serve organizations across any industry or language.

More jobs at Adaption Labs