Skip to content

Senior Staff Software Engineer - AI/ML Inference Engineering

ServiceNowSanta Clara, CALIFORNIA, United StatesHybrid

Estimated salary, not published by the employer: Est. $207k–311k/yr

Needs a free Superjobs account. You'll finish on ServiceNow's SmartRecruiters page.

Skill score

Summary

The Senior Staff engineer leads architecture and operations of an AI inference platform, onboarding models and deploying optimized inference engines on Kubernetes, and designs an Envoy and ext_proc gateway for access control, observability and routing. It suits a senior backend engineer with strong Go or Python, distributed systems, and GPU/LLM serving experience who values pragmatic, measurable performance work.

Function
Data Science
Estimated salary
Est. $207k–311k/yr
Estimated for a staff-level role, ~8 yrs experience, United States. Not published by the employer.
Work mode and location
Hybrid, Santa Clara, CALIFORNIA, United States
Experience
8+ yrs
Employment type
Full-time

Job description

Overview

About the team - The AI Services Platform Group at ServiceNow is a customer-focused innovation group building intelligent software and smart user experiences using existing and latest advanced technologies to enable end-to-end, industry-leading work experiences for customers. We are a group of researchers, applied scientists, engineers, and product managers with a dual mission. We build and evolve the AI platform, and partner with teams to build products and end-to-end AI-powered work experiences. In equal measure, we lay the foundations, research, experiment, and de-risk AI technologies that unlock new work experiences in the future.

We are seeking a Senior Staff Software Engineer - AI/ML Inference Engineering to lead the architecture, optimization, and operational excellence of our AI inference platform. You will own the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production across multiple backends on Kubernetes infrastructure. You'll also architect the intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable advanced request routing. This role combines deep technical expertise in GPU compute, LLM APIs, inference frameworks, containerization, orchestration, and API gateway architecture with a bias toward pragmatic engineering and measurable performance impact.

Skills required

Top skills

  • LLM APIs
  • Python
  • Design patterns
  • Distributed systems

All skills (10)

  • vLLM
  • TensorRT-LLM
  • Triton Inference Server
  • Kubernetes
  • Helm
  • Go
  • GPU compute
  • CI/CD pipelines
  • Observability
  • Envoy

Qualifications

To be successful in this role you have

  • Expertise in Go or Python, OOP, Design Patterns, time and space-efficient algorithms
  • Model-serving and runtime experience, such as vLLM, TensorRT-LLM, Triton Inference Server.
  • Exposure to LLM APIs or ML-serving systems, or a clear ability to ramp up quickly
  • Experience building new products that use challenging algorithms
  • Expertise in coding efficient, object-oriented, modularized and quality software
  • Knowledge of core AI/ML techniques and algorithms
  • Knowledge of unit testing, profiling, and code tuning

AI skills

Artificial intelligence strategy is a critical skill at Experienced proficiency for IC5 — it applies to how you build and to what you build.

  • Confident use of AI coding assistants such as GitHub Copilot or Cursor, paired with the critical eye to review AI-generated code as rigorously as any other — catching logic errors, security anti-patterns, and missed edge cases rather than treating model output as production-ready.
  • AI-assisted debugging and trace analysis, and a clear understanding of the data privacy rules governing what goes into a prompt.

​​​​​​​Mandatory minimum

  • 8+ years of Backend software engineering experience
  • 4+ years of designing and operating distributed systems
  • Bachelor’s degree in computer science, software engineering, or a closely related technical field required.
  • A master’s degree may offset up to one year of the experience minimum. Equivalent practical experience is considered where technical depth is clearly demonstrable.

Nice to have

  • Deploy, scale, and operate services in production Kubernetes environments, including Helm-based deployments and CI/CD pipelines that make releases repeatable and safe to roll back.
  • Own observability for what you build — meaningful metrics, useful logs, real tracing, and alerts that fire on customer impact rather than on noise.
  • Debug and resolve production incidents independently, participate in on-call, run root-cause analysis, and operate against defined service level objectives.

Other details

For positions in this location, we offer a base pay of $201,300 - $352,300, plus equity (when applicable), variable/incentive compensation and benefits. Sales positions generally offer a competitive On Target Earnings (OTE) incentive compensation structure. Please note that the base pay shown is a guideline, and individual total compensation will vary based on factors such as qualifications, skill level, competencies, and work location. We also offer health plans, including flexible spending accounts, a 401(k) Plan with company match, ESPP, matching donations, a flexible time away plan and family leave programs. Compensation is based on the geographic location in which the role is located and is subject to change based on work location.

Work Personas

We approach our distributed world of work with flexibility and trust. Work personas (flexible, remote, or required in office) are categories that are assigned to ServiceNow employees depending on the nature of their work and their assigned work location. Learn more here. To determine eligibility for a work persona, ServiceNow may confirm the distance between your primary residence and the closest ServiceNow office using a third-party service.

Company info

ServiceNow provides a cloud-based AI platform that connects people, systems, and processes so organizations can find smarter ways to work. Enterprises and organizations use the platform, along with its AI and automation technologies, to design solutions that transform how they operate.

More jobs at ServiceNow