Sr. Principal Engineer, Scale-Up (Switch Silicon & Traffic Management)
Upscale AIUS - Santa Clara, CAOn-Site
Needs a free Superjobs account. You'll finish on Upscale AI's Lever page.
Skill score
Summary
This senior individual contributor owns how data-path features are qualified on a scale-up AI networking platform, from the SAI/SDK path through ASIC forwarding and traffic management, and sets test strategy and release sign-off. The role suits a senior validation engineer with deep traffic management, lossless Ethernet, and framework-building experience who can also mentor QA engineers.
- Function
- Development
- Estimated salary
- Est. $254k–380k/yr
- Estimated for a staff-level role, ~15 yrs experience, United States. Not published by the employer.
- Work mode and location
- On-Site, US - Santa Clara, CA
- Experience
- 15+ yrs
- Employment type
- Full-time
Job description
Overview
Senior individual-contributor validation role. You own how data-path features are qualified on our scale-up AI networking platform, from the SAI/SDK programming path through ASIC forwarding and traffic-management behavior. You set the test strategy, own feature readiness and release sign-off, debug the hardest cross-stack failures, and grow the QA engineers around you. Primary depth is traffic management. Feature scope (cycle-accurate model and silicon)
Roles and responsibilities
- Classification and marking: DSCP and 802.1p to traffic class, trust, TC-to-queue, remarking, port versus switch map binding, classification miss, ACL rewrite
- Queueing: unicast and multicast queues, counters, occupancy, watermarks, drops
- Scheduling: strict priority, WRR/DWRR, hierarchy, shaping, rate limiting, fairness under oversubscription
- Buffering: ingress and egress pools, static and dynamic thresholds, priority groups, headroom, reserved versus shared
- PFC and lossless Ethernet: pause, headroom, lossless queues, watchdog, storm and deadlock, DCB
- ECN and WRED: thresholds, drop probability, ECT/CE, interaction with DCQCN under RoCEv2
- Policing: color-blind and color-aware, CIR/PIR/CBS/PBS, trTCM
- Forwarding and counters: L2/L3, ECMP, mirror and sample, telemetry that makes the above observable
- Own the validation strategy, test plans, coverage, and release entry/exit criteria for the features above
- Qualify lossless and congestion behavior under RoCEv2 and GPU collectives (NCCL/RCCL) at line rate, including incast, PFC headroom, ECN accuracy, and no-drop
- Build the frameworks and debug tools the team depends on, including model-based checks of classification, queueing, scheduling, and drop or mark outcomes
- Lead root-cause analysis across software, SDK, model, silicon, and Linux, and decide when the spec, the implementation, or the test is wrong
- Mentor QA engineers, review their plans and automation, and represent validation in design reviews and the development-to-QA handoff
- Support customer deployments, proofs of concept, and field issues on data-path and congestion behavior
Skills required
Top skills
- Traffic management
- PFC
- Scheduling and queueing
- Buffer management
All skills (13)
- Classification and marking
- ECN and WRED
- L2/L3 networking
- ACL fundamentals
- SAI or switch SDK
- SONiC or FBOSS
- Python or Go
- Linux debugging
- Validation frameworks
- RoCEv2 and DCQCN
- NCCL/RCCL
- ASIC bring-up
- Ixia or Spirent
Qualifications
- 15+ years in networking software, system validation, or ASIC/platform QA, with a substantial stretch as a senior individual contributor
- Deep hands-on traffic management: PFC, scheduling, buffers, classification and marking, queueing, ECN/WRED
- Strong L2/L3 and ACL fundamentals, and experience with SAI, a switch SDK, or a NOS such as SONiC or FBOSS
- Ownership of validation frameworks and automation, not only test execution
- Python or Go at framework-review level; solid Linux and packet- and counter-level debugging
- Proven mentoring and influence as an individual contributor, and clear written readiness and root-cause reports
Nice to have
- Ethernet ASIC QA in L2/L3, buffers, scheduling, and queueing
- RoCEv2, RDMA, DCB, PFC tuning, and DCQCN on AI or HPC fabrics; NCCL/RCCL on NVIDIA or AMD GPU clusters
- ASIC bring-up, C-model or emulation, and pre-silicon to post-silicon correlation
- Buffer and headroom sizing; incast, microburst, and oversubscription debug
- Ixia, Keysight, Spirent, or similar line-rate tools
- NVLink or ESUN; customer, field, or forward-deployed work is a plus
Other details
Where you fall within that range depends on your experience, skills, and impact—we benchmark against internal levels to keep things fair and consistent.
Company info
About Upscale AI
Upscale AI builds purpose-built networking infrastructure for AI workloads, providing rack-scale performance on open standards for AI training and inference. Its customers are organizations running AI training and inference workloads that need high-performance networking.