Skip to content

Researcher, AI Evaluations

FigmaSan Francisco, CA • New York, NY • United StatesOn-Site

Estimated salary, not published by the employer: Est. $131k–195k/yr

Needs a free Superjobs account. You'll finish on Figma's Greenhouse page.

Skill score

Summary

The Researcher, AI Evaluations will own evaluation research for AI workstreams, defining quality criteria, designing studies, building rubrics and benchmarks, and validating automated graders against human ratings. This senior individual contributor role suits a researcher with 7+ years of experience who can turn evaluation findings into model and product decisions.

Function
Design
Estimated salary
Est. $131k–195k/yr
Estimated for a mid-level role, ~7 yrs experience, United States. Not published by the employer.
Work mode and location
On-Site, San Francisco, CA • New York, NY • United States
Experience
7+ yrs

Job description

Overview

We’re hiring a Researcher, AI Evaluations to join AI Insights within Figma’s Research team. This is a senior individual contributor role with no direct reports, reporting to the AI Insights lead and working directly with AI Research within AI Platform. You’ll strengthen the team’s model, agent system, and evaluation work through hands-on studies, clear rubrics, and reliable measures of quality that reflect user needs, and your findings will guide model and product improvements.

Roles and responsibilities

  • Own evaluation research for priority AI Research workstreams. Define quality criteria, design studies, analyze results, and recommend actions
  • Build rubrics, representative evaluation datasets, benchmarks, and quality thresholds that reflect user needs and expert design judgment
  • Work with AI Research and Modeling Platform engineers to integrate validated methods into repeatable evaluations and regression tests
  • Explain quality, failure patterns, and uncertainty. Recommend actions that guide model selection, model improvements, product priorities, and release decisions
  • Validate automated graders against human ratings, including LLM-as-judge methods that use a large language model to score outputs. Identify bias, gaps, and uncertainty
  • Connect user research with AI Research’s model development work. Share reusable methods, review work, and mentor peers

Skills required

Top skills

  • Evaluation research
  • Rubric design
  • Human evaluations
  • Inter-rater agreement

All skills (10)

  • Benchmark design
  • Qualitative research
  • Quantitative research
  • Stakeholder management
  • Regression testing
  • Braintrust
  • LangSmith
  • DeepEval
  • Statistical testing
  • LLM-as-judge

Qualifications

  • 7+ years of experience in research, applied research, or a related field, with a proven track record of independently leading complex evaluation projects and delivering measurable impact
  • Hands-on experience evaluating AI products, with demonstrated ability to translate evaluation findings into meaningful model and product improvements
  • Expertise in designing human evaluations, scoring rubrics, datasets, and benchmarks, including measuring inter-rater agreement and validating automated graders
  • Strong qualitative and quantitative research skills, with the ability to analyze data, interpret uncertainty, and draw actionable insights into model behavior and performance
  • Excellent problem-solving and stakeholder management skills, with the ability to navigate ambiguity, prioritize risks, balance speed with rigor, and deliver recommendations that influence cross-functional decisions

Nice to have

  • Experience integrating automated evaluations and regression testing into engineering workflows, with familiarity in tools such as Braintrust, LangSmith, DeepEval, or equivalent platforms
  • Proven ability to develop scalable, reusable evaluation frameworks and drive the adoption of consistent quality standards across teams
  • Background in product design, product management, data science, or engineering, with familiarity in Figma or similar design and developer tools
  • Strong statistical expertise in sampling, significance testing, and agreement metrics, with experience conducting experiments or analyzing performance metrics for AI-powered features
  • Advanced training in a quantitative or behavioral discipline, gained through formal education or equivalent hands-on experience

Other details

This is a full time role that can be held from one of our US hubs or remotely in the United States.

Compensation

Job level and actual compensation will be decided based on factors including, but not limited to, individual qualifications objectively assessed during the interview process (including skills and prior relevant experience, potential impact, and scope of role), market demands, and specific work location.

Benefits

Figma offers equity to employees, as well as a competitive package of additional benefits, including health, dental, and vision coverage; retirement benefits with company contributions; parental leave and reproductive or family planning support; mental health and wellness benefits; and paid time off. Figma provides paid sick leave, holidays, and other leave benefits in compliance with applicable federal, state, and local laws, including the requirements of the Washington Minimum Wage Act and related regulations. Exempt employees are eligible for employer‑provided paid flexible PTO in addition to flexible paid sick leave. PTO is subject to manager approval. Additional benefits may include company recharge days, cell phone and home internet reimbursements, and a number of lifestyle spending accounts. Figma also offers sales incentive compensation for most sales roles and an annual bonus plan for eligible non-sales roles. All compensation and benefits are subject to applicable plan terms and may be modified by Figma at any time, consistent with applicable law.

$169,000—$250,000 USD

Hiring process

To ensure the integrity of our hiring process and facilitate a more personal connection, we require all candidates keep their cameras on during video interviews. Additionally, if hired you will be required to attend in person onboarding.

Company info

Figma’s platform helps teams bring ideas to life—whether you're brainstorming, creating a prototype, translating designs into code, or iterating with AI. From idea to product, Figma empowers teams to streamline workflows, move faster, and work together in real time from anywhere in the world.

More jobs at Figma