Research Engineer, Real Environments
MercorSan FranciscoOn-Site
Needs a free Superjobs account. You'll finish on Mercor's Ashby page.
Skill score
Summary
The person in this role works with large enterprises to turn their real workflow data into RL environments, verifiers, and training datasets for evaluations and frontier labs. They build and ship end-to-end evaluation environments and scale their production, which suits engineers with prior environment-building experience and strong full-stack skills.
- Function
- Development
- Estimated salary
- Est. $120k–180k/yr
- Estimated for a mid-level role, ~3 yrs experience (typical for the level), United States. Not published by the employer.
- Work mode and location
- On-Site, San Francisco
- Employment type
- Full-time
Job description
Overview
You’ll work with large enterprises to capture their data and transform it into high-fidelity RL environments for capability evaluations and training datasets for frontier labs. We focus on pushing the frontier of world-building, verifier engineering, and more alongside our partners.
Your goal will be to automate the process of building evals for real work in the economy.
Roles and responsibilities
- Ship models for workflow extraction, classification, and grading.
- Engineer autonomous task refinement processes which distill data taste into pipelines.
- Deliver data to customers and deploy into real engagements.
- Help define the future of agentic transformation for enterprises around the world.
- Deeply learn about the intricacies of enterprises through building evaluations for all aspects of work.
- Build end-to-end environments for labs & enterprises by platformizing sandbox app clones, load real data into the sandboxes, build prompts from real workflows, and write verifiers leveraging enterprise expertise & golden outputs.
- Systematize the production of environments to scale throughput while maintaining high-quality worlds and verifiers.
Skills required
Top skills
- Full-stack engineering
- RL environments
- Agentic evaluations
- Verifier engineering
All skills (5)
- Data pipelines
- Model evaluation
- Temporal
- Modal
- Synthetic data generation
Qualifications
- Prior experience shipping environments – you’ve contributed to an OSS framework, built environments at previous companies, or worked on agentic evaluations.
- Strong full-stack engineering skills – you’ll be responsible for everything from infrastructure to app code to analytics
- Bias to action – this team is focused on shipping evals, not just philosophizing about them.
- Curiosity – being biased towards understanding and digging deep into model behavior and actually looking at the data.
- Sweat the details that make a simulation indistinguishable from the real thing and have systems-level thinking skills that allow you to scale up quality.
Nice to have
- Experience with Temporal, Modal, or similar orchestration/compute services
- Experience with synthetic data generation for frontier models.Past work auditing and scrutinizing industry-standard evaluations
Other details
- Generous equity grant vested over 4 years
- Health, dental, and vision insurance
- 401(k) with company match
- Unlimited paid vacation and sick days
- Parental leave for birthing and non-birthing parents
- Free mental health services
- $200 monthly wellness reimbursement
- $1.5K monthly meal stipend
- $200 monthly laundry reimbursement
- Free Equinox membership
- Up to $15K relocation bonus
Company info
About Mercor
Mercor’s mission is to organize human intelligence for the AI economy. We build the infrastructure that enables frontier AI labs and enterprises to train, evaluate, and improve AI on real-world professional work. More than five million experts have joined our network, and we pay out over $5 million daily for