Skip to content

AI Quality Engineer (Contract)

SideHyderabad, Telangana, IndiaOn-Site

Estimated salary, not published by the employer: Est. ₹8–12 LPA

Needs a free Superjobs account. You'll finish on Side's site.

Skill score

Summary

The AI Quality Engineer on this contract tests and validates AI-generated outputs, LLM and RAG systems, and AI agents for accuracy, grounding, and hallucinations, and builds evaluation criteria and regression suites. It suits a QA or test engineer with hands-on experience testing generative AI applications who wants to work on AI quality in a hands-on, on-site contract role.

Function
Development
Estimated salary
Est. ₹8–12 LPA
Estimated for a mid-level role, ~3 yrs experience (typical for the level), India. Not published by the employer.
Work mode and location
On-Site, Hyderabad, Telangana, India
Employment type
Contract

Job description

Roles and responsibilities

  • Test and validate AI-generated insights, recommendations, and decision-making workflows.
  • Evaluate LLM and RAG systems for accuracy, relevance, consistency, factuality, and hallucinations.
  • Validate retrieval quality, context relevance, grounding, and response quality in RAG systems.
  • Test AI agents and autonomous workflows across functional, negative, and edge-case scenarios.
  • Define AI evaluation criteria, test datasets, quality metrics, and validation processes.
  • Perform regression testing for models, prompts, RAG configurations, and AI workflows.
  • Collaborate with AI/ML engineers to identify issues and improve AI system quality.

Skills required

Top skills

  • LLM testing
  • RAG evaluation
  • Generative AI testing
  • Hallucination detection

All skills (8)

  • AI agent testing
  • Regression testing
  • Recommendation systems
  • PyTest
  • RAGAS
  • DeepEval
  • LangSmith
  • CI/CD test automation

Qualifications

Required Skills

  • Strong understanding of AI/ML and Generative AI testing.
  • Hands-on experience testing LLM and RAG-based applications.
  • Knowledge of LLM evaluation, hallucination detection, relevance, and response quality.
  • Understanding of AI agents and recommendation systems.
  • Strong analytical and problem-solving skills.

Nice to have

  • Experience with PyTest and automated testing frameworks.
  • Experience building automated AI evaluation and regression frameworks.
  • Familiarity with tools such as RAGAS, DeepEval, LangSmith, or equivalent.
  • Experience with CI/CD-based test automation, performance testing, or AI guardrails.
  • Familiarity with cloud platforms (AWS/Azure/GCP) and observability tools.

Other details

Location:- Hyderabad

Work Mode:- WFO (5 days a week)

Role Type:- Contractual (3 months and extension would depend on project requirement and performance)

  • High accuracy, relevance, and reliability of AI outputs.
  • Strong evaluation coverage across critical AI workflows.
  • Early detection and reduction of hallucinations and AI regressions.
  • Reduced production AI quality issues.
  • Increased confidence and trust in AI-generated insights and recommendations.

Company info

Side is a global video game development and services provider, offering technical and creative solutions to many of the largest developers and studios around the world. Founded in 2009 and drawing on 30+ years' experience from our parent company in Japan, Side has since grown to become a global force

More jobs at Side