Skip to content

SDET 3 - Model Evaluation

AiDASHBengaluru, Karnataka, India; Gurugram, Haryana, IndiaOn-Site

Estimated salary, not published by the employer: Est. ₹8.5–13 LPA

Needs a free Superjobs account. You'll finish on AiDASH's Greenhouse page.

Skill score

Summary

The SDET 3 engineer designs and runs model evaluation frameworks, regression pipelines, data quality gates, and inference benchmarks so that ML models are validated before release, while also owning backend test automation and CI/CD quality. It suits a senior QA or test engineer with 8-10 years of experience who is comfortable with Python, ML evaluation metrics, distributed compute, and performance testing.

Function
Testing
Estimated salary
Est. ₹8.5–13 LPA
Estimated for a mid-level role, ~8 yrs experience, India. Not published by the employer.
Work mode and location
On-Site, Bengaluru, Karnataka, India; Gurugram, Haryana, India
Experience
8–10 yrs

Job description

Overview

We are looking for a dedicated ML Model Evals & Benchmarking Engineer to bridge the gap between machine learning research and high-performance production systems. In this role, you will own the end-to-end evaluation lifecycle, designing rigorous offline and online evaluation frameworks, defining robust model performance metrics, and architecting the scalable infrastructure needed to benchmark throughput, latency, resource utilization, and statistical accuracy across diverse ML/AI models.

Roles and responsibilities

  • Own model quality before production: Establish standard evaluation frameworks using statistical metrics (e.g., Precision, Recall, F1-Score, ROC-AUC) tailored to specific business domains. Build validation frameworks for computer vision models (preferably satellite or geospatial). Catch regressions in statistical metrics before they reach customers.
  • Build version-aware model regression: Set up pipelines that compare inference outputs across model versions on curated golden datasets. Every release should come with a clear, data-backed go/no-go signal.
  • Make data quality a first-class gate: Validate training and evaluation datasets for label quality, class balance, geographic and seasonal coverage, leakage, and drift. Ensure models are tested on data that reflects real-world conditions.
  • Validate inference at scale: Design, build, and scale distributed evaluation pipelines and testing harnesses capable of running large-scale batch and real-time model benchmarks.
  • Harden the platform end-to-end: Benchmark model inference servers for throughput, latency (p50, p99), and hardware efficiency (GPU/CPU/Memory footprints) under heavy concurrent loads.
  • Software Engineering & Scripting: Advanced proficiency in Python (NumPy, Pandas, Scikit-learn, PyTorch/TensorFlow evaluation utilities) for building custom metric calculators and test harnesses.
  • Distributed Compute & Execution: Experience orchestrating heavy evaluation jobs using frameworks like Ray, Apache Spark, or Kubeflow.
  • Raise the bar for the team: Strengthen the shared automation frameworks and set standards for maintainable, non-flaky tests. Show how tools like Cursor and Claude can speed up test authoring, triage, and reporting while staying under human review.
  • Influence release decisions: This is a high-visibility role. Your quality signals will directly shape what ships across Aidash’s product lines

Skills required

Top skills

  • Python
  • ML model evaluation
  • Statistical metrics
  • Test automation

All skills (8)

  • Backend testing
  • CI/CD pipelines
  • Performance testing
  • SQL
  • NoSQL
  • Kubernetes
  • Ray or Spark
  • AWS

Qualifications

We are looking for a senior QA engineer with deep backend automation experience who can validate ML models and backend systems end-to-end, and who actively uses agentic AI development tools (e.g., Cursor, Claude) to accelerate test design, automation, debugging, and reporting — without compromising fundamentals, coverage, or engineering quality.

  • 8–10 years in software testing and test automation, with strong ownership of backend system quality
  • Experience testing ML applications (model/service behavior, data quality, inference pipelines, evaluation/regression of model-backed features)
  • Broader stack familiarity: observability (logs/metrics/traces), cloud (AWS), contract testing, data-pipeline validation
  • Experience improving shared automation frameworks and raising team standards for maintainable tests
  • Solid grasp of testing fundamentals and best practices (test design, risk-based coverage, isolation, maintainability, flaky-test control, clear defect/RCA discipline)
  • Strong coding ability in Python, with willingness to learn the other (or additional languages) as needed.
  • Good understanding of backend system design and how services interact in distributed systems
  • Practical exposure to databases — SQL and/or NoSQL (at least one is mandatory)
  • Working knowledge of application deployment and underlying orchestration (e.g., MLflow, Kubeflow, containers, Kubernetes, service rollout/health concepts)
  • CI/CD pipelines — must have (Jenkins/GitHub Actions/Bitbucket Pipelines or equivalent): triggering suites, reading failures, improving pipeline reliability
  • Performance testing — must have (JMeter/k6/Locust or equivalent): load/benchmark basics, interpreting results, flagging SLA/risk
  • Exposure to agentic development tools such as Cursor and Claude (or similar) for coding, test authoring, log/triage assistance, and documentation — used with human review and quality gates

Company info

AiDASH is leading the PreventionFirst™movement for electric utilities and transforming grid resilience through its pioneering platform that unifies vegetation, asset, storm, and wildfire intelligence. Powered by SatelliteFirst™ Inspection & Monitoring, AiDASH delivers comprehensive visibility across the entire grid at the right frequency and budget, using the right data modality. More

More jobs at AiDASH