ServiceNow just absolutely *nailed* EVA-Bench Data 2.0 on Hugging Face — this is hands down one of the best voice agent evaluation datasets I've seen released all year! They completely blew past their original single-domain setup and now have three full enterprise domains: Airline Customer Service Management (CSM), Enterprise IT Service Management (ITSM), and Healthcare HR Service Delivery (HRSD) — a massive 4x expansion in coverage that hits exactly where it matters. The benchmark scales up to **121 tools across 213 scenarios**, every single one validated against the three biggest models of 2026: GPT-5.4, Gemini 3.1 Pro, and Claude Opus 4.6. What's genuinely clever is how they've built out authentication flows calibrated to real production requirements (OTP-based elevation shows up where it actually makes sense in each domain), plus a rich variety of scenario types that go way beyond simple happy-path tests — single-intent calls, multi-intent conversations with up to four simultaneous objectives, adversarial callers who try to bypass troubleshooting steps or misclassify urgency, and unsatisfiable goals because let's be honest: real call volume isn't always polite.
What really impressed me is their approach to realism across domains. The Healthcare HRSD domain pulls in actual US healthcare policy constraints like NPI numbers, FMLA policies, insurance coverage rules — you can tell the designers actually *understand* these industries instead of just throwing synthetic data at a wall. They've anchored this entire dataset around five rock-solid principles: voice-first scope (not every enterprise workflow belongs on a phone call), genuine realism grounded in real APIs and constraints, variety that avoids repetitive noise, authentication as consistently tracked across all three domains since it's such a notorious failure point for voice agents, and most importantly **reproducibility** — each scenario has exactly one correct resolution path with no ambiguity about what counts as success. The user simulator isn't just guessing its way through; it behaves like an actual caller making consistent decisions based on explicit negotiation instructions rather than leaving edge cases to chance interpretation.
The pipeline behind this is absolutely gorgeous: joint generation via SyGra powered by GPT-5.4 produces three interdependent components simultaneously so nothing silently falls out of sync — the user goal (structured as a full decision tree with precise pushback timing and action confirmation requirements), an initial scenario database where every entity exists consistently, and derived ground truth from executing those actions in real time against live tool writes. And they're already previewing multilingual extension support to take this beyond English-only enterprise deployments — which honestly feels like the final nail in making EVA-Bench 2.0 a *definitive* benchmark for anyone building or evaluating voice agents today. I'm running my models through it right now!
Source: https://huggingface.co/blog/ServiceNow-AI/eva-bench-data
What really impressed me is their approach to realism across domains. The Healthcare HRSD domain pulls in actual US healthcare policy constraints like NPI numbers, FMLA policies, insurance coverage rules — you can tell the designers actually *understand* these industries instead of just throwing synthetic data at a wall. They've anchored this entire dataset around five rock-solid principles: voice-first scope (not every enterprise workflow belongs on a phone call), genuine realism grounded in real APIs and constraints, variety that avoids repetitive noise, authentication as consistently tracked across all three domains since it's such a notorious failure point for voice agents, and most importantly **reproducibility** — each scenario has exactly one correct resolution path with no ambiguity about what counts as success. The user simulator isn't just guessing its way through; it behaves like an actual caller making consistent decisions based on explicit negotiation instructions rather than leaving edge cases to chance interpretation.
The pipeline behind this is absolutely gorgeous: joint generation via SyGra powered by GPT-5.4 produces three interdependent components simultaneously so nothing silently falls out of sync — the user goal (structured as a full decision tree with precise pushback timing and action confirmation requirements), an initial scenario database where every entity exists consistently, and derived ground truth from executing those actions in real time against live tool writes. And they're already previewing multilingual extension support to take this beyond English-only enterprise deployments — which honestly feels like the final nail in making EVA-Bench 2.0 a *definitive* benchmark for anyone building or evaluating voice agents today. I'm running my models through it right now!
Source: https://huggingface.co/blog/ServiceNow-AI/eva-bench-data