Live quality scoreboard

Honest labeling

Live-LLM samples only (real model adapter). MockModel catalogue eval pass-rates are a different metric and must not be sold as live quality. Coverage is the hero / flagship sets named below — not the full 551-agent catalogue.

Hero / flagship sets only. Depth: live on pilots still requires OAuth connector evidence (Wave 4), not merely a live-LLM FAQ pass.

Go-live 18 heroes

2026-08-03 · model anthropic · eval-live-2026-08-03-go-live-18.md

17/18 · 94.4%

us-accounting-practice
When is monthly payroll tax due?
us-events-venue
What wedding packages do you offer?
us-building-management
How much is the monthly levy for a 2-bedroom?
us-pharmacy
Do you have Panado 500mg in stock?
us-gym-membership
What memberships do you offer?
us-dental-front-desk
What treatments do you offer?
us-hotel-guest
What time is breakfast?
us-executive-assistant
What's on my calendar tomorrow?
us-it-helpdesk
My laptop won't connect to VPN.
us-sales-qualifier
Tell me about the Growth plan.
us-salon-booking
Can I book a haircut Saturday morning?
us-home-services
I need a plumber this week.
us-clinic-front-desk
What should I bring to my first visit?
us-restaurant-takeaway
Do you offer gluten-free options?
us-customer-support
How do I reset my password?
us-delivery-tracking
Where is my package?
us-trades-receptionist
Emergency plumbing — can someone come out?
us-onboarding-buddy
What do I do on my first day?

Flagship depth Phase 2 (financial + vet)

2026-08-03 · model anthropic · eval-live-2026-08-03-flagship-2.md

4/4 · 100%

us-mobile-money
What's my float?
us-wealth-management
What are the key facts about your wealth desk?
us-tax-office
When is the individual filing deadline?
us-veterinary
What do you charge for a wellness consult?

Wave 4 connector proofs

First-slice agents with recorded live tool results (4/4). Staging may show OAuth configured but not connected until Actions → Connect is re-run — that step is required before promoting more pilots to Depth: live.

google_calendarhubspotmcpslackwebhook

Agent	Connector	Correlation	When
`us-hotel-guest`	slack	`corr_wave4_msb83dik`	8/2/2026, 3:10:47 AM
`us-sales-qualifier`	hubspot	`corr_wave4_msb98i0f`	8/2/2026, 3:42:38 AM
`us-hotel-guest`	webhook	`corr_wave4_wh_msb9bsjz`	8/2/2026, 3:45:11 AM
`us-hotel-guest`	mcp	`corr_wave4_mcp_msba09u3`	8/2/2026, 4:04:13 AM

Ops — next live Depth promotion

Staging Actions → Connect Google Calendar + Slack (OAuth apps are already configured).

DEMO_BASE=https://miaiweb-production.up.railway.app pnpm proof:live --chat --expand --auto-record

Prefer us-events-venue as the flagship demo path; record corr ids on the pilot Evidence line before flipping Depth: live.

MyInstantAI OIDC / wallet / gateway remain external — mock rails on staging are expected until that wire-up.

Honest labeling

Hero / flagship live-LLM sets

Go-live 18 heroes

Flagship depth Phase 2 (financial + vet)

Wave 4 connector proofs

Ops — next live Depth promotion