Proof

40/40 on a blind test it had never seen.

Kept must never give medical advice and must always hand urgent messages to a person. That is enforced in code before any language model reads a message, and every reply is linted again before it is shown. These are the measured results, including the ones that went badly.

Drawn, then measured again

After YouCam's makeup try-on draws the brows back, YouCam's Face Analyzer measures the drawing and Kept corrects the colour for the next round. Recorded live on 30 Oct for both demo personas.

anna

before treatment #805D47
  1. Round 1ΔE 11.8
  2. Round 2ΔE 11.8
  3. Round 3same shade

Reached the shade measured before treatment.

marcus

before treatment #3E3834
  1. Round 1ΔE 11.4
  2. Round 2ΔE 11.4
  3. Round 3ΔE 19.3

Best round ΔE 11.4. On deep skin the drawn brows read lighter than intended; the loop keeps the best round instead of the last.

The try-on also brightens the whole face (cheek ΔE ≈ 3.5 on both personas). Kept fits the render back onto the person's own photo, bringing that to under 1.

What a colour name on a wig listing means

We searched the Shopify catalogue for 8 colour names and read the hair shade on each listing photo. 24 of 37 measured listings sat closer to another name's average than to their own.

“natural black”

6/10 measured · spread ΔE 62.2

“jet black”

5/10 measured · spread ΔE 20.6

“dark brown”

4/10 measured · spread ΔE 38.8

“medium brown”

1/10 measured · spread ΔE 0

“chocolate brown”

8/10 measured · spread ΔE 36.3

“light brown”

4/10 measured · spread ΔE 35

“auburn”

3/10 measured · spread ΔE 6.2

“honey blonde”

6/10 measured · spread ΔE 50.5

Dated snapshot 2026-10-30; listing photos vary in light and styling, which is part of the point. Data: data/audit-2026-10-30.json.

Medical questions never reach the model

1 · Pre-registered set, keyword router

55/60

60 cases fixed before any run. The negative control switches the gates off: the model still deferred every medical question on its own, but named blocked products in 5 of 10 cases. The gates make the behaviour deterministic instead of model-dependent.

medical
16/20
gates off 0/20
urgent
5/5
gates off 0/5
injection
10/10
gates off 10/10
product
9/10
gates off 5/10
buy
5/5
gates off 5/5
benign
10/10
gates off 10/10

Model gemini-3-8-flash · code bf21455 · full report

2 · Held-out set, keyword router only

17/27

New phrasings the router had never seen. Keywords don't generalise: 7 of 15 medical and 0 of 2 urgent messages got through to the model.

medical
7/15
urgent
0/2
benign
10/10

Model gemini-3-8-flash · code 2c31028 · full report

3 · Same held-out set, router + scope classifier

26/27

A separate model call classifies every message before the Companion sees it, and fails closed. This set was visible while the classifier prompt was written, so it is not a clean test.

medical
15/15
urgent
2/2
benign
9/10

Model gemini-3-8-flash · code 47b13ba · full report

4 · Blind set, written without access to Kept

40/40

40 messages written by a separate author who never saw the router, the classifier or the code, committed before the run. Router alone scored 29/40 on the same set.

medical
14/14
router only 8/14
urgent
6/6
router only 1/6
benign
20/20
router only 20/20

Model gemini-3-8-flash · code f4b6c98 · full report

When the classifier broke, nothing leaked

Our first classifier run was misconfigured: the provider ignored the output schema, so every call failed. Because it fails closed, Kept refused everything — 0/20 ordinary questions answered — but still let no medical question through (14/14 refused). We kept that run in the repository.

What we don't claim

  • That 40/40 on 40 messages means zero errors in the world. It means zero on this set.
  • That the router alone is enough: on the blind set it scored 29/40.
  • Any medical benefit. Kept is cosmetic.

Methods, cases and every report: eval/README.md. Generated 30/10/2026, 10:27:34.