cavendish
QueueExperimentsScorecard
Type 3Hypothesis

Delegated-authority broker for agent tool calls

Auth Broker
  1. proposed
  2. voting
  3. running · 17d
  4. measuring
  5. concluded

Preregistration · v1 · 3 Aug 2026 · immutable after start

An extension creates a new version rather than editing the old.

Hypothesis

A broker that mints per-task, resource-scoped, twenty-minute tokens on behalf of a named user lets an agent complete the banking servicing pattern's twelve tool calls with zero standing credentials and under 80ms added latency per call.

Kill condition

Kill if any of the twelve tool calls requires a standing credential to complete by day 10, or if added p95 latency exceeds 250ms.

MethodBroker in front of a synthetic core-banking API and a document store, both with synthetic records. Agent requests authority per task; broker checks the user's delegation, mints a scoped token, logs the grant. Measure completion, latency, and the red team's attempts to escalate scope across the twelve calls.
Expected cost$500 in tokens, two people for three weeks
Expected duration3 weeks

Running notes · fed from harness sessions and by the pair

AW
manual · Adam Witanowski · 3 Sep 2026· ⌘↩ to post
  1. manual29 Aug 2026LF Lena Fischer

    Escalation attempt 3 of 8 succeeded: a token scoped to one customer read a linked account through a join the API exposes. Scope has to be resource-set, not resource. Fix in progress.

  2. harness21 Aug 2026TO Tom Okafor

    broker + synthetic APIs up; 9 of 12 calls completing scoped; added p95 71ms

  3. manual3 Aug 2026LF Lena Fischer

    Design-review basis, not the tenant we never got (g-auth-broker-oauth-only). Predicted: completes, latency 60ms, one escalation found and closed, confidence 0.5.

Harness notes are auto-captured from Claude Code sessions: model, date, commit, session reference. Never the transcript, code or paths.

Artifacts

repoauth-broker POC + synthetic banking APIsrepo
measCompletion, latency and escalation attempts across twelve callsmeasurement