Skip to content
K-d Tree Explorer, home
All decision records

DR-005Accepteddecided 2026-10-10

Optional AI explanations with the visitor's own key, grounded on the search trace and checked against it

docs/decisions/DR-005-bring-your-own-key-explanations.md

Context

The explorer animates every step of a search, but a reader new to k-d trees can still miss why a search was cheap or expensive. A language model could walk through one search in plain words. The site is static apart from a small place-search proxy (/api/geocode), there is no budget for API usage, API keys are secrets, and a model can state things that are not true. The site must work fully without any of this.

Decision

Add an optional "Explain this search" panel to the explorer.

  • Always available, no AI: a plain-language summary written by rules from the trace (web/src/lib/explain/summary.ts).
  • Optional, with the visitor's own key: an AI walkthrough. The visitor pastes their own Anthropic key (default model Claude Haiku 4.5, option Claude Sonnet 5.5) or OpenAI key (model id editable) into the AI settings. The key is kept in sessionStorage by default, in localStorage only if they tick "remember on this device", and "Forget key" removes it from both. Calls go directly from the browser to the provider. No code on this site sends the key anywhere except the chosen provider's API, and none of this site's server routes (the place-search proxy among them) accept or log it. The page's Content Security Policy allows requests only to this site, the OpenFreeMap map tiles and the two provider APIs.
  • Grounded: the model receives only the facts of the finished search (web/src/lib/explain/facts.ts): the insertion order, the tree's size, the query point, the counts, up to 150 numbered steps with depths, split keys and distances, and up to five business names at the answer (public CLUE data). It does not receive the place name the visitor searched for. The panel shows the exact payload before anything is sent.
  • Checked: the answer must fit a JSON schema (validated with Zod) and repeat four numbers from the trace: total comparisons, locations skipped, subtrees pruned and the step that found the answer. Five automated checks compare these with the trace and confirm that every step it cites exists. The result is shown next to the answer and stored with the call.
  • Labelled and logged: every AI output is labelled "AI-generated" with the model name. Every call is written to an audit log in the browser's IndexedDB with the feature, provider, model, prompt version, the system prompt and message sent, the request settings, the answer or error, latency, token usage, the grounding score and the visitor's decision (accepted, edited or rejected). The key is never stored. The log can be viewed, annotated and exported as JSON or CSV at /ai-log. If the browser refuses IndexedDB (blocked storage, some private modes, a full quota), the entry is kept in memory for the tab instead, and the panel says that the call was not saved.
  • No silent model switching: the server-side refusal fallback that can re-run a declined request on another model is not enabled. A refusal is shown as a refusal, and the log names the model that answered.

Options considered

  1. No AI at all. Safe, but it leaves the reader with only the rule-based summary.
  2. A server-side proxy with my key. Rejected. It costs money I do not have and needs abuse controls.
  3. A proxy with the visitor's key. Rejected. The key would pass through my server.
  4. Browser-direct calls with the visitor's key, grounded on the trace and checked. Chosen.

Why

The trace is a complete, exact account of the search, so an explanation built only from it can be checked mechanically, and the rule-based summary means nobody needs a key to understand a search. The design is informed by the Australian Government's policy for the responsible use of AI in government (DTA), the transparency principles for AI-generated content in the EU AI Act and the NIST AI Risk Management Framework. It is not a claim of compliance with any of them. In practice that guidance shows up as labelled outputs, a record of every call, a human decision on each output, automated checks, and a plain statement of what is sent where.

What happened

  • Unit tests run the whole call with fetch mocked: the request goes to the provider with the browser-access header, carries exactly the trace facts, and an answer that does not fit the schema is reported as invalid output. The key never appears in a request body, an audit entry, an export or an error message, including when a provider echoes it back.
  • The grounding checks catch invented counts and citations of steps that do not exist. They cannot judge whether the prose is a good explanation, so the reader's decision is recorded next to them.
  • I have not run a systematic evaluation of explanation quality: there is no budget for API calls, and the checks above cover only factual agreement with the trace. Visitors who use their own key produce their own logs.
  • The remaining risk is the one the Anthropic SDK names in its dangerouslyAllowBrowser option: any script running on the page, including a browser extension, can read a key held by the page. The site loads no third-party scripts, and the settings dialog recommends a separate key with a low spending limit.

What I'd change

Use short-lived, narrowly scoped credentials for browser calls if providers offer them. With a budget, I would build a small rated set of searches (cheap, expensive, truncated, radius) and score explanations from several models against the checks and a rubric, with intervals over the rated set.