Skip to main content

A8 (Touchstone)

apps/a8 serves the A8 eval model (a8-1-0) on an OpenAI-compatible /eval/v1 surface. A verdict is a deterministic pass over embeddings and closed-form geometry — no LLM touches the request path. Teaching arrives as an expected field on the same wire, and a background worker fits the model and promotes it.

This page covers what is unusual about a8 as a project in this monorepo. It deliberately stays thin; the details live where they can't drift:

ForSee
The public wire contract/docs/a8-reference on the marketing site (apps/web) — that page is the spec
Working in the appapps/a8/README.md
How it deploysapps/u22a8-api/README.md in u22a8-infra

It is the repo's one Python app

Everything else here is TypeScript. a8 is Python because its engine is thousands of lines of numpy — Fisher/LDA under Ledoit–Wolf shrinkage, dual ridge, thin SVD, split-conformal calibration — and because verdicts are decided by argmin and quantile comparisons, where low-order float differences flip a result. Node has no maintained BLAS binding, and the numerics are the product.

It integrates through Nx rather than yarn: apps/a8 has a project.json but no package.json, so Nx discovers it while yarn workspaces ignore it. That is why adding a8 required no changes to the other apps' Dockerfiles.

nx lint a8 # ruff check + format --check
nx test a8 # the full offline suite — no database, no network, no API key
nx run a8:test-db # adds the tests needing PostgreSQL + pgvector
nx serve a8 # PORT=… to override the default 8000
nx run a8:worker # the fit worker

Everything shells out to uv, which is installed in the devcontainer and in CI.

Config lives in apps/a8/.env, not the root .env

Nx injects the workspace-root .env into every task, and a8 needs DATABASE_URL to point at PostgreSQL while web's local Prisma setup uses SQLite. One root value can't be both. Nx loads project-level env files, so a8's belong in apps/a8/.env (already git-ignored).

Moving a verdict is a declaration

When a change can move a verdict, the provenance tokens in engine/provenance.py have to move with it — SELECTION_LAW when reader selection or fit math changes (every pool re-bakes on the next fit), SERVE_LAW when the read path can shift a verdict without θ moving. They are declared claims about behaviour, not source hashes, so a docs commit does not churn every client's system_fingerprint.

What the offline suite actually covers

The published contract, and nothing about how good the model is:

  • tests/api/ — the wire: status taxonomy, x-request-id, usage, unknown-field refusal, boot configuration.
  • tests/engine/test_contract.py — the promises a caller is given: banking as a pure side effect, the min_accuracy ladder, pins and horizons, the shape of an abstention.
  • tests/engine/test_artifacts.py — a DB-backed read must match its in-RAM twin, because they are two implementations of one selection law.
  • tests/persistence/ — θ and bank round trips, where a regression loses data silently rather than loudly.
  • tests/engine/vectors.json — an embedding cassette, so the same text always yields the same vector with no network.

Verdict quality, routing generalization, reader selection and the learning curve are the end-to-end benchmark's job: it measures them against the real encoder on real corpora, which is the only setting where those numbers mean anything. A toy corpus behind a 1536-d cassette cannot speak to model quality, and pinning its answers pins only its own history.

The llm marker does not mean "needs the network"

It means "exercises the embedding path", and those tests replay from the cassette offline by default — they are the fidelity gate, and they run in CI. -m 'not llm' skips them and leaves a green suite that proved nothing; the CI filter is -m 'not db'. Use --run-llm only to re-record the cassette.

Abstention is a feature

a8 answers only at an accuracy it has earned. Asked for a promise it cannot keep, it returns a 200 with an abstention object rather than guessing or erroring. A cold or under-taught model abstaining is correct behaviour, not a bug — see /docs/a8-abstention on the marketing site.