Describe each dataset once; discover it by meaning; fetch from the source at question time. ‹ back
One question moves through six steps. Each can send it back a step, which is why a wrong first guess degrades into a slower answer instead of a wrong one.
question │ ├─ understand shape/subshape ∥ descriptive entities, then measure/period ├─ discover ARD: embed the question, retrieve candidate tables, re-rank them ├─ plan does a candidate's declared capability support that shape? │ no → refuse here, before any request is made ├─ fetch one generic accessor fills the URL template from the OKF descriptor ├─ check is this record actually about what was asked? │ no → backtrack: next table, next entity, next period └─ synthesize answer grounded in the returned record, quoting its figure and source
The planning step is the unusual one. A source that lists one nonprofit's grants can compare two named organizations but cannot rank the whole population — so a ranking question over it is refused, not approximated. Refusing costs query-understanding calls; guessing costs credibility.
Nothing is ingested. The only thing this system stores is descriptions:
OKF descriptors ──embed──▶ ARD index (~10,400 tables, ~60 MB of vectors)
│
question ─────────────────────────┘ picks ONE table
│
▼
the source's own API ──▶ answer
(SEC, Census, Treasury, CDC, IRS, …)
The record that answers your question is fetched from the publisher, in that moment, and discarded. There is no copy to refresh and no schema to migrate. Adding a source means adding a folder with a Markdown file in it — no per-source query code.
The usual approach — Data Commons, a lakehouse, any central warehouse — normalizes many sources into one schema and loads the data into one place. That buys real things: arbitrary joins, fast aggregates, one query language. It costs real things too.
| Warehouse / Data Commons | This | |
|---|---|---|
| Unit of work | a pipeline per source | a description per source |
| Schema | normalize everything up front | keep each source's own |
| Data location | copied into the centre | stays at the publisher |
| Freshness | as of the last load | as of the request |
| Adding a source | model it, map it, backfill it | write one document |
| Good at | joins and aggregates over everything | breadth, currency, provenance |
| Bad at | long tail — the 8,000th field is never worth a pipeline | cross-source joins, population scans |
The trade is deliberate. Normalization is what makes the long tail unaffordable: nobody funds a pipeline for the 8,096th us-gaap concept, so it never arrives. A description is cheap enough to write for all of them, which is why this covers ~10,400 measures rather than a curated few.
The cost is equally real. Cross-source joins are the warehouse's home ground and this system's weak spot, and questions over a whole population need a source that can scan one — which is exactly what the planner checks before it answers.
One source is not live: the IRS 990 grant graph, ~7.8 M funder→recipient edges. The IRS publishes no query API for it, only bulk filings, so there is nothing to call at question time and the edges are built once into a database. Every other source stayed live because its publisher offered a way to ask.
Descriptors are OKF documents; discovery speaks ARD; the query interface is NLWeb. Browse the descriptors ›
This page is the overview. The life of a query › follows one question all the way through — every branch, every backtrack, and where the boundary of what can be asked actually falls.