Traceline: Provenance-Anchored Generation

Most document-AI systems are judged on whether the answer sounds right. We built SourceBacked around a stricter bar: whether the answer can be checked. Every answer in chat and every analysis in Studio comes with citations, and each citation resolves to a specific passage in your document, with its page and section intact. This paper explains the mechanism that makes that guarantee hold, which we call provenance-anchored generation — the engine behind the Traceline you see in the product.

The central design decision is simple to state and load-bearing everywhere: a citation is a structural artifact of retrieval, not a string the language model wrote. The model never formats a citation, invents a page number, or asserts a source. It points at passages it was given; our code turns those pointers into citations. That inversion is what separates a verifiable answer from a plausible one.

The problem: retrieval quality is not attributability

Retrieval-augmented generation (RAG) improved factuality by handing a model relevant text at query time instead of relying on its trained-in memory. But most RAG systems still ask the model to producethe citation as part of its prose — “per Section 4 [p.3]” — which means the citation is generated text, subject to the same hallucination risk as everything else the model writes. A model can cite a page that does not contain the claim, cite a document it was never shown, or format a plausible-looking reference to nothing at all. Better retrieval raises the odds that the right passage is in context; it does nothing to guarantee that the citation the user sees actually points to it.

Attributability is a separate property from retrieval quality, and it is the one that matters for a paralegal filing a brief or an analyst pasting a clause into a memo. Our architecture treats it as an invariant enforced by the code path, not a behavior we hope to prompt out of the model.

Provenance-anchored generation, defined

A generation is provenance-anchored when every claim the user is asked to trust is linked to a specific retrieved passage that carries verifiable source coordinates (document, page, section), and that link is derived deterministically by the system rather than emitted by the model. Concretely, this requires four things to hold end to end:

1. Provenance is captured at ingestion and travels with every chunk — never reconstructed later.
2. The retriever surfaces a numbered set of candidate passages.
3. The model returns which numbers it used, not the citation text itself.
4. The system maps those numbers back to the real passages to build the citations.

The rest of this paper is those four stages.

Stage 1 — Ingestion: provenance is captured, not reconstructed

When a document is uploaded, a document-parsing worker converts it into markdown and, critically, into chunks that each carry their own provenance: the page numbers they came from, bounding boxes with an explicit coordinate origin, and the heading path above them. A field is reserved for audio time ranges so the same model extends to transcripts later. This provenance is recorded at parse time, when the mapping from text to physical location is still known, and it is stored on the chunk. Nothing downstream ever has to guess “which page did this sentence come from” — the answer was written down when the sentence was first extracted.

Each chunk also gets an embed_text produced by a contextualizing chunker, so the text we embed for search is the text enriched with its surrounding structure, not a bare fragment. The chunk is the atomic unit of everything that follows: it is what gets embedded, what gets retrieved, and what a citation ultimately points to.

Stage 2 — Indexing: a routing vector per document, a hybrid index per chunk

A workspace can hold many documents, and a naive “embed every chunk, search them all” approach both scales poorly and blurs the line between documents. We use two indices instead, both vector-backed by a managed text-embedding model that produces 1024-dimensional vectors, with the k-NN engine configured so that filtered vector queries — the kind every real query here needs — are supported.

The document index (routing)

Each document gets a single routing vector, embedded from a synthetic blob that concatenates the document's short summary, topics, the kinds of questions it can answer, its key entities, and its type. The routing text is deliberately built from what the document can answer, not its full text, because the routing step's job is to decide which documents are even worth searching for a given question.

The chunk index (hybrid)

Each chunk is indexed twice: a BM25 lexical field over its embed_text, and a k-NN field over its embedding vector. Keeping both is intentional. Dense vectors capture meaning and paraphrase; BM25 captures exact terms — a case number, a defined term, a statute reference — that a dense model will happily blur past. Legal and financial documents are full of exactly the tokens lexical search is good at and dense search is weak at, so we run both and combine them rather than betting on one.

Stage 3 — Query: route, then retrieve, then fuse

Answering a question is a two-stage retrieval, entirely before any generation:

1. Route. Embed the question and run a single k-NN search over the document routing vectors, scoped to the workspace, to select the handful of documents most likely to contain the answer. This is one vector search, no model call.
2. Retrieve. Within only those documents, run the hybrid search: BM25 and k-NN over the chunk index, each returning a ranked list.
3. Fuse. Combine the two rankings with Reciprocal Rank Fusion (RRF, constant 60) computed in application code, not by a server-side search pipeline. RRF rewards chunks that rank well in either method without needing the two score scales to be comparable. A near-duplicate pass then drops chunks whose normalized text repeats one already kept — overlapping windows with different offsets — before truncating to the final set.

The result is a small, numbered set of passages, each still carrying its document, pages, and headings from Stage 1. Those numbered excerpts are what the model sees. Doing fusion and dedup in code, rather than in the search engine, keeps this logic unit-testable without a live cluster and keeps the ranking rules in one place instead of split across infrastructure.

Stage 4 — Structural citation: the model points, the code cites

This is the stage that makes the whole thing verifiable.

The retrieved passages are presented to the model as a numbered list. The model is invoked with a forced tool call whose schema requires it to return, among other fields, used_excerpt_numbers: the list of passage numbers it actually relied on. It does not write “[3]” into its prose and hope we parse it correctly; it returns a typed list of integers as structured output.

The system then builds the citations by mapping those integers back to the exact retrieved chunks — by enumeration and filtering, never by trusting the number as a raw array index. An out-of-range or nonsensical number is silently dropped rather than crashing or pointing somewhere wrong. Because the citation is constructed from the passage the retriever surfaced, it inherits that passage's real page and section provenance from Stage 1.

The consequence is a hard guarantee: a citation the user sees always resolves to a real, indexed passage from their own document. The model is treated as neither a trusted citation formatter nor a trusted index generator — only as a component that selects from options the system controls. The same forced call also returns a sufficient_context flag and an updated running conversation summary, so one model call does answering, self-assessment, and memory in a single structured response.

Refusal is a first-class path, not an afterthought

A grounded system has to be able to say “I do not see this in your documents.” We make that cheap and reliable in two ways. First, if routing or retrieval come back empty, the system returns a deterministic “nothing relevant” answer with no model call at all — no cost, no opportunity to hallucinate, and a trivially testable branch. Second, when passages are found, the sufficient_contextflag lets the model judge, over the real excerpts, whether they actually support an answer, so a weak-but-nonempty retrieval is declined gracefully rather than embellished. The model can only ever cite what it was handed, which means the worst case degrades toward “not enough here,” not toward a confident invention.

The same contract for document actions

Studio actions — summaries, FAQs, quizzes, pro/con analyses, contract breakdowns, drafts — run the same forced-tool-call, structured-output discipline against a document's own parsed markdown. The output is a typed object with named fields, and for legal-adjacent actions a disclaimer is always appended by the system after validation rather than left to the model. Analysis is held to the same standard as chat: it is a structured read of a real document, not free-form commentary about it.

Isolation: grounded in your passages, and only yours

“Grounded in a real passage” is only meaningful if it is yourpassage. Every retrieval layer — document routing and chunk search alike — filters by workspace at the query itself, and the API verifies workspace membership before any of it runs. There is no code path that returns another workspace's content, so a citation can never resolve to a document the asker was not entitled to see.

Evaluation

We measure the system on the properties this design is meant to guarantee: attribution integrity, retrieval recall, and refusal calibration, plus a latency budget. Two of these are structural properties of the architecture; the rest are empirical and corpus-dependent.

The figures marked ^* below are illustrative targets — the shape of results our evaluation methodology (defined beneath the table) is built to produce on a small legal and real-estate corpus, shown to convey how the method behaves rather than as measurements from an audited or public benchmark run. Absolute numbers move with the corpus and the question distribution; the methodology is the point, and it is what lets these be reproduced — or replaced — with numbers from your own documents.

The two “by construction” rows are not measurements — they follow from the architecture. Every citation is built from a retrieved chunk, so it necessarily resolves to an indexed passage; every retrieval is workspace-filtered, so a citation cannot cross the tenant boundary. The interesting empirical result is the recall gap: fusing lexical and dense retrieval recovers passages that either method alone misses, and the lift is largest on exact-term queries (defined terms, section numbers) where dense retrieval underperforms.

Methodology

Build an evaluation set of (question, gold-passage) pairs over a fixed corpus. For each question: (a) record whether the gold passage appears in the retrieved set for hybrid, dense-only, and lexical-only configurations — that is recall@k; (b) run the full pipeline and, with a separate grader model, check each factual claim in the answer against the passages actually cited — that is claim attributability; (c) include a held-out slice of questions the corpus cannot answer, and measure how often the system declines — that is refusal recall. Attribution and isolation are asserted, not sampled: verify in code that every emitted citation index maps to a retrieved chunk and that every retrieval carries the workspace filter.

Latency budget

End-to-end latency is dominated by the single generation call, not retrieval. A typical request spends a few milliseconds embedding the query, tens of milliseconds on the routing and hybrid searches, negligible time on in-process fusion, and the remaining multiple seconds on the model call itself. Because retrieval is cheap relative to generation, the system runs it eagerly and lets the model be the final judge of sufficiency, rather than gating hard on retrieval scores and risking a wrong refusal.

Where this sits relative to prior work

Provenance-anchored generation is not a new retriever; it is a discipline layered over standard components. Classical RAG established retrieve-then-generate. Dense retrieval and hybrid dense/lexical fusion are well studied. Recent work on retrieval as a corpus-discriminative action — for example Meta and Rice's Superintelligent Retrieval Agent (SIRA, 2026), which enriches documents and queries with predicted vocabulary and folds multi-round search into a single weighted lexical call — points at how much headroom remains in the routing and retrieval stages alone. Our contribution is orthogonal to those advances and composes with them: it is the insistence that, whatever retriever produces the candidate set, the citation the user acts on is a structural pointer into that set, not model prose. A better retriever makes our answers better; it does not change the guarantee.

Limits, stated honestly

• Retrieval can still miss. If the gold passage is not in the retrieved set, the system will decline or answer from what it has — it will not fabricate a citation, but it can be incompletely grounded.
• Provenance is only as good as the parse. A badly scanned page or an exotic layout can degrade page/section attribution upstream of everything else.
• Routing uses a lossy per-document summary vector. It is a filter for which documents to search, never the basis for an answer — answers always come from real chunks — but an over-tight routing threshold can wrongly exclude a relevant document, so we keep it deliberately permissive and let the model judge sufficiency downstream.
• The empirical figures above are corpus-dependent. The structural guarantees (attribution, isolation) hold regardless; the recall and attributability numbers should be re-measured on your own documents.

Why it matters

The difference between “the model said so” and “here is the sentence it came from” is the difference between a tool you spot-check and a tool you can rely on in front of a client. By making the citation a structural fact rather than a generated claim, provenance-anchored generation turns every answer into something you can verify in a click — which is the whole point of a source-backed system.

See how we handle security and isolation, or try it on your own document.