This article is protected. Printing and print-to-PDF are disabled.Read the original at https://arincortex.hashnode.dev
Arin/ tech blogs

Chapter 07 of 16 · 4 min · 44% through

Context quality is part of intelligence

Contents

It is tempting to treat gathering material as the easy part — point the system at things, let it absorb them, worry about clever answers later. That order is backwards. Everything downstream is built on what came in, and material that carries no meaning does not sit quietly in a corner. It actively degrades results, because a fragment with no meaning can appear to match almost anything. Junk is not neutral. It is a small, permanent tax on every future answer.

For a period, a meaningful share of one production corpus fell short of that basic bar — content thin enough that it embedded without ever meaning anything. The fix that lasted was not another cleanup pass. It was moving the rule to the one place every path into the corpus has to cross.

No

Yes

Crawled pages

One gate:

does this carry

real meaning?

Your own uploads

Scheduled digests

Repair passes

Turned away

at the door

Stored, embedded,

made searchable

Fig. 6.1 — One door, and it applies itself, regardless of where the material came from.

A rule that depends on being remembered is a rule that will eventually be forgotten.

A second, more contentious rule sits beside it: not everything that is text is knowledge. Cookie banners, navigation menus and link lists are all text, and all of it embeds beautifully while poisoning a corpus with passages that answer questions about a website’s footer. Filtering that furniture out shrank one corpus by roughly a third — a deletion large enough that it was proposed with numbers attached, argued about, approved, and only then run against a verified backup. Owning the context is not only about where material lives. It is about what is let into it, and enrichment — the more expensive work of inferring relationships — is deliberately kept separate and metered, because there is little point paying to understand material you will later decide to delete.

Three readings on the panel

Open the system and there is no cheerful blank box waiting for a prompt. There is a panel, and it tells the truth about the machine underneath it, because every figure on it is read live rather than remembered from the last time someone looked.

Usage — Per provider, per model. Every call lands in a ledger, so the cost of a habit is something you can look at rather than discover later.

Corpus health — A claim the system re-derives rather than trusts — how much is stored, and whether all of it is actually searchable.

Enrichment — A separate, deliberate expense. Reading a document is cheap; working out how its ideas relate to everything else is not.

None of the three figures is decorative. Usage keeps spending visible instead of surprising. Corpus health keeps the retrieval promise honest — a claim like “fully searchable” is worth nothing if nobody checks it. And treating enrichment as its own metered line item is what keeps a person, rather than a schedule, deciding when the system spends money on understanding what it already stores.

The filtering decision, in the open

Deciding that a large share of a corpus is furniture rather than knowledge is not a decision to make quietly. Cookie banners, navigation menus and link lists are technically text, and technically text is exactly what the storage layer is built to keep — which is precisely the problem. None of it answers a real question; all of it can be made to look, superficially, like it might.

Removing that furniture once shrank a corpus by roughly a third. A deletion at that scale, against material a person is relying on, is not something a system should simply decide to do on its own initiative overnight.

1 · PROPOSE

The scale of the change is stated up front, in numbers, before anything is touched.

2 · ARGUE

The reasoning is contestable — a person can push back on what counts as furniture.

3 · APPROVE

A person, not the system, gives the go‑ahead for an irreversible change of that size.

4 · VERIFY

The deletion runs only against a verified backup, so the decision remains reversible even after it is made.

The lesson generalises past this one cleanup: the size of a change to owned material should decide how much process it earns, and a system that can act on a person’s behalf should treat large, irreversible edits as something to propose, not something to simply do.