This article is protected. Printing and print-to-PDF are disabled.Read the original at https://arincortex.hashnode.dev
Arin/ tech blogs

Chapter 04 of 16 · 5 min · 25% through

Knowledge becomes a map

Contents

Searching by resemblance answers one kind of question extremely well: which of the things I have kept looks most like what I am asking about? That is enough whenever the answer sits inside a single passage of text.

But a great many useful questions are about relationships. What does this depend on? What does it produce? What came before it? What contradicts it? Those answers are often not written down in any one place — they exist across documents written months apart, and nobody ever wrote the sentence that joins them. This is why the corpus is not just stored; it is enriched. Every passage becomes a candidate source of concepts, and of the relationships asserted between them, each one kept tied back to the passage that supports it.

A library

things, filed

Good at:

find this again

A map

things, and how they connect

Good at:

how does this reach that?

Fig. 3.1 — Two ways to hold the same material.

Screenshot — the map, across the whole collection. Every point is a concept; size is connectivity, colour is cluster.

Screenshot — the same map, closer in. Up close, individual concepts and connections resolve.

The scale is real: at one recorded snapshot the graph held over two hundred thousand nodes and more than a million edges, distilled into tens of thousands of concepts joined by tens of thousands of asserted relationships — part-of, depends-on, produces, contrasts-with, and several kinds besides. A collection that size cannot be held in a person’s head, and a list of its contents would not help. What is useful is the shape: where the material is dense, where it is thin, and which subjects turn out to sit next to each other.

One design choice matters more than the map itself: the graph does not automatically get a vote in an ordinary lookup. Similarity and keyword matching ask whether a passage answers the question. Graph connectivity asks a different question — whether the passage sits near the concepts the question named. Those are different kinds of evidence, so the graph is consulted only on the relational route, where a traversal is genuinely the point of the exercise.

A network diagram looks authoritative whether or not it is right. So the map is treated as something extracted from the material — inspectable, and open to dispute — never as a database of settled fact. And when there genuinely is no connection, that is the answer. A system that can say no path found is worth more than one that always produces a sentence.

Eleven kinds of relationship

“Related to” is the easy relationship to extract and the least useful one to have. It is true of almost everything, which means it helps distinguish almost nothing. The graph works with a small, named vocabulary of relationship kinds instead, so that a traversal can say something more specific than that two ideas happened to appear near each other.

  • Part-of — belongs to a larger wholeDepends-on — requires something else to work
  • Produces — results in, or generatesExemplifies — is a concrete case of an idea
  • Is-a — belongs to a categorySolves — addresses a named problem
  • Contrasts-with — differs from, in a way worth notingCauses — brings something else about
  • Precedes — comes before, in time or sequencePrerequisite-of — must be understood first

Naming the relationship is what makes a traversal answer a real question. “What does the extraction step depend on” and “what does it contrast with” are different questions with different correct answers, and a graph that only ever says related cannot tell them apart. Each assertion still carries its source passage, so a labelled edge remains something a person can open and check — a claim, not a fact handed down.

A map that keeps growing

A passage becomes a candidate source of concepts and relationships the moment it is stored, but that first reading is not the only one it will ever get. The map is allowed to keep improving after the fact, on its own schedule, independent of when any particular document first arrived.

This matters because relationships are not always visible from a single document in isolation. A connection between two ideas may only become legible once a third, unrelated document arrives months later and supplies the missing link. A map that only ever looked at material once, at the moment it arrived, would never find that connection at all.

Document, month one

The map, as it stood then

Unrelated document,

month four

A later enrichment pass

revisits earlier material

The map, richer —

a connection neither document

alone would have shown

Fig. 3.2 — Enrichment revisits, it does not just arrive once and stop.

Because that revisiting work costs real money and real time — every enrichment pass is another set of calls to a model — it stays exactly where the rest of this book places metered work: a separate activity a person switches on deliberately, not a background process that runs unattended and unbilled. Growing the map on a schedule is a choice about how thoroughly to understand material already owned, weighed against what that understanding costs, and it is a choice that stays with the owner rather than the system.