For developers: the RAG I use, and how it compares

This page is for people who write code. It describes the retrieval my own second brain runs on every time Claude has to find something: what it does, the choices I made, what I measured, and where it stands against the usual approaches. For the theory of RAG itself, see the agentic RAG module of the agents course.

The principle: the files stay the truth

The database is derived from the Markdown files, never edited. Obsidian, the scripts and git read the files; to change the brain, you change a file. On each search, the engine compares file sizes and dates, reloads what changed, and recomputes the vectors of the changed passages only (each vector is stored under the hash of its text). A page written a minute ago is found. Losing the database costs a rebuild, not data.

The pipeline, step by step

1. Cutting into passages. A document is cut where it changes subject: at its headings, at code declarations, otherwise at blank lines once a passage is long enough. Each passage keeps the trail of headings above it, because a passage often does not say what it is about; its headings do.
2. Words: BM25, computed in SQL. Terms are normalised (accents removed, identifiers like camelCase split, stop words dropped, words truncated to a five-letter prefix as a cheap stem). BM25 (k1 = 1.2) is computed in Postgres, per document and per passage.
3. Meaning: local embeddings. Each passage is embedded with bge-m3 (1,024 dimensions), from its title, its heading trail and its text. The model runs in Ollama, in its own container, reachable on the loopback only. Vectors live in Postgres with pgvector; the nearest passages are found by cosine distance.
4. Links: the graph from module 1. Wikilinks, relative Markdown links and references: in the front matter form a graph. Scores spread from the matches to their neighbours by a random walk with restart, damped on hubs so that a page linked from everywhere does not win every query. A note next to a match comes up even if it holds none of the query’s words.
5. Fusion: two places for meaning, one for words. The two rankings are interleaved, two semantic results then one lexical, instead of the usual reciprocal rank fusion. The lexical third keeps the proper names and identifiers that vectors blur.
6. Several queries, written by the model. Claude sends three or four queries at once: the question, its translation, and the words the documents probably use. That is the cheapest way to cover paraphrases, even with vectors.
7. A token budget, not a number of chunks. The budget goes first to the best passage of each document, in order, then to the next passages. Every line keeps its number in the file, so each answer cites path:line and can be checked in one click.
8. Signals of absence. Each search returns how much of the query the best passage covers (words) and the best cosine similarity (meaning). On 44 cases from my brain, the 7 questions without an answer scored between 0.37 and 0.51, and nearly all the others above 0.52: below that, the answer is probably not there. It is a hint; the text of the passages decides.

What I measured

• On 16 paraphrased questions, recall is 88% with 8 documents and a budget of about 750 tokens, against 56% with 3 documents and a budget four times smaller. Cutting the budget too far costs more in missed answers than it saves.
• An eval command scores a test set without a model: recall, precision, whether a given quote reached the context, and whether absent answers are flagged. The rule: write the test cases before looking at the results, or you measure what the engine already does well.

What it costs to run

• Two Docker containers (Postgres + pgvector, Ollama). No published port, nothing leaves the machine. The text and the vectors are stored in clear in local volumes.
• Monitoring: every session starts with a health check. If a container is down, search falls back to words only, and says so. Twice, parts of this pipeline failed silently for months before I added that check.
• A pinned version of Ollama: in one version, bge-m3 returned NaN on the GPU as soon as a text exceeded about a hundred tokens. Embeddings that fail quietly are the hardest bug to see.
• A zero-dependency fallback: the same engine without the database, words and links only, reading the folder directly.

How it compares

Approach	Strength	Weakness	Against this method
Read the whole index (the Karpathy wiki)	No infrastructure, simple	Cost grows with every note; no paraphrase	Fine up to a few hundred notes, then routing and search take over
Everything in a long context window	No retrieval to get wrong	Paid on every question; quality drops with length	On my brain, removing the full wiki from one prompt took it from about 235,000 to 16,000 tokens
grep	Exact, instant	Dumps matching lines into the context; no synonyms	I use a term index that returns note names only, then targeted search
Vectors only, fixed-size chunks (the usual tutorial)	Finds paraphrases	Blurs names and codes; arbitrary cuts; no line citations	Cuts by headings, keeps a lexical third, cites lines
Hybrid with reciprocal rank fusion	The standard way to merge rankings	One more constant to tune	I interleave 2:1 because it is simple and keeps identifiers. I have not measured it against RRF.
GraphRAG-style (a model extracts entities and summarises communities)	Answers broad questions over a whole corpus	Costly indexing, done by a model; graph drift	Reuses the links already in the notes: no model at indexing time
Hosted vector database (Pinecone, Supabase…)	Nothing to run	Personal notes leave the machine; a dependency	Everything local, at the price of maintaining two containers

Words for names, meaning for paraphrases, links for context, files for truth.

If you build your own

1. Start without a database: files, theme map, links. Add search when Claude starts missing notes you know exist.
2. Write your test set first, with questions that have no answer.
3. Add BM25 before vectors; add vectors when the test set shows paraphrases failing.
4. Monitor from day one: a stopped container looks exactly like a search that found nothing.