← all video summaries
Personal knowledge bases

Build a Compounding LLM Knowledge Base in 5 Minutes with Claude Code

A folder of Markdown files, not a vector database. Walking through Andrej Karpathy's LLM wiki pattern — point Claude Code at your raw notes and it builds a linked, queryable second brain that compounds with every source you add.

Creator: Nate Herk (AI Automation Society)Runtime: 17 minVideo published: April 5, 2026558.8K views when summarized
Watch on YouTube
TL;DR

The short version

  1. Karpathy's idea: skip the vector DB. Give an LLM well-organized Markdown files with an index and links, and it finds answers by reading the index and following links instead of similarity search.
  2. The whole system is a vault with two folders — raw/ (you drop sources in) and wiki/ (Claude Code writes pages, an index.md, and a log.md) — plus a CLAUDE.md that defines the schema and search rules.
  3. Setup is one paste: copy Karpathy's llm-wiki.md gist into Claude Code, tell it to implement the idea, and it scaffolds the folders. Ingesting one article spawned ~23 linked pages automatically.
  4. It compounds. One X user reported a 95% drop in query token usage after collapsing 383 files into a wiki; the knowledge persists across sessions instead of dying with each chat.
  5. It doesn't replace RAG at scale — hundreds of pages is fine, millions of documents is where you go back to a real retrieval pipeline.

01 · The demo

A knowledge graph built from 36 videos

Thirty-six recent YouTube videos, organized into a graph where every node is a tool, technique, or concept and every edge is a real relationship. The instruction was one line: grab the transcripts and organize everything. No manual linking — Claude Code inferred the connections itself.

The Obsidian graph view. Hubs form around frequently-referenced tools; isolated dots are one-off mentions.00:00:17
What a node holdsEach dot is a page with tags, the video link, the raw transcript, a summary, and backlinks — so you can jump from a video to the WAT framework to Claude Code to every other video that touched it.

02 · The idea

Karpathy's wiki pattern, and why it compounds

The whole thing traces back to a post from Andrej Karpathy on building personal knowledge bases with LLMs. His own setup runs about 100 articles and ~400K words. The key admission: he expected to need heavy retrieval machinery and didn't.

Karpathy's note: he uses Obsidian as the frontend, but the data is just Markdown the LLM maintains.00:02:40
I thought that I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries of all documents, and it reads all the important related data fairly easily at this small scale.— Andrej Karpathy

The reason it matters: normal chats are ephemeral — the knowledge dies when the conversation ends. A persistent wiki makes knowledge compound like interest. No vector database, no embeddings, no chunking pipeline. Just a folder of Markdown.

03 · The architecture

raw in, wiki out, CLAUDE.md as the rulebook

Three moving parts. You drop sources into raw/ (read-only, you never edit it). Claude Code writes everything into wiki/ — the individual pages, an index.md table of contents, and a log.md operation history. CLAUDE.md tells the agent how the project works: how to search, how to update, what the schema is.

The file tree: my-wiki/ with raw/ and wiki/ underneath, index.md and log.md inside the wiki, and CLAUDE.md holding the rules.00:03:50
raw/

where you put stuff — treated as read-only source.

wiki/

where the LLM writes every page it generates.

index.md

the table of contents the agent reads first.

log.md

an operation history, appended on every ingest.

CLAUDE.md

the schema and search rules for the project.

04 · Setup

One paste from Karpathy's gist

There is no repo to clone. Karpathy followed up his post with the idea written out as a gist — llm-wiki.md — left deliberately vague so you can hack it to your own project. You copy the whole thing into Claude Code, add a one-line instruction to implement it as your second brain, and it scaffolds the vault.

Karpathy's llm-wiki.md gist — the idea file you paste in. It describes the wiki as a persistent, compounding artifact, not a per-query rebuild.00:07:14
  1. Install Obsidianoptional frontend to see the graph; the system is just Markdown without it.
  2. Create a vaulta new empty folder, opened in VS Code or your terminal.
  3. Paste the gistdrop Karpathy's llm-wiki.md into Claude Code with "implement this as my second brain."
  4. Let it scaffoldit creates raw/, wiki/, a CLAUDE.md, an index, a log, and starter folders.

05 · Ingestion

Drop a source, watch it fan out

To add a source, clip an article into raw/ (the Obsidian Web Clipper drops web pages straight in) and tell Claude Code to ingest it. It reads the article, asks what to emphasize and how granular to go, then chunks it its own way — one article isn't one page. Feeding it the AI 2027 essay produced about 23 linked pages: the source, six people, five organizations, an AI-systems page, concept pages, and an analysis.

Claude Code (Opus 4.6) working its ingest checklist — source summary, entity pages for people and organizations, concept pages — while the graph grows on the left.00:11:23
How long it takesThe single-article ingest ran about 10 minutes; the original batch of 36 YouTube transcripts took about 14. It does the heavy lifting, but you wait on it.

06 · Querying

Point an agent at the vault, with a hot cache up front

The wiki is just a folder, so anything can read it. You can ask questions inside Obsidian, or point a separate project at the directory and let it crawl the index. The executive-assistant build does exactly that: its CLAUDE.md carries a wiki_path and a retrieval protocol — check the hot cache first, then the index, then sub-indexes, and only read full pages when the task actually needs them.

The assistant's CLAUDE.md: "hot cache first," master index, domain sub-index, a page limit, and "don't read from the wiki unless you actually need it."00:13:56

The hot cache (_hot.md) is a ~500-word scratchpad of the most recent context — useful for an assistant, skippable for the YouTube vault. Switching the assistant from static context files to this wiki method dropped its token usage.

07 · The verdict

Does this kill RAG?

No, but kind of. Claude Code built a side-by-side comparison. The wiki finds info by reading indexes and following explicit links, so relationships are real rather than "these chunks seem similar." Infrastructure is just Markdown; cost is basically free beyond tokens; maintenance is running a periodic lint to fix inconsistencies and fill gaps.

The comparison table — Karpathy's wiki vs. semantic-search RAG across how it finds info, relationships, infrastructure, cost, maintenance, and weakness.00:16:06
RAG says "find me chunks that sound similar to this." [The wiki] says "here's a map — go read what you need and follow the connections."— Nate Herk
Where it breaksThe weakness is scale. Hundreds of pages with good indexes is fine. Millions of documents and the cost overtakes a traditional RAG pipeline or knowledge graph — at least with the models available as of April 2026.