10:21Jack Roberts · Tutorial
Claude Code + Graphify = Insane Agentic OS
How a knowledge-graph layer cuts re-reading costs and wires every agent to one shared brain.
June 8thThe argument in one line.
Hermes knows you but not your world — and closing that gap with a self-auditing LLM wiki governed by Karpathy's ingestion protocol turns a personal AI into a compound knowledge system that improves automatically.
Read if. Skip if.
READ IF YOU ARE…
- You're already running Hermes AI agent and want it to draw on meeting notes, email threads, or research without manual copy-paste.
- You want a knowledge base that fact-checks itself and flags contradictions as new content comes in — not just a folder of files.
- You're curious how Karpathy's LLM wiki concept maps to a real Claude Code / Hermes workflow with actual file structure.
- You use Granola, NotebookLM, or any meeting recorder and want those outputs feeding into one unified agent memory.
SKIP IF…
- You haven't set up Hermes agent yet — this assumes a working installation and skips the basics.
- You want a cloud-only or no-code solution; the wiki lives on your local filesystem and requires an IDE to manage.
The full version, fast.
Hermes agent's memory loop only captures what you've said in conversation — your inbox, call transcripts, and expert articles are structurally invisible to it. This tutorial adds an LLM wiki (a local Obsidian vault governed by a CLAUDE.md ingestion protocol) as a second memory layer: every new file triggers a read-discuss-write-link-contradict cycle, so the corpus audits and strengthens itself automatically. Wire Hermes to the vault as a named skill, add cron jobs to auto-ingest daily meeting notes, and you get a bidirectional system where Hermes reads from the wiki and writes back to it — giving both Hermes and Claude Code access to the same growing world knowledge.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Create a free account →Where the time goes.
00:00 – 00:46
01 · Hermes' blind spot
Hook establishes Hermes as the best-memory agent but reveals the structural limitation: it only knows what you've said.
00:46 – 02:39
02 · How Hermes memory works
Explains the notice-file-recall loop with a custom illustrated diagram. Sets up why the loop is one-directional and what falls outside it.
02:39 – 03:57
03 · Karpathy's LLM wiki concept
Introduces the self-rewriting wiki: raw in, audited knowledge out, contradictions flagged automatically. Credits Karpathy's background.
03:57 – 05:35
04 · Setting up the wiki
Scaffold the vault by feeding the LLM wiki URL to Claude/Hermes. Shows resulting file structure in Anti-Gravity IDE.
05:35 – 07:43
05 · CLAUDE.md ingestion protocol
The call.md file governs the ingestion cycle: read → discuss → write source page → update links → flag contradictions. Live query demo.
07:43 – 09:33
06 · Connecting wiki to Hermes as a skill
Add vault path to Hermes, build an LLM Wiki persona/skill, verify Hermes reads answers from the wiki and reports its source.
09:33 – 11:41
07 · Ingesting external knowledge
Live demo: paste George Mack's 'High Agency in 30 Minutes' article into Hermes, index it into the wiki per call.md, then query it from both Hermes and Claude Code.
11:41 – 13:53
08 · Wiring memory both ways
Shows bidirectional flow: Hermes reads wiki AND pushes distilled conversations into it. Sets up daily Granola cron job to auto-ingest meeting notes.
13:53 – 14:29
09 · CTA and next steps
Points to NotebookLM integration video and next Hermes capabilities video.
Lines worth screenshotting.
- Hermes agent's memory loop covers only conversation — emails, meeting transcripts, and expert articles are structurally invisible to it by design.
- Karpathy's LLM wiki audits itself: every new file triggers a read-discuss-write-link-contradiction-flag cycle, so the corpus gets more accurate over time, not just larger.
- The CLAUDE.md ingestion file governs how the wiki behaves — the quality of your knowledge base depends entirely on how well that constitution is written.
- Hermes and Claude Code can point at the same Obsidian vault simultaneously, giving both interfaces access to one shared knowledge layer without duplication.
- Wiring memory both ways means Hermes can crystallize a big conversation directly into the wiki — personal memory and world knowledge compound together.
- Cron jobs set inside Hermes agent can auto-ingest daily meeting notes from tools like Granola, turning the wiki into a passive accumulation system that runs without manual effort.
- NotebookLM outputs can be ingested into the wiki directly, bringing synthesized expert knowledge from Google's research platform into Hermes' reach on demand.
- The skill/persona layer in Hermes gates wiki queries to the right context — the LLM Wiki skill fires for strategy and meeting questions without polluting general chat memory.
Your AI only knows what you've told it.
WHAT TO LEARN
Conversation memory and world knowledge are different systems — and bridging them requires a vault that audits itself, not just a folder of files.
- AI agents that learn from conversation still have a structural blind spot: anything that didn't come through chat — emails, calls, research — is invisible to them by default.
- A self-auditing knowledge base isn't just storage; the ingestion protocol (the CLAUDE.md file) determines whether new knowledge integrates cleanly or just accumulates noise.
- When two knowledge systems share the same vault, consistency becomes automatic — the wiki's contradiction-flagging cycle does what a human curator would do, but on every ingest.
- Bidirectional memory closes the loop: the agent reads from the external knowledge base and writes distilled conclusions back into it, so both layers compound together over time.
- Automating ingestion via scheduled jobs — daily meetings, weekly reading — turns a knowledge base from a project into an infrastructure layer that improves without active maintenance.
- The same vault accessible from multiple interfaces (Hermes, Claude Code) eliminates the context-switching tax of keeping separate notes per tool.
Terms worth knowing.
- LLM wiki
- A local markdown vault where an AI model maintains, links, and audits its own knowledge files — sourced from Andrej Karpathy's concept of a knowledge base the model rewrites as it grows.
- Hermes agent
- A personal AI agent that builds a durable memory of its user across conversations, storing notices and durable facts in memory.md and user.md files it can later recall and search.
- Obsidian RAG
- A retrieval-augmented generation setup where an Obsidian vault of markdown notes serves as the external knowledge store an AI queries at inference time.
- CLAUDE.md (call.md)
- The ingestion-protocol file that tells the AI model exactly how to process new content: read it, write a source page, update linked pages, and flag contradictions.
- Granola
- A meeting-recording app that lets you chat with transcripts of your own calls; used here as a source for auto-ingesting meeting notes into the LLM wiki via a daily cron job.
- Anti-Gravity IDE
- A code editor environment referenced in the video for visualizing and editing the Obsidian wiki file structure locally.
Things they pointed at.
02:39linkKarpathy's LLM wiki
09:33linkGeorge Mack — High Agency in 30 Minutes
13:53channelNotebookLM integration video (Jack Roberts channel)
Lines you could clip.
02:11
“Hermes knows you. It doesn't know your inbox.”
03:08
“The wiki that rewrites itself — the more you add, the better it gets, it audits itself.”
11:51
“We've wired the memory both ways. Hermes can read the wiki and Hermes can write to the wiki.”
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
metaphoranalogy
00:00Hermes has the best memory of any AI agent. It remembers your conversations and gets smarter the more that you use it.
00:08But there's one thing it can't do. And once you fix it, you give Hermes superpowers and unlock completely new capabilities.
00:15So in this video, I'll show you exactly how to fix that to give Hermes a self improving knowledge base based on Andre Kapathi's LLM principles. What will save you hours of time and get you light years ahead of everybody else.
00:29And if you're new, I'm Jack. I built in some analytics startup with a gazillion customers, and now I run my own AI businesses. And here, I just share the stuff that actually works.
00:38So if you haven't ready, grab that beautiful coffee, coffee and let's dive straight in. Now most LLMs feel like they have amnesia. They just forget things randomly, convenient, and they don't even know that they've forgotten them.
00:49This is why Hermes is so good. And before you can understand its limitation and how this actually adds value, And if you stick with this video to the end, you will have new capabilities in your Hermes agent that will feel completely different and get you further ahead.
01:02But to do that, we have to understand what is Hermes memory and how does it work. So the best way to think about it is that Hermes gets better the more that you physically use it. And it has this loop.
01:13That is like the top level way of doing it. I've I've drawn some things you can take pause and have a look at if you want to. Fundamentally though, you have a conversation with Hermes agent, it writes a note that goes into memory.md or user.m d, and then it recalls it when you're having conversation.
01:27This works on notices, you know, pulls durable facts out of ordinary chump, it works on files, and it can also recall and search those things before it answers you. But there's one catch, one limitation to this Hermes system.
01:41And that's the fact that it knows only what you've said. So your inbox, your calls, your docs, your research, those beautiful YouTubers you followed, all this great knowledge is invisible to Hermes.
01:52And that's its blind spot. It knows you super well and I freaking love Hermes. I talk about Hermes all the time, but it doesn't have all of this additional context.
02:00It doesn't know, you know, if you're trying to grow on Instagram or YouTube or you're trying to scale your email marketing system, it doesn't have that knowledge yet. It knows you very well but doesn't have everything to the right hand side of the wall. If you think about it this way, Hermes knows you but it doesn't know your email threads, meeting notes, or notebook outline.
02:16It can of course call those when it's asked upon but it's not in it if you like it's conversational memory in that sense. Said another way Hermes knows you, it doesn't know your inbox. And so the idea to solve this is we want a knowledge base that grows by itself that we can store all of this information that Hermes can call and reference whenever it wants to.
02:34And it can also do very interesting capabilities that you've not seen discussed anywhere on this particular topic, and I'll show you what I mean by that. So if you look at Andre Kapathi, by the way, co founded OpenAI.
02:43If you don't know who's at TLDR, he led AI at Tesla. He's a fan of Eureka Labs, and even coined expression, vibe coder. Okay?
02:50So he he's pretty well known in the space. Now his idea here was an LLM wiki. You might have seen it as referred to as Obsidian Rag.
02:57And its core idea with Hermes agent is that it rewrites itself as it grows. So we can take all of the information that we want to write, like, again, expert commentary, things that we build in Notebook Alam, and effectively what it will do is the more we add to it, the better it gets.
03:11As you add it, it checks linkages, it fact checks things, and it basically creates an ever growing self referential, for anything that you want to learn about or any historical context you have.
03:24And so if you think of it like this, Hermes knows you, the wiki knows your world, and now when we combine this with Hermes agent, your agent can read both and basically do very cool things when we wire these two things together, your world and you. Think about it from that point of view. So now we understand the limitations of Hermes memory system.
03:41The first thing we need to do is get this beautiful LLM Wiki set up so we can actually reference this within Hermes and expand its working knowledge so it can answer questions about essentially anything that you want.
03:53And by the way, once it's set up, you can even visualize and see your entire LLM Wiki Obsidian system all within your dashboard. It is so freaking cool.
04:01So the first step we do is head over to this LLM Wiki. Effectively, what it is is explaining in detail, basically architecture works, how the system works, and it's got all of the detail here, which is freaking awesome. So all you're gonna do is come down and grab this URL, then you can open up your language model of choice.
04:17You can do this in Hermes, but we just had Opus 4.8 drop, which is the world's most capable model as of today. I've got some interesting thoughts on that.
04:25So we can use the OAuth inside the Claude app. What that basically means is we don't have to spend any API credits if we're just using the app and you have the subscription.
04:35Alternatively, you can just open up Hermes and it can follow these exact same instructions without a problem. But for the purposes of this, what I'm gonna do is basically say, hey there, I want you to familiarize yourself with the idea in this URL, and I want you to create for me a desktop folder with all the requisite files and let me know once that is complete.
04:53And then we come down and we simply paste this one in here. And effectively, it will kind of auto set up based on the instructions that exist then. Now since I already have one set up, it's found that I saved it as Obsidian Wiki.
05:04And effectively, what it will do is build out a structure that looks like this. Just so you can visualize it a little bit easier, I'm gonna pull up the anti gravity IDE, not two point o, I might add.
05:13It's anti gravity IDE, just so you understand what this physically looks like. And as you can see, I've added various different things here, like transcription, various different YouTube videos that I think are interesting about, you know, different strategies for how to make better content and that kind of thing. And all of it is within there itself.
05:28And you can ask it questions if you want to. So for example, if I'm querying this wiki, one of the things I might do, and as you can see down here, if you come to call.md, you can see it's got the whole structure out laid here.
05:38I'll put this call.md down below for you as well so you can use this as an additional thing. But you can see it's got an ingestion workflow. So when a new file is added, it's gonna read it, it's gonna discuss it, it's gonna write the source page, okay, and it's gonna update any affected pages and essentially flag any contradictions.
05:54So the idea is, like, if you add a new piece of knowledge and it contradicts for something else, we identify those contradictions and the knowledge base gets stronger over time. So that might say something like, hey there, dude. Could you just give me two quick tips for making better intros?
06:07Okay. And you can actually go ahead and search the Obsidian Wiki. It works exactly the same in cold code.
06:12By the way, if this all sounds like I'm speaking Mandarin, you can watch this video here for a full breakdown on exactly what it all means to build in core code and anti gravity. But for the purposes of this, I'm basically just explaining how it's all connected. Again, you can also do this in Hermes agent.
06:26It will set everything up for you. I'm just showing you in this environment so you can see it visually. And as you can see, drawing a YouTube content strategy synthesis in your wiki.
06:35Here two quickly actionable tips for letting the intros. There you go. And it's basically found that information from our wiki.
06:42Now the cool thing here is that we can also ingest anything that we want to. I'm gonna show you exactly actually how you can take this to a level by essentially doing all of this within Hermes. But to do that, the first thing we have to do is basically connect this file, this desktop where your personal wiki is going to live, And we need to share that with Hermes.
07:01So if you're using this dashboard, all you're gonna do is let you come down here and you'll see the instructions on the left hand side here for the Obsidian Wiki. But basically, you just disconnect this and it will you can just enter in the location of where your vault is and then literally just copy this information here.
07:15Now if you don't have, okay, the actual location, you can just say to Claude or anyone on your laptop, hey, what is the URL location of this?
07:24And once you've done that, you can then connect it. So for example here, I'll just gonna say, hey there, I would like you to reference my, basically, my LLM Wiki, my Obsidian Wiki. Effectively, this is gonna have information that is external to you.
07:37So you know everything about me. This is gonna be a source of information knowledge that you can query in our conversations to better answer my questions.
07:45This is gonna be things like my meeting transcripts, external insights from experts that are gonna help you basically make better decisions. Okay.
07:54And what I think we can do here, if you haven't already, and I recommend you this, is build out what we call, like, a kind of LLM Wiki skill or an Obsidian skill. And effectively, can do this if using personas here. That's freaking awesome.
08:04I've covered that in-depth on the channel, how you can just basically add in anything you want to here. For instance, by clicking on persona, coming down, picking the person, naming it here, and this would be something like, I know, LLM Wiki. And here would be use this skill when answering any questions about strategy or any context about meetings, notes, that kind of thing.
08:22And then just basically build out a system prompt, and then you can pick the model that you wanna use to build that out. Now interestingly here, you can actually then, if you just go ahead and update this and give this prompt over to Hermes, it will be fully up to date. You can also within Hermes, if you want to, just effectively ask them to build up that skill.
08:37I just like to do this personally because I like to visualize and see all the skills so I know exactly what I want to improve and how I want to physically go about it. So for example, if I just clip a chat, I can come down and test this. Hey there.
08:47Basically, go to my LLM Wiki and just tell me, for example, three tips on how to make a better intro. Okay. So we can do this, give that to Hermes.
08:56And now what Hermes is doing, crucially, is consulting this ever growing self referential, self improving database of knowledge. Here you go. You come down here and it's basically consulted this, and I can say, hey, can I just confirm where did you get this information from?
09:10And as you can see, I already can see here it read the file exactly where my Obsidian Wiki is, and it found the information from there. And look, it's gone ahead and grabbed it. So effectively, what we've done now is we've got our own individual LLM Wiki that contains everything about all the calls that we've had, maybe it contains things about knowledge from experts.
09:28And then we have Homi's own internal memory system which is by itself incredible and we combine these two things. Now to build out your personal LLM Wiki, when would you use that?
09:37Well, let me give you an example. Let's say for example, you are searching the internet and you come across a website like this. This is high agency in thirty minutes, which is a beautiful article by way, highly recommend it.
09:47What you can do now is basically have this index. So let's say for example that you really like the principles of high agency, what it means to be high agency and how much it's very valuable. And you kind of want Hermes to understand this stuff when you're talking to it.
09:59Like, maybe you wanna grow on LinkedIn, and therefore you want a LinkedIn master article to be in Hermes memory. Well, what we can do really now is we can copy the text or copy the article. We can go straight over to Hermes.
10:10Essentially, what we can do guys is drop it in and be like, hey there, I want you to go ahead and basically index this article into my Obsidian Wiki, which means then that it will reference this and understand this whenever we physically wanna go ahead and use it. And look, it will just gone down. It's checked it out.
10:23This is George Max, high agency in thirty minutes. Quick tell the offer on the page. Before I create the source page, what do you want emphasized?
10:30So I'm gonna say, what I'd like to do, bro, is basically embed it into the wiki in line with the principles set out in that claw.md, and just kind of index it as part of that. And then from that, it's actually gone ahead and completely indexed it, so I can now ask it questions about high agency and I'll do that.
10:45For example, what are the criteria of a high agency person? Okay. I'm gonna send this one off, and it will now reference that Obsidian Wiki to pull that information.
10:52As you can see, high agency people are spotted by four main signals, weird teenage hobbies, treadmill energy, unpredictable opinions, blah blah blah. So the idea here is if you're just talking to Hermes agent, it will remember stuff. Remember, Hermes' memory is incredible.
11:04You can even ask questions. What did I talk to you about on the April 15? And it will be able to pull those specific things.
11:10The difference here with the wiki is this is a very large knowledge base where we're indexing, and it's a growing corpus of knowledge of things that you find interesting from experts, which Hermes can now reach over, grab, and access. And the cool thing is, if you're ever working in cold code, it has access to the same knowledge.
11:25So I can say, hey there. Do me a favor. What are the spokes of a high agency person?
11:29This will be visible in my Obsidian Wiki. Now as you can see, it's from the scale Obsidian ask. It comes down.
11:35It's found it, which is awesome. And look, it's given us all the information. Despokes from me I'm thinking about blah blah blah.
11:40So now essentially Claude and Hermes agent have access to the same memory. And this is part of the idea of the operating system that we universalize this knowledge and everything's connected in one system rather than having a thousand different things and a thousand different interfaces to use.
11:53And so one of the other cool things that we can do here, and if you think about what we've done here, we've wired the memory both ways. So we can access this LLM Wiki and we can auto ingest files and data into this Wiki without polluting the memory of Hermes agent, which I'm gonna show you in a second. It's very, very cool and we have this beautiful two way system giving Hermes this kind of access, this whole new external system, external knowledge.
12:16And so if you think about it, we've got this great bidirectional relationship. Hermes knows everything about you.
12:21And then we have this LLM Wiki where Hermes itself can just send things over there. Like, if you have a really big conversation and you really wanna crystallize that knowledge, you can essentially just say, hey, index this into my Wiki.
12:32So it grows this growing corpus of knowledge. Equally, anything from calendars, Gmail, meetings, we can just set up auto, basically, automatic cron jobs, also known as just like things that run-in the background to save that.
12:44And we can do that all within Homey's agent. For example, I could say, hey, what I'd like you to do is set up a recurring task on every day that goes through to see if I had any new meetings that are shown in granola.
12:58And if I have, I'd like you to add those please to my wiki under kind of like meeting notes and meeting information. Okay? Go ahead and save that one off over there, which is really cool.
13:08Now Granolah is just the app that I'm using to basically record meetings that I go into. They're not like a sponsor of the video or anything like that. But they're really cool because it just basically lets you chat to all of your meetings.
13:18And now I'm gonna have this database, all this knowledge, I can just reference whenever I need to. And they go ask us a question. So I could say, hey.
13:24Why don't you go ahead and just run it at 9AM every morning? Then I'll say, that sounds like a great place to store it. Just add it wherever you think would be most appropriate inside that database.
13:32And just like that, it's now created a job. So essentially, every single day now that will run, it will check on my meetings. If I've got any meetings, it's now added to my LLM Wiki, which is Kapathi's whole memory system in a nutshell.
13:42And then we have access to that in Hermes whenever we want to. And the coolest thing about this guys is you could do this with Notebooks. Like, you can connect Notebook LM to Hermes Agent.
13:50I'll put a video on screen if you haven't seen that. All CluedCode. Get like, you know, 50 to a 100 different areas of, like, expert knowledge from Google's number one research and intelligence platform, I can ingest it in my local wiki on my computer and then Hermes can just reference that whenever I want to with this skill.
14:07And Hermes itself can create automations to ingest it, so your knowledge base just grows exponentially and will just never forget the stuff that it needs to know. Now memory is great, but it's only one part of the puzzle. You need to understand how to leverage all of the different aspects of Hermes if you wanna get the full capabilities.
14:24So the next thing I'm gonna do is learn those capabilities by watching this video right here.
The Hook
The bait, then the rug-pull.
Every AI agent remembers what you say. None of them remember what you read. This tutorial closes that gap — wiring a self-auditing knowledge base into Hermes so your agent can see your inbox, your meeting notes, and the expert research you've been hoarding, not just the conversations you've had.
Named ideas worth stealing.
02:39model
Karpathy's LLM wiki (Obsidian RAG)
- Raw immutable source dump
- LLM writes new entries
- CLAUDE.md schema it follows
- LLM Wiki auditor reads over files
- Not RAG — plain markdown, zoned and curated
A self-auditing local knowledge base where raw content comes in and the model rewrites, links, and contradiction-flags its own files as it grows.
Steal forAny workflow where you want an AI to maintain its own growing reference library from diverse inputs
11:41model
Bidirectional Hermes memory
- Hermes personal memory (conversation → memory.md)
- LLM wiki (world knowledge, external docs)
- Hermes reads wiki on demand via skill
- Hermes writes distilled conversations to wiki
- Cron jobs auto-ingest from meetings/calendars/Gmail
One super memory wired both ways: Hermes' internal loop covers personal facts; the LLM wiki covers world knowledge; both sides can push and pull from each other.
Steal forDesigning AI agent memory architectures where personal context and domain knowledge need to coexist without polluting each other
How they asked for the click.
VERBAL ASK
13:53next-video
“Memory is great, but it's only one part of the puzzle. You need to understand how to leverage all of the different aspects of Hermes if you want the full capabilities. So the next thing I'm going to do is learn those capabilities by watching this video right here.”
Soft inline CTA — no subscribe ask, just a next-video card. Fits the educational tone but misses an opportunity to capture email or subscription intent.
MENTIONED ON CAMERA
FROM THE DESCRIPTION
PRIMARY CTAWhere the creator wants you to go next.
OTHER LINKSAlso linked in the description.
Visual structure at a glance.
