I've spent the last few months tinkering with a local AI stack on my desktop — Ollama for local inference, Hermes orchestrating the whole thing, and a Claude subscription for the heavy lifting. It works. But there was always one nagging gap I kept dancing around: where does the AI's memory actually live?
For a while I went back and forth on whether to use Obsidian or Trilium as the knowledge store. I tried picking one. I tried picking the other. And then it finally clicked that I was asking the wrong question. They're not competitors. They do genuinely different jobs, and once I let them do those different jobs, the architecture got simpler, not more complicated.
So here's where I landed.
The Realisation: Two Kinds of Memory
The thing nobody tells you about "AI memory" is that it's actually two separate problems wearing one coat:
- Reference knowledge — the curated, stable stuff. Things I've written down deliberately and want to keep. This is read-mostly. The AI pulls from it, but it shouldn't be scribbling all over it.
- Working memory — the daily log. Decisions, scratch notes, "remember I changed X today." This is high-frequency, messy, and constantly being written to by both me and the AI.
The moment I framed it that way, the tool choice answered itself. I already use both apps for exactly these two purposes, so I'd been overthinking it.
- Obsidian holds my
.mdfiles — my curated reference library. Plain markdown on disk, portable, mine forever. - Trilium is where I dump my daily notes. It's a proper database with an API, which turns out to matter a lot for the working-memory job.
One is the library. The other is the journal. Don't make the library be the journal.
Why The Split Plays To Each App's Strengths
This isn't arbitrary. Each app is built for the role I'm giving it.
Obsidian being plain markdown means anything can read it — Claude Code just cds into the folder, and an in-vault RAG plugin like Smart Connections does local semantic search through Ollama without phoning anywhere. Brilliant for retrieval. But writing back to markdown with an automated agent is fragile — no schema, no permissions, and with files syncing around my network, I'm one race condition away from a merge headache. So I don't let the AI write here. If it produces something worth keeping, I promote it by hand.
Trilium is the opposite shape. It's a database behind an API (ETAPI), with native MCP support and proper permission control — read-only or read/write, my choice. That makes the agent-writes-back loop actually reliable. Dated entries, structured attributes, an interface Hermes and Claude can both hit cleanly. Exactly what you want for a journal the AI helps maintain.
So: Obsidian gets read-only treatment because it's my canon. Trilium gets read/write because it's the working log and it's structurally built to handle it.
The Architecture
Here's the whole thing on one diagram:
Obsidian Vault (.md) Trilium (daily notes / DB)
reference knowledge working memory + journal
│ │
│ READ-mostly │ READ + WRITE
▼ ▼
┌─────────────┐ ┌──────────────┐
│ Smart Conn. │ │ Built-in AI │
│ + Ollama │ │ + ETAPI/MCP │
│ Claude Code │ │ │
└──────┬──────┘ └──────┬───────┘
│ │
└──────────► Hermes ◄─────────────┘
(orchestrator: routes
reads vs writes)
│
┌───────┴───────┐
▼ ▼
Ollama Claude
(local worker) (escalation)The key shift here is that Hermes stops being just an orchestrator and becomes a router. It knows where each operation belongs. The rule set is almost embarrassingly simple:
- Reads hit both stores — reference from Obsidian, recent context from Trilium.
- Writes go to Trilium only, scoped to the daily-notes subtree.
- Obsidian writes are human-gated. Always.
Underneath both, the same two-tier model I've been running all along: Ollama does the routine local work (free, private, fast), and Claude gets the escalation — the complex reasoning and long-context synthesis that the local models can't touch.
The Flow That Makes It Worth The Effort
Running two apps only pays off because of how they feed each other:
- Capture. Daily stuff — decisions, meeting notes, half-formed ideas — gets written to Trilium. By me, or by the agent.
- Retrieve. When I ask a question, Hermes pulls reference context from Obsidian and recent working memory from Trilium, then stitches them into the prompt. Recent Trilium entries first (what's top of mind), then Obsidian reference matches (the background), then my actual question. Keeps the prompt focused and stops it bloating past the context window.
- Promote. Every so often, the good stuff from the Trilium daily log graduates into a proper Obsidian
.mdnote. The journal is ephemeral; the library is forever. This step is manual on purpose — it's the quality gate that keeps my curated vault clean.
That promotion step is genuinely the whole point. Trilium absorbs all the messy, high-frequency writes — which protects my Obsidian vault from being cluttered up by an over-eager agent — and only vetted, distilled knowledge ever crosses the line into the library.
A Couple Of Guardrails
Because I don't trust anything with write access until it's earned it:
- Trilium's AI features are still experimental, so I started the MCP connection and Hermes both at read-only, and only promoted to read/write once I'd watched them behave for a while.
- The agent writes into a dedicated subtree — an
AI Memoryparent note — rather than scattering machine-written content through my real notes. Keeps it quarantined, easy to audit, trivial to roll back if it goes sideways.
Where This Leaves Me
No separate vector database, no extra infrastructure to babysit. Obsidian's RAG handles reference retrieval, Trilium's ETAPI handles dated working memory, and each app does the one thing it's actually good at. Hermes routes between them, Ollama does the grunt work, Claude handles the hard problems.
The lesson, as usual, was that I'd been overcomplicating it. The answer wasn't a clever tool or a new piece of kit — it was just letting two tools I already used keep doing their jobs, and being disciplined about which one the AI is allowed to write to.
If you've made it this far: yes, I really did spend an unreasonable amount of time deciding where a chatbot is allowed to save its notes. Worth it though.
