File Handles for Agent Memory

Agent memory is becoming a storage problem.

The useful projects I checked did not treat memory as a magic summary injected above the prompt. They gave the agent handles: page paths, file scopes, URI trees, mailboxes, external ids, audit logs, and replayable retrieval. That is the part worth studying. Once an agent can switch models, delegate work, and run across days, a transcript summary becomes too soft. You need a store that can say where a fact came from, which project owns it, who wrote it, and why the next agent should recieve it.

The stronger memory systems expose context as scoped files, pages, ids, traces, and lifecycle events. The model reads a bounded view. The store keeps the history.

My Trial Bench

I used Windows 11, PowerShell, Node 24, npm 11, uv, Python 3.12, Rust via rustup, and no paid model calls. I cloned the projects, ran first-run commands, ran focused tests where the setup would allow it, and inspected the code paths that decide whether memory is a file, a database row, a vector, or a message.

The checks favored storage behavior: scoped writes, atomic mutation, retrieval filters, durable sync, migration guards, and whether the install path explains itself before asking for a full app build.

Project	Check	Result	Read
ai-memory	CLI help; focused Rust tests	Help built after Rust 1.95 install. 252 + 97 + 91 tests passed across store, wiki, and workstream crates.	Strongest storage contract in the set: wiki files, SQLite index, FTS5, hooks, MCP, managed workstreams, checkpoints.
OpenViking	Editable install + VFS source/test inspection	Web studio built. Install then failed when ov_cli used default Rust 1.89 while requiring 1.91.1. Retried with Rust 1.95 and stopped after a long silent build.	Good context-database shape. First-run build is heavy and toolchain-sensitive.
Munder Difflin	npm install; focused Node tests	863 packages installed with scripts disabled. Queue and task-ledger tests passed: 6 passed. A wider path test failed 4 Windows assertions.	Real local multi-agent harness. Memory/mailbox design is concrete, but setup drags a large desktop-app stack.
Turbovec	downstream smoke; adversarial durability tests	Smoke binary passed. Durability tests returned 8 passed after warning on torn commits and rolling back cleanly.	Useful low-level retrieval primitive: id-mapped ANN, allowlists, mutation, write/load, incremental sync.

The Store Has to Own Scope

ai-memory has the cleanest boundary between memory and current code. The README calls the wiki "plain markdown in a git repo", but the code does more than write notes. It keeps an indexed store beside the files, installs lifecycle hooks, exposes MCP tools, and can launch managed workstreams across different agent CLIs.

The first command was not fast:

cargo run -p ai-memory-cli -- --help

info: syncing channel updates for '1.95-x86_64-pc-windows-msvc'
...
Finished `dev` profile [unoptimized + debuginfo] target(s) in 10m 39s
Running `target\debug\ai-memory.exe --help`

The help output matters because it names the product:

Commands:
  run
  workstream-search
  audit-contamination
  search
  read-page
  write-page
  serve
  backup
  restore
  reindex
  install-hooks
  install-mcp
  checkpoints
  restore-page
  finalize-session
  move-project
  move-session

That is a memory system, not a prompt trick. It can search the wiki through FTS5, write pages atomically, restore pages from git checkpoints, move a session across projects, audit contamination, and rebuild the database from files. The "DB is rebuildable from files" command is the right kind of boring. If the index becomes corrupt, the markdown tree is still the durable source.

The focused Rust test run gave a strong signal:

cargo test -p ai-memory-store -p ai-memory-wiki -p ai-memory-workstream --lib

returned:

ai_memory_store       252 passed; 0 failed; finished in 133.14s
ai_memory_wiki         97 passed; 0 failed; finished in 4.18s
ai_memory_workstream   91 passed; 0 failed; finished in 0.65s

The test names show the design pressure. I saw tests for schema-ahead errors, migration rollback, scoped handoff delivery, cross-project contamination, FTS path indexing, atomic writers, symlink rejection, wiki checkpoint restore, secret scrubbing at write boundaries, managed workstream isolation, transcript import filtering, and exact native-session resume across several agent CLIs.

That is the memory layer I trust more than "the model will remember". The store records observations, distills wiki pages, and keeps project scope separate from agent identity. The agent can use memory, but it does not get to define what memory means.

OpenViking Makes Context Browsable

OpenViking uses a different mental model: context lives under a viking:// virtual filesystem. Memories, resources, and skills sit under typed paths. Each entry can have a short abstract, a larger overview, and the full content. Retrieval can walk directories instead of throwing a bag of chunks at the model.

The shape is good:

viking://
  resources/
    project/
      .abstract
      .overview
      docs/
  user/
    {user_id}/
      memories/
      resources/
      skills/
      peers/

The VFS tests are the strongest part I could inspect without a full install:

tests/agfs/test_viking_fs_git.py

uses real paths like:

viking://resources/a.md
viking://resources/proj/.abstract.md
viking://session

and checks commit, show, log, restore, deleted files, path budgets, encrypted reads, derived .abstract.md and .overview.md files, and reindex hooks. That is the right abstraction for an agent. It can ask for tree, find, grep, or a path restore. A human can debug the same route.

The local build was rough:

uv run python -m pytest tests/agfs/test_viking_fs_git.py -q

got far enough to install and build the web UI:

added 1046 packages, and audited 1047 packages in 9m
29 vulnerabilities (3 low, 11 moderate, 12 high, 3 critical)
vite v7.3.2 building client environment for production...
3950 modules transformed.
built in 52.88s

then failed while building the Rust CLI:

Failed to build ov CLI from source
error: rustc 1.89.0 is not supported by the following packages:
  ov_cli@0.0.0 requires rustc 1.91.1

I retried with:

$env:RUSTUP_TOOLCHAIN = "1.95-x86_64-pc-windows-msvc"
uv run python -m pytest tests/agfs/test_viking_fs_git.py -q

and stopped the run after several silent minutes in the local build step. That does not make the architecture weak. It does make the first-run story too heavy for a context database. A developer trying to verify the VFS should not have to build a full studio bundle, audit a thousand npm packages, and then discover the Rust toolchain floor through a failed editable wheel.

The useful lesson still stands: agent context should be browsable before it is semantic. A vector hit without a path is hard to challenge. A viking://resources/project/docs/auth.md.overview path gives the model a handle and gives the operator a place to inspect.

A Harness Needs Mailboxes, Not Mind Reading

Munder Difflin is noisy on purpose. The branding wraps a serious local harness: real terminal-agent CLIs in pseudo-terminals, a shared hive, per-agent memory, inbox/outbox delivery, a task ledger, git worktrees, circuit breakers, and a visible desktop floor.

I do not care about the avatar layer for this note. I care that the harness turns coordination into files and queues.

The first test run failed before project code because the clone had no dependencies:

node --test test/queue-delivery.test.cjs test/hive-task-mutation.test.cjs test/expand-tilde.test.cjs

Error: Cannot find module 'typescript'
Require stack:
  test/load-ts.cjs

I installed dependencies with native scripts disabled:

npm install --ignore-scripts --no-audit --no-fund

which returned:

added 863 packages in 7m

The focused tests then passed:

node --test test/queue-delivery.test.cjs test/hive-task-mutation.test.cjs

returned:

patching a stale UI card preserves a concurrently appended webhook card ... ok
atomic add is idempotent and delete removes only the named card ... ok
patch refuses an unknown card without rewriting the ledger ... ok
renderer task actions never send a whole stale ledger back to main ... ok
queue item is acknowledged only after delivery succeeds ... ok
failed delivery remains unacknowledged for retry ... ok

tests 6
pass 6
fail 0
duration_ms 35600.9264

The wider test set found a Windows portability wart:

tests 18
pass 14
fail 4

The failures expected POSIX-style paths such as /a/b/c and /other/hive; Windows normalized them to C:\a\b\c and C:\other\hive. That is not a memory-system bug, but it is a real harness bug class. A local agent app that wants to run cross-platform cannot leave path assumptions in tests that pretend Windows is POSIX.

The part I liked is the stale-ledger regression test. It simulates a UI holding an old task snapshot while a webhook appends another card. The fix path mutates one card against the latest main-process ledger instead of sending the whole stale JSON collection back through IPC. That detail belongs in an agent harness. Agents will race each other. The store has to make stale writes hard.

Munder also has a clear mailbox rule in code and prompts: agents read memory.md, drain inbox, write outbox messages, and move handled messages to .done. That is primitive, but primitive in the good way. You can inspect it with a file browser.

Vector Search Still Needs IDs

Turbovec sits below the agent-memory layer. It is a Rust vector index with Python bindings, built around TurboQuant compression. The reason I kept it in this article is not the compression claim. The reason is the API contract: IdMapIndex, allowlisted search, mutation, write/load, and sync().

Agent memory stores need dense retrieval. They also need ACLs, project filters, deletion, and crash-safe persistence. A vector search layer that cannot answer "only search this tenant's ids" forces the caller into over-fetch and post-filter. That leaks cost and sometimes leaks data.

The downstream smoke test passed:

cargo run --manifest-path examples/downstream-smoke/Cargo.toml

returned:

downstream-smoke: OK (256 db vectors, 4 queries, top-5 search,
write/load round-trip, trait bounds, Arc<RwLock<_>> concurrency,
error paths, mutation + masking + k-clamping, to_bytes/from_bytes
+ Durability + from_parts, IdMapIndex)

The smoke test uses the crate the way a downstream service would. It constructs an index, searches, writes to disk, loads back, shares the index through Arc<RwLock<_>>, rejects bad dimensions and non-finite values, mutates the index, checks masks, checks k-clamping, and exercises external ids.

The durability test passed too:

cargo test -p turbovec --test adversarial_durability -- --nocapture

returned:

running 8 tests
...
test result: ok. 8 passed; 0 failed; finished in 43.35s

The warnings are the important output:

turbovec: warning: ... the newest commit (generation 2) is incomplete
-- its sync did not finish -- so generation 1 was loaded instead;
changes made after that commit are lost

That is the right failure mode. A torn sync loads the previous generation or the new one. It should not load a third state. For context stores, this matters because memory writes often happen during agent shutdown, crash recovery, or compaction. If a store corrupts itself during those moments, the next agent inherits bad history.

Turbovec is not a full memory system. It does not know what a project is. It does not know instruction authority. It gives the higher layer the mechanical pieces a good memory store needs:

Piece	Why an agent store needs it
external ids	Map vectors back to pages, chunks, sessions, and ACL rows.
allowlisted search	Query only the active project, tenant, or candidate set.
O(1) remove by id	Forget deleted or expired memory without rebuilding the world.
durable sync	Survive crashes during mutation-heavy writes.
write/load round trip	Let a service restart without re-embedding everything.

The Contract I Want

The strongest pattern across these projects is simple: context should enter the model through a storage contract, not through a pile of remembered text.

I would start with this record:

{
  "context_id": "ctx_2026_08_19_0031",
  "workspace": "default",
  "project": "billing-service",
  "path": "decisions/refund-policy.md",
  "kind": "decision",
  "authority": "maintained",
  "owner": "human",
  "source_event": "session:018f/tool:write_page",
  "body_digest": "sha256:...",
  "index": {
    "fts": true,
    "entities": ["refund", "stripe", "chargeback"],
    "vector_id": 881244
  },
  "retrieval": {
    "query": "refund job retry behavior",
    "filters": {
      "workspace": "default",
      "project": "billing-service",
      "authority": ["maintained", "recent-session"]
    },
    "trace": [
      "fts:refund",
      "entity:stripe",
      "vector:allowlist[project=billing-service]"
    ]
  }
}

That record gives you answers to questions the model cannot answer by itself:

Question	Field
Which project owns this memory?	workspace, project
Can this page override current code?	authority
Which event wrote it?	source_event
Can I inspect the exact text?	path, body_digest
Did retrieval cross a tenant or project boundary?	filters, trace
Can dense search respect scope?	vector_id plus allowlist

The agent can still summarize this for context. The store keeps the fact.

Cut List

I skipped video-generation workflows, browser-stealth automation, generic skill catalogs, and wrapper apps that did not expose a memory or retrieval contract I could inspect. Some of them had real code. They did not support this thesis.

I also skipped pure benchmark news and model-release links. They can be useful, but they do not tell you how an agent should carry state between runs.

The filter was:

• Does the project give context a stable path, id, or namespace?
• Can retrieval explain where a result came from?
• Can writes happen atomically under concurrent agent work?
• Can the store rebuild or roll back after failure?
• Does the first-run path expose the contract without a full product ritual?

ai-memory passed that filter cleanly. Turbovec passed it at the vector-index layer. Munder passed it for local coordination primitives, with setup weight. OpenViking passed it on architecture and source tests, but the local install path did not finish on this machine.

Close

Agent memory should feel more like a small database than a diary. The model can draft the note, but the system has to decide scope, authority, indexing, and retention.

The projects worth copying give agents file handles: page paths, URI trees, mailboxes, ids, logs, and recovery points. That is less magical than "long-term memory", and far easier to debug when the next run gets the wrong idea.

Sources

• akitaonrails/ai-memory
• volcengine/OpenViking
• chaitanyagiri/munder-difflin
• RyanCodrai/turbovec
• TurboQuant paper