Measuring AI agent memory across a major refactor
A ~9,300-node TypeScript SaaS platform was rebuilt with NeuralMind watching. The memory didn't go stale — it grew into the new architecture. Here are the numbers, where each one came from, and how to run the same measurement on your repo.
Avg token reduction
48.8×
~1,033 tokens/query vs 50K+ naive
Personal synapse edges
36 → 135
the learning layer tracked the new code
Full rebuild
326 s
then back to ~30 s incremental
The setting
A private, mid-size TypeScript SaaS platform (~9,300 indexed nodes), maintained by NeuralMind's own maintainer — see the honesty notes below. A major internal rebuild ("Phase 3 → Phase 4") landed a new backend module, a shared business-logic layer, an admin UI, and end-to-end specs.
NeuralMind ran throughout with lifecycle hooks installed, so the synapse layer observed the work as it happened. The repo is private, so the numbers are anonymized; every one of them came from the shipped CLI.
The numbers
| Metric | Before (Phase 3) | After (Phase 4) | Change |
|---|---|---|---|
| Total nodes | 9,190 | 9,293 | +103 |
| Communities | — | 810 | new resolution — no comparable before |
| Personal synapse edges | 36 | 135 | +275% |
| Shared synapse edges | 10,079 | 10,183 | +104 |
| Shared edge weight | 2,774.73 | 2,924.66 | +5.4% |
| Wake-up tokens | — | 455 | |
| Avg query tokens | — | 1,033 | |
| Avg token reduction | — | 48.8× | vs loading files naively |
| Full --force rebuild | — | 326 s (~5.4 min) | incremental rebuilds ~30 s after |
What the numbers say
48.8× token reduction. After the rebuild, an average code question cost ~1,033 tokens of context instead of the 50K+ a naive "load the relevant files" approach would spend. That is one repo's measured ratio from neuralmind benchmark ., consistent with the 40–70× real-repo range — not a universal guarantee.
Personal synapse edges tripled (36 → 135). The Hebbian synapse layer learned co-activations across the new code as it was being written and queried. The memory didn't go stale through the rebuild; it grew into the new architecture.
Shared edge weight +5.4%. The static graph's cross-links got denser as the new shared business-logic layer connected previously separate areas — densification, not just growth: edge weight rose 5.4% on only 1.1% more nodes.
Rebuild cost stayed flat. One full --force rebuild at 326 s to pick up the new structure, then incremental rebuilds back to ~30 s. Re-indexing is not a tax you pay per edit.
Where each number comes from
Honesty first: the before/after table is hand-assembled from the output of several commands — there is no single neuralmind compare-phases command.
| Metric | Command |
|---|---|
| Total nodes, communities | neuralmind stats . --json |
| Personal/shared edges, edge weight | neuralmind stats . — the Memory namespaces block |
| Wake-up / query tokens, reduction | neuralmind benchmark . |
| Rebuild time | timed neuralmind build . --force |
| Synapses fired per query | neuralmind metrics . |
| Real logged spend | neuralmind savings . |
Run the same measurement on your refactor
1. Snapshot before you start
neuralmind stats . --json > .neuralmind-before.json
neuralmind benchmark . # baseline reduction number
2. Refactor with the memory watching
The synapse layer only learns from what it observes — install hooks before the work starts.
neuralmind install-hooks .
neuralmind watch & # optional: always-on learning from edits
3. Rebuild and snapshot after
time neuralmind build . --force
neuralmind stats . --json > .neuralmind-after.json
4. Re-benchmark and price it
neuralmind benchmark . # reduction on the post-refactor graph
neuralmind savings . # what your logged queries cost vs naive
Diff the two JSON files. The interesting deltas: node count (did the graph track the new code?), personal edges (did the memory learn the new co-activations?), shared edge weight (did the graph get denser or just bigger?). The full step-by-step walkthrough lives in the docs.
Honesty notes
One repo, one developer, measured by the maintainer on a private codebase. It is an existence proof and a recipe, not an independent benchmark. You cannot re-run it on the source repo — but you can run the identical method on yours, which is the point.
48.8× is a reduction in retrieval input tokens, not your total LLM bill. For typical end-to-end agent sessions the realistic total-cost saving is smaller — see the honest assessment.
The CI-gated, reproducible numbers live in Benchmarks & Results and on the benchmarks section of this site; this page is deliberately labeled a field report.