Measure your own
The number that decides anything for you is the one from your codebase. Here is how to get it — and why we want you to share it.
We do not have testimonials yet. We do not have a video demonstrating a before/after on a real user’s codebase. What we have is a reproducible benchmark that runs locally, on your machine, against your code, in about fifteen minutes. Run it. Read your own number. If it is good, tell us — we will publish it.
The honest pitch
Anyone can run a benchmark on fixture data. That is why the public benchmark exists — because you should not take any vendor’s word for it. But a benchmark on your own repository, with your own queries, on your own machine — that is the only number that actually tells you whether NeuralMind will help your workflow.
If you run it and the number is good, we want to publish it. Attributed or anonymized — your choice. We are building the social proof we currently lack.
If you run it and the number is not good, we want to know that too. Email us the output and we will tell you why, or fix it. That is the honest deal.
Step by step
Clone and install
Get the source checkout. NeuralMind ships the benchmark harness in the repo, not the PyPI wheel, so you need a clone.
$ git clone https://github.com/dfrostar/neuralmind && cd neuralmind
$ pip install -e . tiktoken
Build your index
One command indexes your codebase. Tree-sitter parses the structure, TurboVec compresses embeddings 4-bit. Processing runs on your machine.
$ neuralmind build .
Run the benchmark
Compares NeuralMind against full-file context and ripgrep on your own queries. Reports gold-file recall and token cost together.
$ neuralmind benchmark .
# or for JSON output:
$ neuralmind benchmark . --json
Read your number
The output shows: naive tokens, NeuralMind tokens, reduction ratio, and per-query recall. No cherry-picking — every query is reported.
# Example output:
Naive: 41,729 tokens/query
NeuralMind: 913 tokens/query
Reduction: 45.7× fewer
# This is the requests repo. Your number will differ.
Share your results
Email [email protected] with your output. We will publish your results (attributed or anonymized, your choice). Help us build the social proof we do not have yet.
What you get
- Tokens/query
Tokens/query
Mean context size in tokens (tiktoken o200k_base). Compare NeuralMind vs naive full-file vs ripgrep.
- Reduction
Reduction
How many fewer tokens NeuralMind sends vs the naive baseline. 12-50× is the published real-repo range.
- Gold-file recall
Gold-file recall
For each query: did the objectively-correct file land in the assembled context? Objective, no LLM judge.
- MRR
MRR
Mean reciprocal rank of the gold file. 1.0 = always first; 0.5 = always second.
Already run it?
Paste your output to [email protected]. We will publish it on this page (attributed or anonymized, your choice). The only requirement: the output must be from your own codebase, under your own run. We are building social proof the honest way — one real number at a time.