The index was bigger than the files
An index that weighs more than the library it points to is an odd kind of progress. In the setup described with the LEANN paper, a 76 GB text corpus needs 173 GB of stored embeddings plus a 15 GB graph. Together those are the 188 GB HNSW index, before anyone asks a question. The documents are the small part.
On the project’s own numbers, the storage cut is large enough to change what fits on a laptop. The wait is large enough to decide where that index belongs. That is the argument worth pricing. After that, the job is to see what the numbers actually measure.
LEANN is an open-source approximate nearest-neighbor index, credited to UC Berkeley’s Sky Computing Lab. It keeps a pruned search graph and a small table of compressed codes, drops the full embeddings, and recomputes them only for the nodes a query actually walks. The practical bet is semantic search and retrieval-augmented generation over mail, chat, a codebase, or files a company would rather not ship to a hosted vector database.
The figures below come from the paper, the repository, or Marktechpost’s breakdown of those tables. They are self-reported, not an independent audit. The preprint is arXiv 2506.08276. Marktechpost also says the work won Best Research Paper at MLSys 2026.
A map, not a warehouse of vectors
Ordinary semantic search turns each chunk of text into an embedding, a long list of numbers, then stores those lists so a query can find near neighbors without reading every document. Graph methods such as HNSW, LEANN’s default backend, make that search fast. They also make the index fat. FAISS, the similarity-search library underneath LEANN, is excellent at the fast part. It does not, by itself, solve the fat part.
LEANN’s wager is that a graph walk touches only a small slice of the nodes, so the full vectors do not have to live on disk. Product quantization is the compromise that makes the wager searchable. It is a standard compression method: each embedding becomes a short code, cheap to compare, too coarse to trust as the final measurement. A two-level search uses those codes to rank candidates, then recomputes exact embeddings only for the top ones. Dynamic batching groups candidates across hops, 64 at a time in the example given with the method, so a GPU is not woken for a single vector.
The graph is pruned, but not blindly. Most nodes keep few edges. The top 2% of high-degree hub nodes keep more, so the map stays connected. Picture a city that stores the road network and a thumbnail of each block, then sends a surveyor down only the streets a given trip will use. You save the archive. You pay the surveyor on every trip.
LEANN is an index. It is not a language model, so context length does not apply. Project materials name Contriever, a 110-million-parameter embedding model, as the default. Generation is a separate choice: Ollama, Hugging Face, Anthropic, or any OpenAI-compatible API. That split matters more than the install command.
Two headline numbers that do not agree
The repository benchmark puts a 60-million-chunk Wikipedia index at 6 GB instead of 201 GB. That is a 97% cut in index storage, index against index: the 6 GB figure replaces a 201 GB FAISS index, not the raw files.
The paper’s setup is a different pair of headlines. On a 76 GB corpus, also described as 60 million chunks and called RPJ-Wiki, LEANN is reported at 4 GB against 188 GB for HNSW. A chart in Marktechpost’s explainer, tied to Table 1 and Section 7.2 of the paper’s second version, lines that corpus up as DiskANN at 270 GB, HNSW at 188 GB, the raw text at 76 GB, BM25 at 59 GB, product quantization alone at 20 GB, and LEANN at 4 GB.
Those two 60-million-chunk results, 6 GB and 4 GB, are not reconciled. The materials also do not show that “Wikipedia” and “RPJ-Wiki” are the same corpus in every comparison. Embedder, quantization, graph settings, or the corpus itself could differ. Quote one figure, and say which benchmark it came from.
Watch the denominator. Four gigabytes is 5.3% of that 76 GB corpus and about 2% of the 188 GB HNSW index. Those are different claims.
DiskANN needs the same care, in the other direction. It is offered as a backend when you want faster search, and on that chart it is the largest index of the set. Its case is speed. On the reported tests it stays in the instant range, and on the Mac it still answers where the explainer lists no HNSW times. Choosing it will not shrink the index. On the speed metric it is built for, the reported tests do not show a loss.
Matching accuracy is a setting, not a slogan
Posts promoting the repository sometimes say there is no accuracy loss, and that the system is fully local with no cloud dependency. The storage cut is real on the repository’s arithmetic. The other claims are narrower than the slogans.
Latency in the paper is measured at 90% Recall@3. That means the search returns 90% of the true nearest neighbors inside the top 3. The paper uses that retrieval operating point to time the search. It does not mean answers are 90% correct, and it does not mean zero retrieval loss. Downstream accuracy is a different number: after retrieval, did the model produce the right answer.
On Natural Questions, with Qwen3-4B on an RTX 4090, the reported comparison has LEANN and HNSW both at 25.5% downstream accuracy, with the index at 4 GB rather than 188 GB. BM25 scores 18.3% there, and product quantization used as compression alone scores 17.9%. A 4-billion-parameter generator produced the 25.5% figure, so that score is not a verdict on retrieval by itself. At this operating point, throwing away the stored vectors did not move the answer metric relative to HNSW. The match is against the heavy index, and 25.5% is not a high absolute score.
The bill arrives in seconds
On that Natural Questions run, HNSW retrieval is 0.05 seconds and LEANN retrieval is 2.48 seconds. Generation for the row is listed at 20.86 seconds, and the end-to-end figure given for LEANN is 23.34 seconds. Generation is still most of the wait. Retrieval is no longer a rounding error.
Across the RTX 4090 tests, at 90% Recall@3, LEANN retrieval runs from 1.12 seconds on GPQA to 7.12 seconds on HotpotQA. HNSW stays around 0.04 to 0.05 seconds. DiskANN sits in the same instant band, 0.03 to 0.11 seconds, if you can afford the disk. A coarser recomputation baseline, IVF-Recompute, is not in that conversation: 21.88 seconds on GPQA and 429.46 seconds on HotpotQA on the same workstation. In the paper’s ablations, two-level search averages a 1.4x speedup, and batching raises that to 1.8x. Recomputation still accounts for about 76% of LEANN’s query latency. Useful engineering. Not enough to feel like in-memory graph search.
On an M1 Ultra, compare LEANN with DiskANN, because the explainer lists no HNSW times for that machine and marks those cells out of memory. The surrounding write-up does not state a crash, so treat the label as the explainer’s, not as a measured failure report. DiskANN stays between 0.29 seconds on GPQA and 1.91 seconds on HotpotQA. LEANN takes 5.22 seconds on GPQA, 13.84 on Natural Questions, 17.15 on TriviaQA, and 44.80 on HotpotQA.
That last HotpotQA number needs its other half. Generation on the same Mac row is 44.67 seconds, and the explainer puts LEANN’s share of pipeline time at 50.1%. Retrieval has become the wait. A CPU-only Linux install exists. No CPU latency is published.
Disk was the constraint everyone photographed. Latency is the constraint that shows up when someone tries the demo and expects the cursor to settle.
Where it earns a place, and where it does not
For a company, the interesting use is a semantic index that fits beside the files, not another chat window. Internal lookup over documents, mail, and chat can tolerate a few seconds, especially when the model is already going to spend a long time generating. On the reported RTX 4090 runs, generation ran from 17.17 seconds on TriviaQA to 69.60 seconds on GPQA, with Natural Questions at 20.86 and HotpotQA at 23.28. A sales team sitting on years of email is a fair candidate. The repository lists an email collection of about 780, 000 items at 79 MB for LEANN against 2.4 GB for FAISS. Skip it for a customer-facing box, a typeahead, or any workflow where HNSW or DiskANN would have answered in a few hundredths of a second.
The same split applies to an agent workflow. The project says LEANN can run as a semantic-search server for Claude Code, with readers for live sources such as Slack and bookmarks, plus packaged apps for documents, Apple Mail, Chrome history, WeChat, iMessage, and chat exports. Some of those apps are macOS only. On 30 SWE-Bench Pro tasks drawn from ContextBench, the team reports 24.2% initial relevant-code recall versus 11.4% for BM25, and 8.4% fewer tokens than that BM25 baseline. That figure is initial code recall and token use. It is not a task-completion rate, and 30 tasks is a small slice. Reason enough to try the index inside a workflow that already uses AI agents. Not enough to declare the agent finished.
Local is a configuration, not a property of the logo. An Ollama path can keep generation on the machine. Point generation at Anthropic or an OpenAI-compatible API and the retrieved passages may leave the building anyway. A local index is attractive because private material need not be uploaded for the search step. Read that as a design implication. It is not a security review of the connectors, the credentials, or what a cloud model does with the text you just retrieved.
The savings are also at rest, not during construction. Every chunk is still embedded once. The explainer reports that a 15-shard build cuts peak build storage by about 5x, and it does not skip that pass. Update, delete, and refresh costs are largely undocumented. A mailbox that changes all afternoon is a different problem from a frozen Wikipedia dump.
The project is described as MIT-licensed, with a PyPI package installable as uv pip install leann, and as running on Ubuntu, Arch, WSL, macOS, and Windows. The code has been associated with the StarTrail-org repository. None of that replaces a pilot on your own files, with a latency budget written down before the demo.
Questions worth asking first
- Is the 6 GB index the same result as the 4 GB index?
No. The repository benchmark is 6 GB against 201 GB for 60 million Wikipedia chunks. The paper’s setup is 4 GB against 188 GB on a 76 GB, 60-million-chunk corpus it calls RPJ-Wiki. The gap is unexplained, and the two names are not shown to be one corpus in every comparison.
- Does answer quality survive the compression?
On the reported Natural Questions test, LEANN matched HNSW at 25.5% downstream accuracy with Qwen3-4B, at a 90% Recall@3 operating point. That matches the heavy index. It is not a high score, and it is not a perfect retriever. BM25 was at 18.3% and product quantization alone at 17.9% in that comparison.
- How slow is the slow part?
On the reported RTX 4090 tests, retrieval ran from 1.12 seconds to 7.12 seconds, against about 0.05 seconds for HNSW. On the M1 Ultra, HotpotQA retrieval was 44.80 seconds beside 44.67 seconds of generation, about half the pipeline. Recomputation is still about 76% of query latency.
- Does a laptop install mean the data stays private?
Only if generation stays local too. Ollama can do that. Hugging Face, Anthropic, and OpenAI-compatible APIs are also supported, and no reported test audits connector security or what a cloud model does with the passages you just retrieved.
- Is this ready to sit under an agent?
It can be tried. The project says LEANN can run as a semantic-search server for Claude Code, and on 30 SWE-Bench Pro tasks it reported better initial code recall than BM25, plus 8.4% fewer tokens than that baseline. That sample does not measure whether the agent finishes the task.
- What do you still pay up front?
A full embedding pass over every chunk. A 15-shard build is reported to cut peak build storage by about 5x, and it does not skip the pass. How the index behaves when documents are added or deleted is mostly unmeasured.
LEANN is a serious attempt to make retrieval fit on the machine you already own. On the project’s own numbers, the storage cut is large enough to change what is practical. The wait is large enough to change where you should put it. Use it where a person, or an agent, was already going to sit through generation. Leave it alone where the old index’s hundredths of a second were the product.
Check the citation, not the slogan
The storage and latency claims are easier to price once you are looking at the paper’s own record rather than a recap.