Berkeley Open-Sources LEANN: 4GB Memory vs. HNSW’s 90% Recall, Seconds-Per-Query Latency
UC Berkeley’s LEANN trades memory for computation by using a pruned graph and recalculating embeddings during queries. Its RAG index with 60 million chunks shrinks from 201GB to just 6GB; at 90% recall, memory usage drops from HNSW’s 188GB to 4GB, at the cost of second-level search latency.
RAG systems typically split documents into chunks and generate embedding vectors. Vector indexes are responsible for finding similar items among massive vectors, and HNSW is one of the most common types. It’s fast, but both its graph structure and vectors consume memory—so when data volumes grow, machine costs often become a bottleneck.
UC Berkeley’s proposed LEANN offers a different trade-off. According to MarkTechPost author Asif Razzaq, LEANN does not store full embeddings in the index—instead, it saves a pruned graph, then recalculates required embeddings during queries.
The report provides two key sets of figures:
| Scenario | LEANN | Baseline |
|---|---|---|
| RAG index with 60 million chunks | 6GB | Original index: 201GB |
| Achieving HNSW-level accuracy at 90% recall | 4GB | HNSW: 188GB |
The cost is clear. LEANN’s search latency is at the second level, making it unsuitable for online retrieval services that require millisecond response times as a hard metric. It is better suited for budget-sensitive scenarios, low-frequency queries, batch processing, or resource-constrained environments.
The highlight of this project is not merely shrinking the index size—it is shifting storage pressure to computational resources consumed during queries. The memory saved is tangible, and the increased latency is also tangible. This trade-off is more worth evaluating than a parameter table alone.
Source: MarkTechPost, Asif Razzaq
发布时间: 2026-10-28 05:54