60-second video, indexed in 29 seconds: local video search can now run on a laptop

Google DeepMind's EmbeddingGemma 2 maps text, images, video, and audio into the same vector space. Avi Chawla used this model to build a local video search interface: you enter natural language, and it can find matching frames and jump to the corresponding timestamp. The whole pipeline runs on a laptop.

Avi gave several numbers in his tweet. The test machine was an Apple M3 Pro, running on CPU. Indexing a 60-second video took 29 seconds. Nine queries took an average of 25 ms. The saved frame vectors occupied 180 KiB. The video is indexed only once. After that, for each search, the query text is embedded into a vector and compared with the saved frame vectors. The processing never leaves the local machine. The tweet also mentioned that a developer has written a native Rust implementation for local inference.

There are two concepts in this description that are easy to conflate. The memory footprint of model weights can be reduced through quantization. The storage cost of the retrieval index is a separate matter. Unsloth provides a 176 MB 4-bit GGUF checkpoint for text embedding, not including the full multimodal processing pipeline. Video frame vectors look small, but as the number of videos grows, the index will keep growing.

Avi then turned the topic to binary quantization. He wrote a long article explaining how to compress RAG's raw vector storage by 32x. The approach is not complicated: each dimension of a float32 vector takes 32 bits, and binary quantization turns each dimension into 1 bit. Storage and memory footprint drop to about 1/32. During retrieval, Hamming distance is used to compare two binary vectors and see how many bits differ.

The article mentions that Perplexity uses this technique in its search index, Azure uses it in its retrieval pipeline, and HubSpot uses it in its AI assistant. Avi built an example with LlamaIndex, Milvus, and Kimi-K2 on Groq: convert document embeddings into binary vectors, store them in Milvus's BIN_FLAT index with HAMMING as the metric; the query is also binary-quantized, top-k context is retrieved, and then passed to an LLM to generate an answer. He queried more than 36 million vectors on the PubMed dataset in under 30 ms. The code is on GitHub, with the link at the end of the article.

For general readers, you can understand it as extreme compression for vectors. A full-precision vector is like a coordinate table filled with decimal points, while a binary vector keeps only which side of zero each dimension falls on. Similarity comparison becomes counting the number of differing bits. It is fast and saves storage, at the cost of coarser information.

In the replies, someone gave concrete numbers for the cost. Vishwraj Jadav said he tested binary quantization at FortiqAI with about 50,000 Indian court cases: retrieval was much faster, top-k recall was strong, and top-1 accuracy dropped by 8 to 12 percentage points. His recommendation is to use binary for recall and float vectors for reranking. Another reply, from Abdel Latrache, also mentioned that at large scale Hamming distance is suitable for performance optimization, but binary vectors need reranking to recover accuracy.

Avi also left a reminder at the end of the article. Binary quantization makes the vector search layer more efficient, and production retrieval rarely consists of just one vector lookup. A real agent pulls context from Slack, GitHub, Jira, databases, and documents at the same time. Authentication, synchronization, query routing, permissions, and reranking all become first-class citizens alongside embedding search. Binary quantization is a strong piece of retrieval infrastructure, not the whole story of a retrieval system.

Back to video search. A 60-second video indexed in 29 seconds, queries in 25 ms, frame vectors at 180 KiB—these numbers show that local multimodal retrieval has entered the usable range. It is suitable for privacy-sensitive, offline, and personal media library scenarios. To truly go into production, the index will grow with the number of videos, and binary recall will still need to be followed by reranking. Saving memory and preserving accuracy need to be designed separately.

Sources: