Video search stays on the laptop: index in 29 seconds, find a frame in 25 milliseconds

Avi Chawla built a local video search interface with EmbeddingGemma 2. Enter a sentence, and the interface finds the matching frame and jumps to the corresponding timestamp.

EmbeddingGemma 2 comes from Google DeepMind and maps text, images, video, and audio into the same vector space. Text and video frames can therefore be compared for similarity. Developers have already written a native Rust implementation for local inference. Unsloth also provides a 176 MB, 4-bit GGUF checkpoint for text embedding; this version does not include the full multimodal pipeline.

What he tested was video search: on an Apple M3 Pro using CPU, indexing a 60-second video took 29 seconds, 9 queries averaged 25 milliseconds, and the saved frame vectors took up 180 KiB. The video is indexed only once. Each new search only embeds the query and then compares it with the saved frame vectors. All processing stays on the laptop. Below is the full workflow he recorded:

These numbers cover only a 60-second video. As videos get longer, index storage will grow. Model quantization reduces weight memory, but the retrieval index has its own storage cost. Avi Chawla also wrote a separate article on binary quantization, explaining how to compress vector storage to 1/32 and including a runnable retrieval implementation.

The approach in that article: first use BAAI/bge-large-en-v1.5 to generate float32 embeddings, record dimensions greater than 0 as 1 and the rest as 0, then use np.packbits to pack them into bytes. In Milvus, the field type is BINARY_VECTOR, the index is BIN_FLAT, and the distance is HAMMING. Queries undergo the same binarization, and top-k are taken. The scale given in the article is querying 36M+ vectors on the PubMed dataset in under 30 ms, with answer generation in under 1 s. The tech stack includes LlamaIndex, Milvus, and Kimi-K2 hosted on Groq. The full code is at fastest-rag-milvus-groq.

The appeal of binary quantization is straightforward: one bit per dimension, which is 32 times less raw storage than float32. Perplexity uses it in its search index, Azure uses it in its retrieval pipeline, and HubSpot uses it in its AI assistant. These are claims from the original article.

The cost is also direct. A user, Vishwraj Jadav, tested it on about 50,000 Indian court cases: retrieval was fast and top-k recall was strong, but top-1 accuracy dropped by 8 to 12 percentage points. His advice for legal RAG is binary for retrieval and float for reranking. Another user, Abdel Latrache, also mentioned that Hamming distance works well for large-scale data and that binary quantization greatly helps performance, but reranking is still needed to recover accuracy.

Video search has another layer of cost. A user, Agentik, said he tried sampling frames every second to search screen recordings, and the index quickly ballooned; later he embedded only frames where the scene changes. Sampling less saves space, but whether a shot that appears for only one second will be missed needs separate testing. Ruslan's comment is more direct: only when you search for something that appears for just one second do you know what a smaller index missed.

For ordinary users, this direction paints a concrete picture: type “the child riding a bike alone for the first time” and find that clip among years of family videos, without files leaving the local machine. A user, K4k0, described it this way. As things stand, models and multimodal embeddings can already run locally; what is really hard is indexing strategy, sampling frequency, storage growth, and accuracy trade-offs. Indexing one minute of video in 29 seconds and finding a frame in 25 milliseconds is just an appetizer. Once videos pile up, the real accounting begins.

Original: Avi Chawla's post; binary quantization article.