Large models love to hallucinate; this open-source vision model has learned to say “None”

Before a robot gets to work, it must first see clearly. Decision models like JEV handle judgment, but before judgment, something has to interpret the scene accurately: find everything, count it correctly, and say none when there is none—no hallucinations.

VLX-Seek, open-sourced by Om AI Lab, is an edge vision model built for exactly this. A user called the API and tested it on four videos, all real footage found online.

Four scenarios:

1. Aerial view of a parking lot, checking for “white cars.” About 20 per frame, off by one or two from manual counting; the difference comes from cars partially cut off at the frame edge.
2. Aerial view of a cherry blossom avenue with lots of buses and cars, checking for “people riding horses.” All 3 people riding horses at the bridgehead were boxed, while nearby cars and pedestrians were not counted.
3. At a construction site, checking for “workers wearing yellow hard hats”: 3 to 5 per frame. Then checking for “people not wearing hard hats,” it first lists the helmet color of each of the 13 people in the frame, excludes those wearing helmets, and leaves 4 without one.

4. After the drone flies past the bridge, it checks again for “people riding horses.” It returns None, with 0 boxes.

Of the four scenarios, the fourth is the most worth pondering. Large models love to fill in the blanks when looking at images; they can even invent something that isn't there. Anyone building products knows this pitfall. VLX-Seek's choice is: none means none.

Why is it fast? It first assigns numbers to candidate regions in the frame, with each number corresponding to a region token. When answering, the model only says “number 3 and number 7” instead of emitting coordinate numbers one by one. Traditional approaches have the model directly output sequences of [x1, y1, x2, y2] coordinates; with many targets, output length grows multiplicatively, and a single formatting error ruins it. VLX-Seek changes localization from “generating coordinates” to “referencing regions,” so even dozens of targets don't slow it down.

The tests called the official API, 3B version; one round trip takes 1 to 3 seconds, including uploading a 1440p image. Official data: the 3B model runs a 1080p image in 730 milliseconds on Jetson T5000. If you want speed, you need to run it on edge devices like Jetson; calling it in the cloud is mainly for convenience.

It's not 100%. In crowded scenes with people blocking each other, it occasionally misses things; that should be said up front. The 10B version is much stronger—worth trying.

Deployment details: the 10B weights of VLX-Seek 1.5 were open-sourced in July under Apache 2.0 and can be downloaded from Hugging Face. The API is OpenAI-compatible; put an image plus one sentence in messages, and the response comes directly with box coordinates. If you work on tools, inspection, warehousing, agriculture, or similar, give it a try.

GitHub: https://github.com/om-ai-lab/VLX-Seek
Model demo: http://om-agent.com

One developer commented: getting the scene right first is indeed critical; if you see it wrong up front, everything downstream goes wrong. Another said: if the boxes are off and the count is wrong, everything after that is ruined. Someone also asked: if the same target repeatedly enters and leaves the frame in a video, will counting be duplicated? That's exactly the right question. They also have VLX-Flow, dedicated to continuously watching video streams. Its weights haven't been released yet; whether it can solve cross-frame counting will be known after release.