The Overlooked Meta Paper: JEM Beats DINOv3 with 1/12 the Data

Tanishq Mathew Abraham dug up a Meta FAIR paper on X titled “Scalable Patch-Level Self-Supervised Learning.” He said the paper has gone completely unnoticed.

The paper proposes a new framework, JEM, which belongs to student-teacher self-supervised methods: the student network predicts the representations produced by the teacher network, with the core being to align patch representations of the same image under different views. The authors add an explicit regularization term to ensure each patch representation contains enough information and to prevent structural collapse. A user asked under the tweet: the DINO series has always relied on tricks like centering to prevent collapse—what does JEM rely on? He replied: the paper has an entropy-based regularization term.

The results are in the abstract. The paper validates across the 300M to 7B parameter range. JEM surpasses DINOv2 on dense tasks under the same conditions; the 7B-parameter version beats the DINOv3 checkpoint on panoptic segmentation, using only 1/12 of DINOv3's training data and without an additional refinement stage. Someone in the comments specifically checked the abstract and confirmed that it is JEM, not DINOv3, that uses 12x less data.

This result is counterintuitive. Self-supervised learning does not rely on human annotations, and such models are usually thought to be built up through larger scale and more data. JEM overtaking DINOv3 with far less data at least shows that DINOv3's data efficiency still has some room for doubt. If the community can reproduce it without problems, this will become the next direction for self-supervised pretraining to compete on.

The paper's authors include Maximilian Seitzer, Maxime Oquab, Oriane Siméoni, and others, from Meta FAIR.

A paper with a plain title that doesn't even include the word SOTA is easy to get swept away in the information feed. The results are there, and they have more reach than the title itself.