Seeing as Humans Do: Learning from Motion to Segment Anything Without Supervision

Accepted to European Conference on Computer Vision (ECCV), 2026

Important

Code is under internal review and will be released soon.

Overview

Segment Anything models are typically trained with massive collections of manually annotated masks. We present Motion-Grounded Segment Anything (MoSA), a scalable unsupervised framework that instead learns a transferable objectness prior from unlabeled videos.

MoSA learns in three progressive stages:

1. Motion pseudo-label generation. A Multi-Granularity Motion Segmentation (MGMS) pipeline extracts high-quality object and part masks from large-scale unlabeled videos.
2. Learning objectness from motion. A Perceptual Grouping Model (PGM), trained with Perceptual Grouping Contrastive Learning (PGCL), turns sparse motion supervision into a general, appearance-driven notion of objectness.
3. Adaptation to Segment Anything. The learned prior is transferred to whole-image and point-prompted segmentation models for zero-shot inference on static images.

Despite using zero manual annotations, MoSA establishes a new state of the art among unsupervised segmentation methods and achieves performance comparable to the fully supervised SAM.

Highlights

• Fully unsupervised: no human-provided labels, boxes, or masks are used for training.
• Scalable motion supervision: MGMS produces approximately 21 million multi-granularity pseudo-labels from 10,000 hours of unlabeled video.
• Generalizable perceptual grouping: PGCL learns object coherence beyond moving regions and transfers to unseen static objects and domains.
• Multi-granularity segmentation: MoSA handles objects, instances, and fine-grained parts in both automatic and promptable settings.

Main Results

Whole-image segmentation

MoSA achieves an average recall of 42.1 AR across seven benchmarks, consistently outperforming prior unsupervised methods.

Method	COCO	LVIS	ADE20K	EntitySeg	SA-1B	PartImageNet	PACO	Average
CutLER	28.1	20.2	26.3	23.1	17.0	28.7	8.9	21.8
SOHES	30.5	29.1	31.1	33.5	33.3	36.0	17.1	30.1
UnSAM	40.5	37.7	35.7	39.6	41.9	51.6	27.5	39.2
MoSA (ours)	43.5	42.2	38.4	41.1	48.2	52.7	28.4	42.1

All results are reported using Average Recall (AR). MoSA uses a ResNet-50 backbone and 1% of SA-1B images during adaptation.

Point-promptable segmentation

On COCO Val2017, MoSA improves over the previous unsupervised state of the art while using the same Swin-Tiny backbone and 1% of SA-1B images.

Method	Supervision	MaxIoU	OracleIoU
SAM	Supervised	52.1	68.2
UnSAM	Not strictly unsupervised†	40.3	59.5
MoSA (ours)	Unsupervised	41.6	63.4

_{† UnSAM uses CascadePSP, which is trained with manual annotations, for mask refinement.}

Updates

• 2026/06: The paper has been accepted to ECCV 2026.
• 2026/07: Code is under internal review.