Google DeepMind's GenCeption uses video generator for computer vision with 500x less training data

Google DeepMind researchers developed GenCeption, which repurposes Alibaba's Wan2.1 video generator for computer vision tasks including depth estimation, segmentation, and 3D pose estimation. The model matches state-of-the-art specialized systems while training on only 7,500 synthetic videos—between 7 and 500 times less data than competing approaches.

Architecture and approach

GenCeption modifies a pre-trained text-to-video diffusion model to perform vision tasks in a single forward pass rather than through iterative denoising steps. The system uses text prompts to specify tasks and represents all outputs—depth maps, surface normals, segmentation masks—as standard three-channel RGB images.

The training dataset consisted primarily of 7,500 synthetic videos created by combining 800 digital human models with 200 motion capture sequences rendered in Blender. Real video data was used only for language-guided segmentation tasks.

Benchmark performance

According to the research paper, GenCeption achieves:

• Depth estimation matching DepthAnything 3
• Surface normal estimation exceeding NormalCrafter and Lotus-2
• 3D pose recognition outperforming Genmo and TRAM
• Language-guided segmentation matching Meta's SAM 3 combined with Gemini 3.5 Flash

Competing methods like D4RT and VGGT Omega trained on millions of videos. GenCeption reaches comparable results using 7 to 500 times less data, according to the study. When tested under identical conditions, pretraining on video generation also surpassed methods including V-JEPA and VideoMAE V2.

Generalization capabilities

Despite training almost entirely on synthetic videos showing single individuals, GenCeption processes real-world footage containing multiple people, animals, and humanoid robots. The researchers claim some outputs contain more detail than the Blender renderings used for training, preserving features like cat whiskers and individual hair strands.

The model has 14 billion parameters in its larger configuration and processes an 81-frame video in approximately 10 seconds.

Technical limitations

Joint training across all tasks degraded 3D keypoint estimation performance. The researchers attribute this to interference between task-specific components and mechanisms learned during pretraining. Processing speed remains a constraint, with the smaller model requiring about 6 seconds per 81-frame video.

Contested world model claims

The authors argue that video generators contain "world models" suitable for foundation models in computer vision, analogous to large language models in text processing. This claim faces pushback from the research community.

An international team recently proposed OpenWorldLib with a definition that explicitly excludes text-to-video models due to lack of real-world feedback. Yann LeCun, former Meta chief AI scientist, has argued that generative video models are "a dead end," advocating instead for approaches like Meta's V-JEPA 2 that predict abstract concepts rather than pixels.

A Tsinghua University benchmark showed models including Sora 2, Seedance 2.0, and Veo 3.1 repeatedly failing basic physics and logic tests despite producing visually convincing output.

What this means

GenCeption demonstrates that pre-trained video generators can be adapted for classical computer vision tasks with minimal additional training data. The 7-500x reduction in required training data compared to specialized models suggests efficiency gains, though the approach doesn't resolve fundamental questions about whether video generators truly learn physical world models. The research provides evidence that video generation pretraining creates useful visual representations, but stops short of claiming these models understand physics or causality—a more modest position than claiming they function as universal world models.