ByteDance Open-Sources Bernini-R Video Diffusion Model With Semantic Planning Architecture
ByteDance released Bernini-R, an open-source video generation and editing model that combines an MLLM-based semantic planner with a DiT-based renderer. The model requires Hopper-class GPUs (H100/H800/H200) for optimal performance and supports multiple tasks including text-to-video, video editing, and reference-guided generation.
ByteDance Open-Sources Bernini-R Video Diffusion Model With Semantic Planning Architecture
ByteDance released Bernini-R, an open-source video generation and editing model that combines an MLLM-based semantic planner with a DiT-based renderer. The company made the inference code and model weights available on June 1, 2025, following a research paper published May 22.
Technical Architecture
Bernini-R is built on the Wan2.2 base model (Wan-AI/Wan2.2-T2V-A14B) and uses a dual-checkpoint architecture with separate high-noise and low-noise transformer weights. The model requires:
- Python 3.11.2
- CUDA 12.4 (minimum 12.3)
- PyTorch 2.5.1+cu124
- Hopper GPU (H100/H800/H200) recommended for FlashAttention-3 support
- Other CUDA GPUs fall back to FlashAttention-2 or PyTorch SDPA
The system uses pinned dependencies: diffusers 0.35.2, accelerate 0.34.2, and transformers 4.57.3.
Capabilities and Performance
The model supports seven task types:
- Text-to-image (t2i)
- Image editing (i2i)
- Text-to-video (t2v)
- Video editing (v2v)
- Motion-aware video editing (mv2v)
- Reference + video editing (rv2v)
- Reference-to-video (r2v)
According to ByteDance, Bernini reaches "the first tier among leading closed-source commercial models" on video editing tasks. This claim is based on a self-built arena platform where human annotators blindly vote on paired edits, aggregated into Bradley-Terry scores and win-rate matrices. No specific numerical benchmarks were disclosed.
Default inference outputs 480p video at 16fps (848px max image size). The system supports higher resolutions, with examples showing 720p/24fps output at 1280px max size.
Deployment Requirements
Single-GPU inference works for image tasks (t2i, i2i with --num_frames 1). Video tasks use 8-GPU configurations via torchrun with Ulysses sequence parallelism. The --ulysses flag controls N-way sequence parallel per sample, with remaining GPUs running data parallel over task lists.
Multi-GPU setups require Open-VeOmni (Apache-2.0, Python 3.11) for sequence parallelism, though single-GPU inference does not need this dependency.
Model Access
Two distribution methods are available:
- Diffusers format (recommended): ByteDance/Bernini-R-Diffusers on Hugging Face — self-contained directory bundling VAE, UMT5 text encoder, tokenizer, and Bernini-R weights
- Separate checkpoints: Base Wan2.2 model plus Bernini-R high-noise/low-noise weights from ByteDance/Bernini-R
ByteDance recommends using a prompt enhancer (--use_pe flag) through an OpenAI-compatible endpoint for best generation quality. The enhancer requires configuring BERNINI_PE_API_KEY, BERNINI_PE_BASE_URL, and BERNINI_PE_MODEL environment variables.
What This Means
Bernini-R represents ByteDance's entry into open-source video generation, competing with models from Stability AI and others in the video diffusion space. The Hopper GPU requirement (H100/H800/H200) creates a high barrier to entry — these GPUs cost $25,000-40,000 each and are primarily available through cloud providers. The dual-checkpoint architecture and multi-GPU requirements suggest this is designed for research labs and companies with substantial compute budgets rather than individual developers. ByteDance's self-reported performance claims need independent verification, as the company provided no standardized benchmark scores against public datasets.
Related Articles
DeepSeek Releases V4-Flash-Vision-Exp, First Multimodal Model in V4 Family
DeepSeek has released DeepSeek-V4-Flash-Vision-Exp, its first experimental multimodal model in the V4 family, adding visual understanding to the V4-Flash architecture. The 305B-parameter model shows substantial gains on multimodal agent benchmarks while holding steady on text-only tasks.
Tencent Open-Sources Hy4 Preview: 770B-Parameter MoE Model with 1M-Token Context
Tencent's Hy Team has open-sourced Hy4 preview, a 770-billion-parameter Mixture-of-Experts model with 49 billion activated parameters and a 1-million-token context window. The model is available under Apache 2.0 alongside an FP8-quantized variant, with Tencent claiming it beats GLM 5.3 and Kimi K3 on internal engineering evaluations.
GLM-5.3-Flash Debuts as Zhipu AI's First Natively Multimodal Model, 320B Parameters with 18B Active
Zhipu AI has released GLM-5.3-Flash, the first natively multimodal model in its GLM-5 series, built on a 320B-parameter mixture-of-experts architecture with only 18B active parameters. The company claims it outperforms GLM-5.2 while approaching Claude Opus 4.8 on coding and agentic benchmarks at a fraction of the cost. Unsloth has published quantized GGUF versions for local inference.
Z.ai Launches GLM-5.3-Flash: 1M-Token Context, Image Support, Claimed 10x Cost Cut Over GLM-5.2
Z.ai has released GLM-5.3-Flash, a 320-billion-parameter Mixture-of-Experts model with 18 billion active parameters, a 1-million-token context window, and image input support. The model launched on LM Studio's Bionic platform hours after its official unveiling, with LM Studio claiming it is 9-10x cheaper to run than GLM-5.2.
Comments
Loading...