model releaseByteDance

ByteDance releases Lance, 3B-parameter unified multimodal model handling image and video generation, editing, and unders

TL;DR

ByteDance has released Lance, a 3-billion parameter multimodal model that performs image and video generation, editing, and understanding within a single framework. The model was trained entirely from scratch using 128 A100 GPUs and achieves 84.67% on DPG-Bench and 74% on GenEval, competing with larger models despite its compact size.

2 min read
0

ByteDance Releases Lance, 3B Unified Multimodal Model

ByteDance has released Lance, a 3-billion parameter model that handles text-to-image generation, text-to-video generation, image editing, video editing, and visual question answering in a single unified framework. The model was trained entirely from scratch using 128 A100 GPUs.

Technical Specifications

Lance operates with 3 billion active parameters and supports video generation up to 121 frames at 768×768 resolution (480p preset). According to ByteDance, the model uses flow matching scheduling with a default timestep shift of 3.5 and 30 denoising steps. The architecture requires at least 40GB VRAM for inference.

The model's training used a "staged multi-task recipe," though ByteDance has not disclosed the training dataset size, training duration, or data cutoff date. Pricing information has not been announced.

Benchmark Performance

On DPG-Bench, a comprehensive image generation evaluation, Lance scores 84.67% overall, with particularly strong performance in relation understanding (93.38%) and entity recognition (91.07%). The model trails larger unified models like TUNA-27B (86.54%) and InternVL-U (85.18%) but outperforms the 7B BAGEL model.

For GenEval, which tests compositional image generation across attributes like object count and spatial positioning, Lance achieves 74% overall. This matches SD3-Medium (2B parameters) but falls behind FLUX.1-dev's 75% (though FLUX.1-dev uses 12B parameters).

ByteDance reports competitive scores on specific categories: 99% for single-object generation, 94% for two-object generation, and 72% for counting accuracy.

Capabilities

The model handles six distinct task types through a unified interface: text-to-image, text-to-video, image editing, video editing, image understanding (visual question answering), and video understanding (video captioning and analysis). ByteDance demonstrates video understanding capabilities including counting actions, spatial reasoning, and temporal analysis.

For generation tasks, Lance uses classifier-free guidance with a default scale of 4.0 for text conditioning. The model supports multi-turn consistency editing, maintaining coherent changes across sequential edit operations.

Availability

Model weights are available on Hugging Face under the bytedance-research organization. ByteDance provides a command-line inference tool and Gradio interface. The system requires Python 3.10+ and CUDA 12.4+.

What This Means

Lance represents ByteDance's entry into unified multimodal AI, directly competing with models like DeepSeek-Janus, Show-o, and OmniGen. At 3B parameters, it's significantly smaller than most unified models while maintaining competitive performance on standard benchmarks. The efficiency suggests progress in model architecture design, though the lack of disclosed training details makes it difficult to assess reproducibility or training costs beyond the stated 128-GPU budget. The model's commercial viability will depend on pricing, which ByteDance has not yet announced.

Related Articles

research

Bytedance-led team argues three training steps are enough for AI agents, releases competitive open model

A team from Bytedance and several US universities argues in a new paper that three training steps are enough to build capable AI agents. The authors say stacked, multi-phase pipelines make it hard to tell which step helps and make reproduction expensive. According to The Decoder, the work is backed by a competitive open model.

model release

Thinking Machines Lab releases Inkling 2 Mini: 26B-parameter looped MoE with 3B active and 1M-token context

Thinking Machines Lab has released Inkling 2 Mini, a multimodal Mixture-of-Experts model with about 26B physical total parameters and 3B active per token. It repeats its middle layers three times, supports a 1M-token context, and ships with a DSpark drafter. Hugging Face reports it leads 32 of 48 benchmarks against four comparable models.

model release

Tencent releases Hy4 flagship weights: 770B-parameter MoE, 1,048,576-token context, conditional license

Tencent released downloadable weights for its flagship Hy4 on November 20, with a System Card. The multimodal MoE has a 770-billion-parameter language backbone, 49 billion active parameters per text token and a 1,048,576-token context window, under a license with commercial conditions for model-as-a-service operators.

model release

Tencent releases Hy4 flash: 220B MoE, 12B active parameters, 1M-token context

Tencent has released Hy4 flash, a multimodal Mixture-of-Experts model with 220B total parameters, 12B active per token, and a 1,048,576-token context window. Tencent calls it its most cost-effective Hy model to date and claims a per-task cost of 1/20 that of Hy4 on its internal WorkIndex benchmark.

Comments

Loading...