Black Forest Labs trains FLUX 3 on images, video and sound together, founders say

TL;DR

Black Forest Labs co-founder Andreas Blattmann and Dustin Podell, who led FLUX Kontext, explain why FLUX 3 trains on images, video and sound together. The Latent Space interview also covers autoregression versus diffusion, video upscaling and Kontext's origins.

3 min read
0

Black Forest Labs (BFL) trains FLUX 3 on images, video and sound together, according to an interview with Andreas Blattmann and Dustin Podell published by Latent Space (an English-language outlet). Blattmann co-created Stable Diffusion and co-invented the latent diffusion method behind it. Podell led FLUX Kontext and was the lead author of SDXL.

Why three signals

Blattmann describes visual intelligence with a ball-catching example: when someone tosses you a ball, your hand starts moving before the ball arrives, even though you could not write down its trajectory. Per the interview, each signal contributes something different:

  • A still shows where the ball is.
  • A video shows how it got there.
  • The sound of impact indicates when it hit something, even if the camera missed the contact.

That is BFL's argument for training FLUX 3 on all three together. The interview and show notes point to FLUX 3 Video demos that generate video and sound together.

What a moving camera teaches

Blattmann says ordinary two-dimensional video contains a great deal of evidence about what objects look like from other angles. If a camera pans around a person, the model has to produce the back of their head at some point. The host notes that this does not settle whether a model "understands physics," a point the speakers debate, but it gives a better test than whether a single frame looks convincing.

Video upscaler built on the FLUX 3 backbone

BFL derives its video upscaler from the same FLUX 3 backbone. The reason is temporal consistency: if each frame of a clip is sharpened independently, the added detail can shimmer on playback because the upscaler made a slightly different decision on every frame. Consistency matters even when the user only wants a higher-resolution file.

Autoregression versus diffusion

Asked whether autoregression is taking over image generation, Blattmann credits both sides. An autoregressive model can learn from all the tokens in a training example at once. On the diffusion side, BFL can distill a long sampling process down to a handful of steps. Which advantage matters depends on whether the cost that counts is training the model or waiting for an image to finish.

How Kontext started

Podell says he suspected an image model that could draw a dog wearing a hat already knew something about where hats go. He and a colleague collected a small set of photos labeled with relationships. A training run of a few hours got the model to put a logo on a shirt. The result was rough but good enough to show at the next offsite. The team then spent months on the data and the model.

Podell also says users fed FLUX old, degraded family photos, restored them and showed the results to their families. He says he had not predicted that use.

Other topics in the episode

The chapter list also covers SDXL, flow matching, how BFL started, training data, testing an edit, world action models, robotics and who uses FLUX. The host says BFL formed when Anj Midha helped the core group from Stability AI spin out.

What this means

BFL's stated rationale for FLUX 3 is that stills, motion and sound each carry different information about the physical world, so training on them jointly is meant to build visual intelligence rather than only image quality. The upscaler example shows the same backbone being reused where frame-to-frame consistency matters. The Kontext story shows how the team tests an idea cheaply, with a few hours of training on a small labeled set, before committing months to data and model work.

Related Articles

analysis

Replicate advises starting with MiniMax H3 Highspeed at $0.06 per second, using H3 for 2K

Replicate recommends starting with minimax/h3-highspeed, dropping to minimax/h3-lite for large batches, and moving up to minimax/h3 for 2K output or difficult shots. A 10-second clip at 768P costs $0.60 on Highspeed, $0.40 on Lite and $0.80 on H3.

analysis

Moonshot AI shifts Kimi focus from coding to science, citing diminishing returns in coding

Moonshot AI is redirecting its Kimi language models toward biology, chemistry and pharma research because it sees diminishing returns in coding. International director Giacomo Ficari said science demands smarter models than law, finance or code. DP Technology is already integrating Kimi K3.1 Pro into its BioMaster research assistant.

analysis

OpenAI report examines how more than 1,800 scientists use ChatGPT in research

OpenAI has published "Science at the Frontier: How scientists use ChatGPT," covering more than 1,800 scientists who use ChatGPT in research. The findings address daily workflows, reported productivity, and expectations for discovery.

analysis

Anthropic says Claude Mythos Preview onboarded AMD MI355X GPUs in about three days, versus six to 12 months

Anthropic told IPO investors that Claude Mythos Preview did the software work to run its AI on AMD MI355X GPUs in about three days, a job it says historically took six to 12 months. The three days cover prompt processing for one model only, and no performance comparison with Nvidia has been given.

Comments

Loading...