model architecture
3 articles tagged with model architecture
DeepSeek Ships V4.1-Flash With Novel Encoder-Decoder Architecture, Cuts KV Cache to 1/8 of Predecessor
DeepSeek released V4.1-Flash, a 763B-parameter model built on a new causal encoder-decoder architecture that splits 8B active parameters for prefill and 16B for decode. The model adds native vision support, a 1M-token context window, and shrinks KV cache footprint to roughly 1/8 of DeepSeek V4 Flash, while retiring V4 Pro.
Safety Researchers Warn OpenAI's Unreleased Astra Model May Hide Its Reasoning From Monitors
OpenAI has delayed the release of its next flagship model, Astra, after reports it may use a more opaque 'recurrent depth' architecture that hides more of its reasoning from safety monitors. AI safety researchers, including Redwood Research's Ryan Greenblatt, called the potential shift one of the worst developments for AI safety to date.
AI2 Research: Hybrid Models Excel at Content Words, Transformers Better at Token Repetition
Allen Institute for AI researchers conducted token-level analysis comparing their 7B-parameter Olmo 3 transformer and Olmo Hybrid models. The study finds hybrid architectures show a loss gap advantage of 0.04 on content words (nouns, verbs, adjectives) versus 0.02 on function words, while transformers match or exceed hybrids on repeated tokens and closing braces.