Rakuten releases RakutenAI-3.0, 671B-parameter Japanese-optimized mixture-of-experts model
Rakuten Group has released RakutenAI-3.0, a 671 billion parameter mixture-of-experts (MoE) model designed specifically for Japanese language tasks. The model activates 37 billion parameters per token and supports a 128K context window. It is available under the Apache License 2.0 on Hugging Face.
Rakuten Releases 671B Parameter Model Optimized for Japanese
Rakuten Group has published RakutenAI-3.0, a 671 billion parameter mixture-of-experts language model engineered for Japanese language understanding and generation. The model activates 37 billion parameters per token and supports a 128,000 token context window.
Technical Specifications
The model uses a mixture-of-experts architecture, a design pattern that maintains computational efficiency by selectively activating only a subset of parameters for each input token. RakutenAI-3.0 is trained on a combination of publicly available open-source data and Rakuten's proprietary bilingual Japanese-English datasets.
Key specifications:
- Total parameters: 671 billion
- Active parameters per token: 37 billion
- Context window: 128,000 tokens
- Supported languages: Japanese and English
- Model format: F32, BF16, and F8_E4M3 quantization variants available
- License: Apache License 2.0
Deployment and Access
RakutenAI-3.0 is available on Hugging Face for download and local deployment. The company provides inference instructions using SGLang with recommended specifications requiring 8 tensor parallelism and 85% static memory allocation. The model has recorded 425 downloads in its first month on Hugging Face.
No official inference API or hosted endpoints have been announced. The model card indicates the model is not currently deployed by commercial inference providers.
Positioning
Rakuten positions RakutenAI-3.0 as delivering "superior grasp of Japanese language and culture" compared to existing models. The emphasis on Japanese-optimized training reflects increasing focus by regional technology companies on language-specific LLMs, following similar releases from companies like Alibaba (Qwen) and Baidu.
Limitations
Rakuten's documentation explicitly acknowledges that RakutenAI-3.0 can generate biased, inaccurate, or unsafe outputs like other large language models. The company recommends implementing appropriate safeguards for production deployments.
What This Means
Rakuten's entry into open-source Japanese-optimized LLMs signals sustained competition in regional language models. At 671B parameters with a 128K context window, it competes in scale with existing open models but targets a specific linguistic niche. The Apache 2.0 license and community release suggest Rakuten is prioritizing ecosystem participation over proprietary monetization, similar to Meta's approach with Llama. The model's availability only through local deployment (no hosted API) limits accessibility for developers without substantial compute resources.
Related Articles
OpenAI Releases Whisper Large-v3, Cutting Speech Recognition Errors 10-20% Across Languages
OpenAI has released Whisper large-v3, an open-weight automatic speech recognition and translation model trained on 5 million hours of audio. The model reduces transcription errors by 10-20% compared to its predecessor and adds native Cantonese support.
Upstage Releases Solar Pro 4 With 524K Token Context Window at $0.03/M Input Pricing
Upstage has released Solar Pro 4, a large language model with a 524K token context window aimed at agentic workflows, document processing, and coding. The model is priced at $0.03 per million input tokens and $0.12 per million output tokens, and is available now via OpenRouter.
Meta Open-Sources Muse Spark 1.2, Announces On-Device Model Family Muse Glimmer
Meta CEO Mark Zuckerberg announced the company will open-source its Muse Spark 1.2 model and launch a new on-device model family called Muse Glimmer. The move positions Meta against closed-model rivals OpenAI and Anthropic and against Chinese open-weight labs like DeepSeek and Alibaba.
Meta Releases Muse Glimmer 30B, an On-Device Agentic Model with Built-In Perception Encoder
Meta Superintelligence Lab has released Muse Glimmer, a 29.6-billion-parameter multimodal model distilled from Muse Spark for autonomous agentic tasks that run entirely on consumer hardware. The Apache 2.0-licensed model ships with a dedicated perception encoder, 131K+ token context, and speculative decoding for local speedups up to 3.1x.
Comments
Loading...