NVIDIA Nemotron 3.5 Lightning Arrives on Amazon SageMaker JumpStart, Targets High-Volume Agentic Workloads
NVIDIA's Nemotron 3.5 Lightning, a 30B-parameter hybrid Mixture-of-Experts model with only 3B active parameters, is now available for one-click deployment on Amazon SageMaker JumpStart. NVIDIA claims up to 4x higher throughput and 30% faster task completion for high-volume agentic workloads compared to larger frontier models.
NVIDIA Nemotron 3.5 Lightning Now Available on Amazon SageMaker JumpStart
NVIDIA's Nemotron 3.5 Lightning, an open foundation model built for high-volume agentic workloads, is now deployable through Amazon SageMaker JumpStart without manual serving infrastructure setup. The model uses a hybrid Mixture-of-Experts (MoE) architecture with 30B total parameters but only 3B active per forward pass, allowing it to run on a single supported GPU.
Specifications
Nemotron 3.5 Lightning is distilled from NVIDIA's larger Nemotron 3 Ultra and was developed in collaboration with the Nemotron Coalition. Key specs, according to NVIDIA:
- Architecture: Hybrid Mixture-of-Experts
- Parameters: 30B total / 3B active
- Context window: Up to 1M tokens
- Modality: Text in, text out
- Decoding: DFlash speculative decoding for reduced per-token latency
- Throughput: Up to 4x higher than comparable models on high-volume agentic tasks, per NVIDIA
- Task completion speed: Up to 30% faster, according to NVIDIA
The model ships in two variants on SageMaker JumpStart: NVFP4 (model ID huggingface-reasoning-nemotron-3-5-lightning-30b-a3b-nvfp4) and BF16 (huggingface-reasoning-nemotron-3-5-lightning-30b-a3b-bf16). It is also deployable directly from its Hugging Face model page via SageMaker AI, or through the SageMaker Python SDK using JumpStartModel.
Benchmark results
NVIDIA published accuracy comparisons between the BF16 and NVFP4 variants, measured under a consistent internal harness, with evaluation recipes available in NeMo Gym:
| Benchmark | BF16 | NVFP4 |
|---|---|---|
| MMLU Pro | 81.94 | 81.62 |
| GPQA Diamond | 75.44 | 75.57 |
| SWE-bench Verified | 51.56 | 52.80 |
| PinchBench | 85.37 | 83.43 |
| IFBench | 71.88 | 72.88 |
| AA-LCR | 52.00 | 49.19 |
NVIDIA notes these numbers may differ from other vendors' self-reported results since they were generated under NVIDIA's own test harness rather than independently verified.
Pricing and deployment
SageMaker JumpStart does not charge per-token for this open-weight model; instead, users pay standard SageMaker AI compute charges for the GPU instance hosting the endpoint (for example, ml.g6e.12xlarge, ml.p4d.24xlarge, or ml.p5.48xlarge). Per-token or per-hour pricing specifics were not disclosed in NVIDIA's or AWS's announcement. Users are responsible for deleting the endpoint after use to avoid ongoing charges.
Use case framing
NVIDIA and AWS position Lightning as the low-cost, high-throughput tier in a "system-of-models" setup, where frontier models handle complex planning and orchestration while Lightning handles repetitive, specialized steps — alert classification, form field extraction, policy checks — that make up a large share of agentic call volume. Suggested use cases include personal assistants, financial document processing, cybersecurity alert triage, telecom network monitoring, and retail catalog and order management. Organizations can further post-train the model using NVIDIA NeMo for domain-specific tools and policies, though SageMaker JumpStart itself does not currently expose customization options.
What this means
Lightning is not a frontier-capability play — it's an infrastructure and cost-efficiency play. By activating only 3B of 30B parameters per pass and supporting single-GPU deployment, NVIDIA is targeting the growing cost problem of running always-on agents that make many small, repetitive model calls. The claimed 4x throughput and 30% faster task completion figures are NVIDIA's own, measured under its internal test harness (NeMo Gym), so third-party validation will be needed before enterprises can treat these numbers as guaranteed. The lack of disclosed token pricing on SageMaker — users pay for GPU-hours rather than usage — makes direct cost comparisons against hosted API models like GPT or Claude non-trivial, and buyers will need to run their own throughput math based on instance costs.
Related Articles
OpenAI's GPT-6 Astra Reportedly Automates AI Engineering Tasks at Under $6 an Hour, According to Latent Space Testing
A Latent Space report describes GPT-6 Astra, a new OpenAI model the blog says can autonomously handle AI engineering tasks—training models, labeling data, deploying systems—at an estimated cost of under $6 per hour. The claims, including 97.6% on FrontierMath and 99.9% on ARC-AGI-3, come from independent blog testing rather than an official OpenAI announcement.
InclusionAI Releases Ling 3.0 Flash Fin, a Finance-Focused MoE Model with 5.1B Active Parameters
InclusionAI has released Ling 3.0 Flash Fin, a finance-specialized mixture-of-experts model built on Ling 3.0 Flash. The model activates 5.1B of its 124B total parameters and targets long-horizon investment planning tasks while retaining general reasoning, coding, and math capabilities.
Meta Releases Muse Spark 1.3, a Free Multimodal Reasoning Model with 1M-Token Context
Meta has released Muse Spark 1.3, a multimodal reasoning model with a 1M-token context window, listed as free on OpenRouter. The model targets long-running agentic, multi-agent, and coding workflows, though audio input support remains incomplete.
OpenAI's GPT-6 Astra Cuts Hallucinations, But Indirect Prompt Injection Attacks Still Succeed 8.5% of the Time
OpenAI's new GPT-6 Astra model shows major improvements in hallucination rates and jailbreak resistance over predecessor GPT-5.6 Sol, according to OpenAI's system card. However, indirect prompt injection attacks hidden in documents still succeed 8.5% of the time in external testing by Gray Swan, down from 27% but still above rival Claude Opus 5's 4.8% rate.
Comments
Loading...