researchinclusionAI

inclusionAI releases SingProbe, an 8.13M-parameter streaming guardrail for Ling-3.1-flash-1019

TL;DR

inclusionAI published Ling-3.1-flash-1019-singprobe, an 8.13M-parameter probe that reads three hidden-state layers of Ling-3.1-flash-1019 and scores query intent, response unsafety and hallucination risk at each generated token. The model card reports a response safety F1 of 0.8707 and a hallucination AUC of 0.8041.

3 min read
0

inclusionAI has published Ling-3.1-flash-1019-singprobe (SingProbe), an intrinsic streaming guardrail with 8.13M parameters that attaches to the Ling-3.1-flash-1019 base model and scores safety and hallucination signals at every generated token.

How it works

According to the model card, the probe's attention head reads the base model's hidden states at three layers (zero-based indices 23, 47 and 71) during generation. At each token it outputs scores for eight query intents, response unsafety, and hallucination risk.

  • Probe size: 8.13M parameters, released in bfloat16
  • Architecture: 8 attention heads, head dimension 64, 2,048-token window
  • Outputs: 8 query intent labels, 1 response unsafety score, 1 hallucination score
  • Documentation: the SingProbe technical report (arXiv 2608.30703, published Aug 31, authored by the Sing Team) and a SingProbe repository

Reported results

The figures below come from the model card's own evaluation. Each benchmark has equal weight within its suite, and one threshold per F1 row is selected across the suite to maximize mean F1. Query F1 uses generated responses. Response safety F1 uses the responses supplied by each benchmark.

Task Metric SingProbe Reference baseline
Query intent classification (6 benchmarks) F1 0.8687 YuFeng-XGuard-Reason-8B (0.8714)
Response safety classification (8 benchmarks) F1 0.8707 Qwen3Guard-Gen-8B-strict (0.8604)
Streaming safety (3 benchmarks) Response AUC / token AUC 0.9844 / 0.9411 Qwen3Guard-Stream-8B-strict (0.9640 / 0.8893)
Hallucination detection (6 benchmarks) AUC 0.8041 DRIFT (0.8000)

SingProbe trails the YuFeng-XGuard-Reason-8B baseline on query intent F1 and leads the listed baselines on the other three tasks.

On deployment behavior, the card reports a benign-response false-positive rate of 0.008%, averaged over 5 datasets at the evaluation's pre-calibration response threshold of 0.983085. The calibrated release thresholds are 0.5 for response unsafety and 0.5 for hallucination.

Running it

The card says to use the token-probe build from SGLang branch 261029. The example launch uses eight H20 GPUs with tensor parallelism of 8 and expert parallelism of 8. It sets a context length of 16,384 tokens, a chunked prefill size of 1,024, a maximum of 8,192 prefill tokens, a maximum of 32,768 total tokens and a maximum of 16 running requests. The base model path and a --probe-ckpt path for the probe weights are passed to sglang.launch_server.

Per-token scores are returned in meta_info.token_probe_probs in the /generate response. Keys label_0 through label_7 score query intent, label_8 scores response unsafety, and label_9 scores hallucination risk.

No Inference Provider currently deploys the model on Hugging Face. Pricing is not applicable to the weights themselves, and no context window is listed for the probe beyond its 2,048-token attention window.

What this means

SingProbe takes a different route from separate guard models. It reads the hidden states of the model that is already generating, so safety and hallucination signals arrive token by token alongside the output. The baselines in the comparison table are named as 8B models (YuFeng-XGuard-Reason-8B, Qwen3Guard-Gen-8B-strict, Qwen3Guard-Stream-8B-strict), against SingProbe's 8.13M parameters. The results are self-reported on the card's own benchmark suites and threshold-selection protocol. The tight margins on hallucination AUC and query intent F1 matter when comparing it with those baselines.

The probe is tied to Ling-3.1-flash-1019 through its tapped layer indices, and the documented serving path depends on a specific SGLang branch. Teams evaluating it should plan around that base model and runtime.

Related Articles

research

Ai2 launches Theorizer in Asta and open-sources the AI agent that proposes scientific theories

Ai2 launched Theorizer, an AI agent that combines findings from scientific papers to propose explanations for observations, inside its Asta platform. The code is open-source, and Ai2 covers token costs up to a limit in the web interface. A lung cancer case study shows the workflow.

research

Salesforce's BLIP-Live video agent cuts unnecessary responses by over 50%, company claims

Salesforce AI Research introduced BLIP-Live, a general model for live video agents that decides when to speak and when to stay silent. According to Salesforce, in one evaluation across 326 ego-centric videos it reduced unnecessary responses of state-of-the-art models by over 50%.

research

Tunix agentic GRPO lifts Gemma 4 E2B from 9.4% to 95% win rate in Plane Strike

Google's developer blog details how Tunix's agentic GRPO pipeline trained Gemma 4 E2B to play Plane Strike with tool calls. The win rate on held-out boards rose from 9.4% to 95% after 100 steps on a single Cloud TPU v6e-8 host.

research

Google adds Streaming DiLoCo to MaxText, syncing 1/65th of a model per step for cross-datacenter training

Google DeepMind and Google Research engineers describe how MaxText implements Streaming DiLoCo, a low-communication optimizer that splits a model into fragments and syncs one at a time. In a Qwen3-32B worked example, data parallel exchanges ~65.5 GB per step, while Streaming DiLoCo exchanges ~1.0 GB per step.

Comments

Loading...