content moderation

Musubi released PolicyLM-1.7B on Tuesday, a 1.7-billion-parameter open-weights decision model built for real-time content moderation. The company claims it applies plain-English policies to messages in under 50 milliseconds and needs no retraining when policies change.

Taiwan retail platform uniopen adapted Amazon Nova 2 Lite to its two-axis moderation policy using LoRA supervised fine-tuning in Amazon SageMaker AI, plus a prompt-format change. According to AWS, Per Behavior Macro F1 rose from 0.5852 to 0.8550 and Subject Type Macro F1 from 0.4162 to 0.8491, both above production targets.

A researcher's multi-turn jailbreak technique reliably pushes Claude Opus 4.6, Opus 3, and Haiku 4.5 into generating sexually explicit content that Anthropic's usage policy explicitly prohibits. Newer models, Opus 4.7 through Opus 5, resist the same technique.

Anthropic has begun embedding invisible watermarks in Claude's text outputs to comply with the EU AI Act's Transparency Code. The move has triggered backlash from some users worried the watermarks will expose their undisclosed use of AI at work or in school.

Mistral has released Shieldstral, a 3B open-weights safety classifier that reframes content moderation as a policy-adaptive question-answering task. The model claims to match or outperform guard models up to 7x its size on text safety and multimodal benchmarks, and runs on a single 16GB GPU.

Meta has released a web-based detection tool for identifying images created or edited with its new Muse Image model using invisible Content Seal watermarks. The watermarks persist through cropping, compression, resizing, and screenshots, though the tool has rate limits and isn't compatible with SynthID or C2PA standards.

Meta launched AI Mode on Facebook, a search feature that uses Meta AI to generate answers from public posts, Groups, and Reels instead of displaying traditional search results. The feature allows users to ask questions in plain language and receive synthesized responses based on platform discussions.

NVIDIA has released Nemotron 3.5 Content Safety, a 4B-parameter model built on Google Gemma 3 4B IT that provides multimodal safety classification across approximately 140 languages. The model includes a 128K context window, custom enterprise policy enforcement, auditable reasoning traces, and is releasing its training dataset.

YouTube is replacing keyword search with Gemini-powered conversational queries for Premium members over 18. The platform also integrates Gemini Omni, Google's latest generative model, into Shorts Remix to enable AI-assisted video recreations with creator consent controls.

OpenAI launched an optional Trusted Contact feature for ChatGPT that notifies designated emergency contacts when the system detects discussions about self-harm or suicide. The feature requires manual review by trained personnel before sending notifications, and does not share chat transcripts with contacts.