Vals AI study: Chinese models' pro-China lean is lower than expected, and shifts with prompt language

Benchmark firm Vals AI found that Chinese language models show a lower pro-government lean on China-related questions than expected, with major differences between individual models, according to a study reported by The Decoder on Nov 1, 2026.

The researchers asked Chinese, American and other models hundreds of questions about China, varying the prompt language, usage context and response format. Vals calls the measured alignment with Chinese government positions a "pro-China lean." Scores run from -1 to +1: positive values indicate agreement with official positions or a pro-government stance, negative values indicate more critical responses. The scale measures alignment with certain government positions, not factual accuracy.

Key findings

• Model variation is wide. GLM-5.3 Flash explicitly confirmed that China's human rights violations are well documented when asked. Some non-Chinese models occasionally aligned with official Chinese positions.
• Prompt language matters. For most models, answers in Simplified Chinese were more pro-government than answers in English, Russian or Spanish. The effect appeared in both Chinese and American models but was far more pronounced in Chinese ones. Vals also found a smaller but consistent gap between Simplified Chinese and Traditional Chinese.
• Context matters. When a system prompt said the model was used internally by a company on mainland China, some models answered in a more pro-government way than when the prompt indicated US-based use. Vals suspects models infer the expected perspective from the language itself.
• Refusals are rare. Only certain DeepSeek models blocked sensitive questions in Vals' tests. Most other Chinese models rarely refused and instead sometimes adopted official positions. Kimi K3, for example, denied the existence of concentration camps for Uyghurs in Xinjiang, calling the facilities vocational training centers designed to combat extremism and terrorism.
• Format changes scores. Under yes/no answer formats, GLM-5.3 Flash and MiMo V2.6 Pro scored much less pro-government than with open-ended responses.
• Agentic setups lower scores. With coding-assistant system instructions and tool definitions, several Chinese models scored lower on the pro-government scale, though they rarely used the tools. Giving models web search access had a similar effect even though they never actually searched.

Everyday tasks

According to Vals, political tendencies were much harder to detect in realistic work tasks than in targeted test questions. Individual examples still appeared. When translating a list of country codes for a travel app into Simplified Chinese, most models rendered "Taiwan" without a qualifier. Qwen3.8 Max produced "Taiwan, China," a label used by the Chinese government.

OpenAI's image model GPT Image 2.5 Sunburst also showed language-dependent differences in the map examples Vals published. Prompted in Chinese, it labeled Taiwan as "Taiwan Province" and added the nine-dash line. The English-language result shown did not include these elements.

Fine-tuning effect

Thomson 1.0 Small, a model fine-tuned by Thomson Reuters from Qwen3.6 35B, adopted official Chinese positions far less often than the base model. The share dropped from 84.6 to 15.0 percent. Some individual answers still defended Beijing's One-China principle and "peaceful reunification" with Taiwan.

Methodology limits

Vals says its political questions are not representative of typical user prompts. Questions were translated with GPT-6 Astra and spot-checked. Responses were scored automatically by GPT-5.6 Luna. Refusals counted as neutral by default, so a model that blocks every question could appear neutral on the scale. Many Chinese models were tested through third-party providers rather than official services, which may apply additional content filters. Results therefore do not necessarily carry over to publicly available chatbots.

Earlier studies by Aleph Alpha and NewsGuard had already shown that Chinese models frequently adopt official narratives on sensitive topics. Chinese regulations require publicly offered AI services to align with "core socialist values."

What this means

The results show that a model's political output depends on how it is prompted, not only on who built it. Prompt language, a stated deployment location, answer format and agentic scaffolding each moved scores in Vals' tests. Anyone comparing models on politically sensitive topics needs to hold those variables constant. The neutral treatment of refusals and the use of third-party providers also mean the scores describe tested API behavior, not the behavior of official chatbots. The Thomson 1.0 Small result shows that targeted fine-tuning changed measured behavior substantially, though some pro-Beijing answers remained.