Qwen3.6-35B-A3B Outperforms Claude Opus 4.7 on SVG Generation Test
In an informal SVG generation benchmark, Alibaba's Qwen3.6-35B-A3B model running locally via a 20.9GB quantized version outperformed Anthropic's newly released Claude Opus 4.7. The test, which asked models to generate SVG illustrations of pelicans and flamingos on bicycles, showed the smaller local model producing more accurate bicycle frames and more creative outputs.
Qwen3.6-35B-A3B Outperforms Claude Opus 4.7 on SVG Generation Test
Alibaba's Qwen3.6-35B-A3B model running locally produced more accurate SVG illustrations than Anthropic's Claude Opus 4.7 in an informal benchmark test, according to developer Simon Willison's comparison published April 16.
The test asked both models to generate SVG code for a "pelican riding a bicycle." Qwen3.6-35B-A3B, running via a 20.9GB quantized model (Qwen3.6-35B-A3B-UD-Q4_K_S.gguf) on a MacBook Pro M5 through LM Studio, produced a correct bicycle frame with clouds and a detailed pelican pouch. Claude Opus 4.7 generated an incorrect bicycle frame shape in both standard and maximum thinking mode.
Benchmark Details
The Qwen model ran entirely locally using the quantized GGUF format from Unsloth. Opus 4.7 ran via Anthropic's API. Both models were tested on the same prompt without modification.
In a follow-up test using "flamingo riding a unicycle" to verify the results weren't due to training on the specific benchmark, Qwen3.6-35B-A3B again produced what Willison judged to be superior output, including creative details like sunglasses and a bowtie on the flamingo, along with SVG comments.
Model Specifications
Qwen3.6-35B-A3B:
- Parameter count: 35 billion
- Quantized size: 20.9GB (Q4_K_S format)
- Deployment: Local via LM Studio
- Released: April 16, 2026 (announced by Alibaba)
Claude Opus 4.7:
- Parameter count: Not disclosed
- Deployment: API only
- Released: April 16, 2026 (announced by Anthropic)
- Tested with both standard and maximum thinking levels
Analysis Limitations
Willison noted that this informal benchmark tests only a narrow capability and should not be interpreted as evidence that the quantized Qwen model is generally more capable than Opus 4.7. "I very much doubt that a 21GB quantized version of their latest model is more powerful or useful than Anthropic's latest proprietary release," he wrote.
The benchmark has historically correlated with general model capability improvements since October 2024, when early models produced poor results. Recent flagship models like Gemini 3.1 Pro have generated production-quality illustrations on this test.
What This Means
This result demonstrates that specialized performance on specific tasks can vary significantly between models regardless of overall capability or size. A 35B parameter model running locally in quantized form matched or exceeded a flagship proprietary model on SVG generation, while likely trailing in most other benchmarks.
The finding also highlights the growing sophistication of local LLMs. A model small enough to run on consumer hardware (20.9GB) can now compete with cloud-based flagship models on certain creative tasks, though general-purpose performance gaps remain significant.
For developers specifically needing SVG generation capabilities, this suggests testing multiple models on actual use cases rather than relying solely on general benchmark scores or parameter counts.
Related Articles
Robot Safety Benchmark Finds GPT-6 Astra and Claude Fable 5.1 Rarely Refuse Dangerous Commands
A new benchmark called RoboHarm tested whether AI models controlling robotic arms would refuse dangerous commands. GPT-6 Astra completed 60 of 100 dangerous tasks and Claude Fable 5.1 completed 34, with neither model showing a reliable safety layer.
Composio Benchmark: Claude Code Fastest Agent Framework, But Costs Nearly 3x More Than OpenCode
Composio benchmarked DeepSeek V4 Flash across four agent frameworks—Claude Code, Codex, OpenCode, and Oh My Pi—on 30 real-world tasks. Claude Code finished fastest at 122 seconds per task but cost $0.195, nearly three times OpenCode's $0.073, while Oh My Pi had the highest success rate at 17/30 but took 272 seconds per task.
Qwen3.8 Max Matches Claude Opus 4.8 on Intelligence Index, But Costs 2x More Per Task Than Predecessor
Alibaba's Qwen3.8 Max jumps 10 points to 56 on the Artificial Analysis Intelligence Index, putting it on par with Claude Opus 4.8. But Kimi K3 still edges it out at a lower per-task cost, and Qwen3.8 Max shows a sharp rise in hallucination rate.
OpenAI Says GPT-5.6 Sol Scores 38.3% on ARC-AGI-3, Beating Claude Opus 5 — But Only With a Custom Harness
OpenAI says GPT-5.6 Sol scores 38.3 percent on ARC-AGI-3 using a custom Responses API configuration, edging out Claude Opus 5's 30.2 percent. Under the official benchmark harness, GPT-5.6 Sol's score drops to 7.8 percent, raising questions about what the comparison actually measures.
Comments
Loading...