Visa test: 22 AI shopping agents succeeded about 57% of the time; Gemini and Meta agents top at 82%

Visa tested 22 AI agents on everyday purchases and found an overall success rate of only around 57%, according to results Fortune reported on November 5, 2026. Two agents led the field at 82%.

How the test worked

Visa instructed each of the 22 agents to purchase three items: a TV, a laptop and a hair appliance. Fortune describes the study as first of its kind. Each agent completed 15 separate shopping expeditions, and the outcomes are published on a new Visa website.

The test was designed to mimic real consumer behavior. In the TV scenario, the simulated shopper was assigned a list of required attributes, including compatibility with a child's gaming console, a 30-day return window and Dolby Vision. The shopper, itself an agent, volunteered some of that information but not all of it. To succeed, a shopping agent had to question the shopper and draw out the missing details in a reasonable time, then buy a product that matched the shopper's needs.

In many cases, the agents bought the wrong product or failed to suggest anything suitable.

Results

• Top scores: The Aragon 4 version of Google's Gemini and the Wonder version of Meta's Making each made a successful purchase 82% of the time.
• Third place: A Claude agent reached a 76% success rate.
• Sixth place: An OpenAI agent reached a 64% success rate.
• Overall: Success across all agents was around 57%.

Fortune reports that agents powered by more cutting-edge AI models performed best. Visa's chart plots each agent's success rate against the cost to run each shopping inquiry. Visa notes that the five most expensive models succeeded 67% of the time on average, while the five cheapest scored 50%. Per-inquiry cost figures were not given in the article text.

Visa's position

Rubail Birwadker, Visa's Global Head of Growth, told Fortune: "We're not there yet from a trust or technology perspective. But we'll soon get into a world where consumers can trust and delegate." He said agent-driven shopping is in its early days but advancing rapidly.

Birwadker noted that the experiment covers product discovery, which is only one element of agentic commerce. Payments are another: Visa and Mastercard are already educating merchants and building tools for agentic commerce, according to Fortune.

Liability is also unresolved. It is still unclear who bears responsibility when an agent makes a rogue purchase or commits fraud. Birwadker said these questions will be settled over time, as they were for earlier ecommerce, but that agent-based shopping will be a very different experience from online shopping today.

Alongside the results, Visa launched a website where consumers can interact with the agents used in the study.

What this means

The test measured more than whether an agent can complete a checkout. Success depended on drawing out preferences the shopper had not fully stated, then matching them to a product. Under that standard, the best agents succeeded 82% of the time and the overall average was around 57%.

The results also show a gap between vendors' agents: the Claude agent scored 76% and the OpenAI agent 64%. Visa's cost comparison shows that the five most expensive models averaged 67% against 50% for the cheapest five.

Birwadker says product discovery is one piece of agentic commerce. Payments and liability for rogue purchases or fraud remain open issues alongside it.