Agoranome Research
Machine Choice Benchmark
Which live x402 offers buyer-agent models choose, published by Agoranome
In web search, web.tollkit.dev took 37% of 27 picks. In chain data, agent402.tools took 65% of 26 picks; the first slot gained 14 points.
Report #1 · 2026-10-27 · first report · 108 decisions
Web search
27 decisions; 0 chose none.
| Host | Picks | Share of picks | Picked when shown |
|---|---|---|---|
| web.tollkit.dev | 10 | 37% | 56% (34–75%) |
| api.agentstools.dev | 7 | 26% | 78% (45–94%) |
| serp402.com | 3 | 11% | 33% (12–65%) |
| api.kadec0.xyz | 2 | 7% | 22% (6–55%) |
| webtools402.176-109-107-191.sslip.io | 2 | 7% | 22% (6–55%) |
| agdata.shveik.dev | 1 | 4% | 11% (2–44%) |
| agentsvc.io | 1 | 4% | 11% (2–44%) |
| search.openverbs.com | 1 | 4% | 6% (1–26%) |
| agent-data-tools.revmesh2074.workers.dev | 0 | 0% | 0% (0–18%) |
| agent402.tools | 0 | 0% | 0% (0–30%) |
Position
| Slot | Picked | If position didn't matter |
|---|---|---|
| 1 | 22% (11–41%) | 18% |
| 2 | 37% (22–56%) | 18% |
| 3 | 11% (4–28%) | 18% |
| 4 | 11% (4–28%) | 18% |
| 5 | 15% (6–32%) | 18% |
| 6 | 6% (1–26%) | 17% |
Price
Not estimated: needs at least 30 decisions with prices; there were 27.
Proof fields
| Field added | Pairs | With | Without | Verdict |
|---|---|---|---|---|
| settlement | 7 | 43% (16–75%) | 43% (16–75%) | inconclusive |
| capture | 9 | 22% (6–55%) | 33% (12–65%) | inconclusive |
| receipts | 9 | 33% (12–65%) | 11% (2–44%) | inconclusive |
| any | 25 | 32% (17–52%) | 28% (14–48%) | inconclusive |
Chain data
26 decisions; 0 chose none.
| Host | Picks | Share of picks | Picked when shown |
|---|---|---|---|
| agent402.tools | 17 | 65% | 65% (46–81%) |
| chain.tollkit.dev | 7 | 27% | 78% (45–94%) |
| api.marketintelligenceapi.com | 2 | 8% | 22% (6–55%) |
| 402.com.tr | 0 | 0% | 0% (0–32%) |
| agent-data-tools.revmesh2074.workers.dev | 0 | 0% | 0% (0–12%) |
| agent-data.centorprat.workers.dev | 0 | 0% | 0% (0–30%) |
| chain.intel.rallylive.ca | 0 | 0% | 0% (0–18%) |
| onchaincheck.dev | 0 | 0% | 0% (0–32%) |
| webtools402.176-109-107-191.sslip.io | 0 | 0% | 0% (0–30%) |
| x402-gateway.blackwoodstate8.workers.dev | 0 | 0% | 0% (0–18%) |
Position
| Slot | Picked | If position didn't matter |
|---|---|---|
| 1 | 31% (16–50%) | 17% |
| 2 | 12% (4–29%) | 17% |
| 3 | 0% (0–13%) | 17% |
| 4 | 12% (4–29%) | 17% |
| 5 | 15% (6–34%) | 17% |
| 6 | 31% (16–50%) | 17% |
Price
Not estimated: needs at least 30 decisions with prices; there were 26.
Proof fields
| Field added | Pairs | With | Without | Verdict |
|---|---|---|---|---|
| settlement | 9 | 100% (70–100%) | 100% (70–100%) | inconclusive |
| capture | 9 | 33% (12–65%) | 33% (12–65%) | inconclusive |
| receipts | 8 | 0% (0–32%) | 0% (0–32%) | inconclusive |
| any | 26 | 46% (29–65%) | 46% (29–65%) | inconclusive |
Panel
Panelist B (hosted), Panelist C (hosted), Panelist D (hosted)
Method
Tasks. The first 3 buyer-agent task phrasings per category from Agoranome's gap finder, the same every week.
Offers. The top 6 Bazaar search results per task, deduplicated, shown as listed; descriptions cut to 500 characters.
Presentation. Headless: each offer as JSON (price, networks, method, discovery metadata, 30-day usage), the way an x402 client reads it, with the task and a $1.00 spend cap.
Layout. One seeded shuffle per task, rotated one slot per trial so every offer visits every position; personas rotate; every panelist and arm sees identical layouts.
Proof arm. In each trial one offer, drawn at random, gets one Fair Offer field added by the experimenter (it is not that seller's claim), rotating through settlement, pay on delivery (the same settlement enforced by an auth-capture escrow), receipts, public receipts, sample and pledge. Lift is its pick rate against the same trial without the field.
Statistics. Shares carry Wilson 95% intervals. Price elasticity comes from a conditional logit over log price, first position, log 30-day payers and output example. Proof lift counts only when Newcombe's interval for the difference excludes zero. The shock index is the total variation distance between this report's picks and the previous report's, in points; 25 or more is flagged.
Neutrality. Agoranome's own listings are run the same way when search returns them, and shown losing when they lose. No seller pays to be included or ranked.
Budget. Model spend capped at $0.50 per report.
Licence. Reports and raw trials are published under CC BY 4.0.
Data sharing. At least one hosted provider on the panel may use the offers and tasks its panelists see to train or improve its models.
Download this report (JSON) · Every raw trial (JSON) · Every trial of every report (about the dataset) · Licensed CC BY 4.0
Related: the Machine Buyer Safety report measures how buyer clients fare against the attacks a paying agent meets.
Corrections, data questions and replication requests: research@agoranome.com
Every report
- #1 2026-10-27: In web search, web.tollkit.dev took 37% of 27 picks. In chain data, agent402.tools took 65% of 26 picks; the first slot gained 14 points.