Coinbase has published results showing that its fine-tuned version of Alibaba’s open-weight Qwen3.5-9B outperformed Anthropic’s Claude Opus 4.5 on fraud-detection tests for Coinbase Onramp, the service that lets users purchase crypto through third-party applications.
The benchmark covered 16,140 historical transactions across 7,293 users, including 813 confirmed fraudulent transactions gathered over nine weeks. Coinbase’s customized model posted higher scores across all four detection metrics: precision improved by 7.5 percentage points, recall by 12.0 pp, F1 score by 9.6 pp, and dollar-weighted recall by 35.4 percentage points — the largest reported gain, measuring the share of fraudulent transaction value detected. Median AI request latency fell from 1.515 seconds for Opus 4.5 to 0.683 seconds for Qwen, a 55% reduction, while P99 tail latency fell from 3.312 seconds to 1.036 seconds.
Instead of building a new foundation model, Coinbase adapted Qwen3.5-9B through reinforcement learning with verifiable rewards, using historical fraud labels. The training stack included SkyRL for reinforcement-learning management, GRPO for response comparisons, and LoRA to reduce parameter updates. The workload ran on a single NVIDIA RTX PRO 6000 GPU with 96GB of memory, with reported compute cost below $100, excluding data collection and infrastructure expenses. Deployment used NVIDIA L40S GPUs with autoscaling between two and eight replicas.
Coinbase’s October 7 analysis compared newer versions of Claude Opus, Claude Sonnet, and OpenAI GPT models on the same fraud benchmark, finding that newer releases did not always perform better. For example, GPT-5.6 improved precision but reduced recall and dollar-weighted recall, meaning it became more selective while missing a larger share of fraudulent activity — an unfavorable trade-off for financial risk management.
The work builds on an earlier AI-powered risk agent integrated into payment infrastructure, detailed on October 6. In an online A/B experiment, the agent-assisted system recorded 30% fewer fraudulent transactions and 22% less fraudulent transaction value compared with existing controls alone. The agent reviews selected transactions after conventional machine-learning models and rules perform initial checks, identifies suspicious patterns across recent activity, and can block a transaction that passed upstream checks but cannot override an existing block.
Brian Armstrong said the point was not that smaller models are universally better, but that specialized models can outperform generalists on narrow tasks. Coinbase’s engineering team emphasized ongoing evaluation, model versioning, and rollback procedures as fraud patterns change.
