Fine at 15 tools, lost at 300

ComplexMCP is an eval for the last mile of tool use. Not ten isolated APIs. More than 300 tools across seven stateful sandboxes, with dependencies and seed-controlled failures, on the same protocol production agents already speak: MCP. Frontier models stay under 60% success. Humans clear 90%+.

The authors name three failure modes worth stealing for your own traces. Retrieval saturation: as the action space grows, the agent cannot pick the right tool. Over-confidence: it skips checking environment state and acts on a guess. Strategic defeatism: it treats a recoverable API error as the end of the job and writes a rationale for giving up.

That is why a demo with a dozen tools does not predict a production mesh. Keep the tool surface small, partition servers by job phase, and put deterministic checks in front of irreversible calls. The model is not the whole agent. The sandbox is.