Pydantic AI Updates: October 29, 2026

1. Pydantic Benchmark Finds Code Mode Cuts Cost 58% on Long Agent Tasks

Pydantic AI published a benchmark by Michael Pfaffenberger comparing plain tool calling with Pydantic AI Harness’s CodeMode, where the model writes a Python program that calls tools inside the Monty sandbox instead of calling them one at a time. The test used five small Pydantic AI apps (cart, calendar, chatroom, kanban, inventory) with only single-item CRUD tools, eight models including gpt-6.1-sol, claude-sonnet-5-5 and deepseek-v4.1-flash, and 5,276 runs graded by a deterministic end-state check. On long multi-step jobs, code mode cut cost by 58% and wall time by 51% and raised success by 16 points, with failures falling from 67 of 288 runs to 21. On scaling tasks from 5 to 200 items, success without code mode dropped to 62-79% while code mode held 92-100%. The author found the savings come from smaller requests rather than fewer round trips, since models already batch tool calls. Small jobs carry a cost: at 5 items code mode was 21% to 38% more expensive on three of five tasks, and on everyday tasks it was 16% cheaper overall with no meaningful success gain. Monty runs each snippet with no host access by default, a 30-second execution limit and a 256 MiB heap cap. Source