Better Models: Worse Tools

Newer versions of Anthropic’s Claude models, including Opus 4.8 and Sonnet 5, are struggling to correctly use custom coding tools in third-party applications like Pi. While these models often perform the intended code edits correctly, they frequently include unauthorized or invented fields within the tool’s data structure. Because these arguments do not match the required schema, the software rejects the requests, forcing the model to retry the task.

This issue is unexpected because older versions of Claude did not exhibit this behavior. The problem appears to stem from the way newer models are trained. Anthropic has likely optimized these models through reinforcement learning to excel at the specific search-and-replace tools built into their own Claude Code environment. This specialized training seems to interfere with the models' ability to adapt to the different, custom edit tools used by external coding platforms.

This trend creates a significant challenge for developers of coding agents. If state-of-the-art models are increasingly fine-tuned for specific proprietary tools, third-party platforms may be forced to implement multiple, redundant tool interfaces just to ensure compatibility. This shift suggests that as AI models become more powerful, they may also become less flexible when interacting with diverse, non-standardized software environments.