Why AI Infrastructure must evolve for Agent Experience — Akshat Bubna, Modal CTO

Modal is a cloud platform designed specifically for the unique demands of AI workloads, moving beyond the limitations of traditional infrastructure like Kubernetes. While standard cloud stacks were built for human developers who could manually manage dashboards and YAML files, the rise of AI agents requires a more automated, programmatic approach. Modal provides an environment where agents can write code, execute it, inspect outputs, and iterate rapidly within isolated, self-provisioning sandboxes.

The platform focuses on what it calls agent experience, providing the infrastructure necessary for AI to operate autonomously. This includes elastic inference for custom models, GPU bursting, and the ability to manage massive, concurrent workloads—such as running 100,000 sandboxes for reinforcement learning rollouts. By co-locating infrastructure requirements directly with the code using decorators, Modal allows agents to handle their own environment needs without requiring human intervention.

Modal’s architecture is built to handle the bursty, compute-heavy nature of AI, offering features like GPU snapshotting, speculative decoding, and a multi-cloud capacity pool across 17 providers. This shift toward agent-native infrastructure is critical because modern AI applications require tight feedback loops and specialized environments that traditional web-focused cloud providers were not designed to support. As agents increasingly take over the development lifecycle, Modal aims to provide the observability and hard guardrails necessary to ensure these systems remain reliable and efficient in production.