Google unveils Agent Compute Engine to cut idle AI costs

Google Cloud announced Agent Compute Engine, a new infrastructure service that lets enterprises host AI agents in isolated microVMs. The service suspends agents when they are idle, keeps their execution state, and restores them on demand, a move intended to cut the idle‑time compute costs that can slow AI adoption.

How the service works

MicroVMs give the service the speed of containers but the isolation of full virtual machines. Each sandbox runs its own kernel and is protected by Titan security chips that verify appliance firmware and guard against side‑channel attacks. The checkpoints capture memory and CPU state so an agent can resume exactly where it left off, avoiding the latency of rebuilding its environment.

The benefits are most pronounced for long‑running agents that hold state for hours or days, analysts Amit Kumar Jena of Kanerika and Manoj Chandra Jha of Nord‑IQ Research said that being able to pause an agent without engineering a save‑and‑resume layer reduces operational overhead and can lower spending on idle resources.

However, the service brings trade‑offs. Because it is tightly coupled to Google Cloud, moving workloads elsewhere could be difficult, and storage for the microVM snapshots could raise bills at scale, Jena warned. Developers also warned that checkpointing can produce bugs when an agent is paused between an API call and its confirmation, a risk Jena highlighted, and that long‑running sandboxes can drift from their original configuration, a concern noted by Jena. Jha said the savings depend on how many agents spend more than a few minutes idle.

Google plans to roll out Agent Compute Engine in the coming months alongside a new version of Agent Gateway, which will add model‑routing and token‑management features to help customers control inference costs. The service is not yet in preview, so enterprises will need to evaluate the roadmap and potential savings before deploying.