Firecracker VMM Boot Paths That Give Each Agent a Real Kernel at Scheduler Depth

Firecracker boots each agent with its own isolated kernel instead of sharing the host's.

· 8 min read
Cover illustration for “Firecracker VMM Boot Paths That Give Each Agent a Real Kernel at Scheduler Depth”
Sandbox & Runtime · October 29, 2026 · 8 min read · 1,910 words

With Firecracker micro-VM isolation for AI agents, each agent has its own Linux kernel, while agents in a container share the host kernel. Its firmware-free boot paths set up that separate guest environment, and snapshot-restore lets agents be scheduled without keeping every kernel running.

What firmware does on a conventional VM boot and why Firecracker refuses to do it

A conventional x86 VM starts at a reset vector in 16-bit real mode, runs firmware to initialize the chipset and probe buses, and invokes a bootloader to locate and load the kernel. The kernel’s early setup then takes the CPU through protected mode into long mode. Useful agent work is still waiting. These stages negotiate between components that initially lack information, because firmware does not know which kernel will run and the kernel does not know its machine.

Inside a Firecracker microVM, those unknowns disappear. The VMM has opened the kernel file, set aside guest memory, and set up a fixed, minimal device model. It therefore needs no BIOS or UEFI, option ROMs, PCI enumeration, ACPI discovery, or bootloader configuration to explain the environment. The remaining devices are virtio-net, virtio-block, virtio-vsock, and a serial console. Because the VMM already knows what the kernel needs to learn, it can supply that description directly and begin guest execution in kernel code.

The bzImage zero-page path and how the VMM plays bootloader

The Linux/x86 64-bit boot protocol sets out a contract that a bootloader has traditionally met. Firecracker fulfills that contract itself. It reads the setup header near the front of the bzImage to identify the protected-mode kernel body, its load address, and the applicable protocol version and flags. It then copies the kernel body into guest RAM and prepares the information that the kernel expects at entry.

That information lives in boot_params, informally called the zero page. Its e820 memory map describes the RAM the VMM allocated, replacing memory discovery with an explicit description. A command-line pointer points to the string passed through Firecracker’s boot-source API, and the initrd fields give the archive’s location and size if there is one. Other fields keep setup-header information, including the protocol version and load flags. Nothing needs to read a bootloader configuration file.

To complete the handoff, the VMM puts a pointer to boot_params in the register expected by the kernel’s 64-bit entry point. It configures page tables, places the virtual CPU in long mode, and jumps to that entry. The kernel bypasses its earlier mode-switching setup entirely. For the agent developer, the interface is a boot-source API call supplying a kernel image and command line. This works well, but its setup header and zero page retain the structure of an older bootloader contract.

The PVH path and its explicit ELF entry point

Firecracker loads a vmlinux ELF image rather than a compressed bzImage and reads the XEN_ELFNOTE_PHYS32_ENTRY note to locate the PVH entry point. The note removes a source of inference. Instead of interpreting a bzImage setup header and its offsets, the VMM follows a declaration supplied by the kernel.

The handoff uses hvm_start_info, a smaller structure than boot_params. Firecracker populates it, passes a pointer, and begins execution at the named entry point with the CPU already in protected mode. The kernel does a little early setup before it reaches its C entry. This differs from the bzImage path’s direct entry into 64-bit long mode, but neither path starts with firmware.

The Xen lineage explains the naming. The official Xen PVH ABI document identifies the ELF note as the declaration that a kernel supports this boot contract. A VMM able to construct the handoff structure and enter at the declared address can use it. PVH is therefore the tidier interface in terms of entry-point discovery and handoff size. Both paths start with real guest kernel code and a VMM-supplied description of its environment.

How can each AI agent be given its own isolated kernel without containers?

The direct-boot handoff places an independent Linux kernel inside each agent’s microVM. That kernel has its own scheduler, memory subsystem, and syscall handling. Isolation reaches below the agent process into the operating system that executes it. This is the architectural consequence of direct kernel boot, while the hardware boundary comes from KVM guest execution.

Containers provide a different boundary because their processes call into the same host kernel. A shared-kernel vulnerability that can be exploited can therefore put multiple tenants at risk through the same syscall implementation. Separate microVMs do not share that guest-kernel instance. Compromising one guest kernel does not, by itself, compromise another guest’s kernel or cross the hardware boundary to the host. This distinction is stronger and more precise than treating any VM as an unconditional guarantee against escape.

“Scheduler depth” names where that separation reaches. The guest kernel executes at KVM guest ring 0, with host intervention at VMEXITs for events including device accesses, interrupts, and faults. Each agent’s guest scheduler is separate from those of other agents, rather than merely scheduling processes isolated by container controls. That boundary changes the threat model for agents that run arbitrary, LLM-generated code, hold credentials, and keep long-lived sessions. Prompt injection, resource abuse, and lateral movement are still risks, but untrusted code no longer enters a kernel that every other agent shares.

The rootfs decision that completes the boot path

A running kernel needs userland to become a serving agent, and an initial RAM filesystem participates in that startup if supplied through the initrd_path setting in the boot-source API. With an initrd, the kernel unpacks a cpio archive into a tmpfs in guest RAM and executes a program from it. An initrd-only rootfs is fast to reach but ephemeral by default.

Without an initrd, the kernel must probe virtio-blk, find the device named by the root= command-line argument, mount its filesystem, and execute userland from there. A hybrid can use a small initrd for early setup before pivoting onto a block device. The kernel entry protocol describes how execution begins, while the rootfs arrangement determines where the agent works and what can persist.

For agents, persistence includes installed packages, credentials, working files, and browser session data. The virtio-block path keeps that durable state without having to keep the whole filesystem in RAM. It also supports copy-on-write storage arrangements in which a read-only base rootfs is shared and each agent receives a private overlay for writes. Agents using the same image can consequently share most of their disk pages.

Cold-boot latency versus snapshot-restore latency

Even without firmware, it takes time to get to a serving agent. Firecracker process startup accounts for a small fraction of the measured launch, while API configuration and InstanceStart each take tens of milliseconds. Guest boot to a serving agent occupies the dominant span, on the order of hundreds of milliseconds on KVM hardware. Kernel entry is not application readiness. A launch measurement must distinguish them before assigning responsibility for delay.

The tinylabscom/mvm issue #2299, opened August 10, 2026, reports this dominant guest-startup span on real KVM hardware. The issue measures completion when the agent answers on vsock, so userspace may account for part of the delay. The issue also reports that an alternative KVM-based VMM is substantially slower on the same hardware, suggesting the problem extends beyond Firecracker while leaving the largest contributor uncertain.

A microVM boots once, reaches a ready state, and records its memory, CPU registers, and device state. Subsequent launches memory-map the snapshot and restore execution state. ForgeVM’s snapshot implementation demonstrates restores on the order of ~28ms, including process startup, memory mapping, CPU and device restoration, and vsock reconnection. Direct boot governs the initial kernel handoff and guest environment, while restoration governs the steady-state startup latency experienced by agents launched from prepared snapshots.

Diagram: Cold Boot vs. Snapshot-Restore: Where the Time Goes. Visualizes: Show a magnitude comparison of two agent launch paths on KVM hardware.

How snapshot-restore makes per-agent kernel isolation economically viable at scale

A real kernel for each agent sounds costly if every kernel has to keep running. Snapshot scheduling removes that condition. Most agents are idle most of the time, so an idle agent can become a stored snapshot, releasing its compute allocation. The file incurs storage cost until execution is needed again. This shifts the economic unit from the number of agents that exist to the number active concurrently.

When a user message arrives, the scheduler restores the agent in tens of milliseconds. With its execution snapshot and persistent disk state, the agent can resume with files, credentials, installed packages, and browser session data available. When its turn ends or it pauses for a human, another snapshot allows compute to be released. Human-in-the-loop pauses make this especially consequential because the wait can last seconds, hours, or longer. Keeping a kernel running the whole time would mean paying for compute while waiting for a human to respond.

The resulting cost difference can be an order of magnitude, with the gap growing as fleet size increases. Copy-on-write sharing also cuts the resources that active instances need. Agents derived from the same base snapshot can share most memory pages, while writes create private divergent pages. Per-agent isolation becomes viable through selective residency.

What Vercel Sandbox reveals about turning boot decisions into a developer product

Vercel Sandbox, released January 30, 2026, exposes the dedicated-kernel model as a developer primitive. Each sandbox runs in its own Firecracker microVM, and its filesystem is snapshotted when the sandbox stops. Billing applies only to active CPU time, excluding time spent waiting for I/O. The product boundary hides the boot protocol. Developers receive an isolated execution environment without constructing zero pages or interpreting ELF notes themselves.

These choices connect isolation, persistence, and billing, but they are distinct guarantees. A filesystem snapshot preserves files, while a full execution snapshot also includes memory, registers, and device state. Persistent packages, credentials, and browser data require the durable storage role described by the virtio-block path. The billing policy alone, though, does not say how every internal scheduling transition works.

The developer-facing tooling exposes a separate maturity question. The add-mcp tool, invoked with npx -y add-mcp -g, detects installed AI clients and configures Vercel MCP automatically. Vercel’s official MCP server launched in August 2025 as read-only, while its first state-changing tool, deploy_to_vercel, shipped on July 23, 2026, eleven and a half months later. Infrastructure and tooling advance on different schedules.

Open challenges in cold boot, VMM competition, and execution continuity

Engineers still have work to do to cut cold-boot latency. The mvm issue identifies the serving-agent wait as the largest measured span without establishing a single culprit. Purpose-built VMM competition makes that decomposition important, because comparisons must distinguish process launch, kernel entry, and application readiness. The slower alternative measured on the same KVM hardware also shows why replacing the VMM is not automatically a remedy. The relevant comparison is the complete path to a usable agent.

Execution continuity is a deeper challenge than speed. The August 2026 research identified as arXiv:2608.29381 argues that “a faithfully restored checkpoint may resume an execution whose states, assumptions, and external effects never coexisted in any valid history.” Even when memory and registers are restored correctly, the resumed agent logic may be out of step with the world outside the snapshot. OS mechanisms alone do not resolve that distinction.

These challenges identify separate work on cold-start attribution, runtime comparison, and the validity of resumed execution. Firecracker’s direct-boot paths and snapshot model provide a strong foundation for that work, rather than a reason to retreat to shared-kernel isolation. Understanding the boot handoff makes it possible to locate each remaining problem at the layer where it can actually be addressed.

More in Sandbox & Runtime