Serverless Cold Starts Explained: MicroVMs, Snapshots and What You Can Control

Serverless promised “just run some code”, and for the first call after a quiet period it keeps that promise slowly. Understanding where the time goes tells you which part you can fix and which part belongs to the platform.

Why serverless exists

The original observation was that customers kept whole fleets of virtual servers alive to do occasional small jobs: write a record, process an uploaded file, react to an event. Those machines sat idle and still cost money. The product answer was a service that allocates compute on demand, runs a function, bills for the time used and hides servers entirely.

At launch that meant a handler function per event, automatic scaling per request, stateless execution (anything that must persist goes to storage), and billing that started with 100 ms-class rounding and moved to 1 ms granularity. Memory is the only size knob, with CPU scaling proportionally.

What happens on a cold start

1. Provision an environment

   The platform finds capacity and sets up an isolated sandbox for your function.

2. Download the code

   Your package or container image is fetched. Bigger artefacts take longer.

3. Start the runtime

   The language runtime boots. A JVM with class loading takes much longer than a small Go or Node process.

4. Run your initialisation

   Code outside the handler runs: imports, SDK clients, configuration, database connections, model loads.

5. Invoke the handler

   Only now does your request start executing.

A warm invocation reuses an existing environment, skipping steps one to four. That explains why latency is bimodal: most calls are fast and the first after idle (or during a scale-out burst) is slow.

Platform work, bottom up

Lightweight microVMs

An early design dedicated a full virtual machine instance per tenant. It was secure and wasteful. Firecracker, a minimal virtual machine monitor built on the Linux kernel’s virtualization support, creates microVMs that boot in a fraction of a second with a tiny memory footprint, yet keep hardware-level isolation between customers. It lets many isolated sandboxes share a host efficiently, which cuts both start time and cost.

Snapshot restore

For runtimes with heavy initialisation, SnapStart moves that work to publish time: initialise the function once, snapshot the memory and disk state, and restore new environments from the snapshot. Reported startup reductions run up to 90 percent for Java.

Layers and containers

Layers let functions share libraries and binaries without repackaging them in each deployment, so packages stay smaller. Container image support (up to several gigabytes) fits large dependencies such as machine learning inference, with the trade-off that large images take longer to fetch cold.

What you control

Lever	Effect on cold start	Notes
Smaller package	Faster download	Tree-shake, drop unused dependencies, prune dev files
Fast-starting runtime	Faster runtime boot	Go, Rust, Node and Python usually beat a plain JVM unless snapshotted
Lazy initialisation	Moves work off the critical path	Initialise a client on first use, not at import
Initialise outside the handler	Warm invocations reuse it	Connection clients, config, compiled regexes
More memory	More CPU, faster init	Often cheaper overall despite the higher rate per ms
Snapshot restore	Skips initialisation	Check restore hooks and uniqueness
Provisioned concurrency	Eliminates cold starts for reserved capacity	Pays for idle readiness. Use for predictable, latency-critical paths
Warmers (scheduled pings)	Keeps a few environments alive	Fragile. Does not help during scale-out bursts

import boto3

_client = None                     # created once per environment, reused when warm

def get_client():
    global _client
    if _client is None:
        _client = boto3.client("dynamodb")
    return _client

def handler(event, context):
    return get_client().get_item(...)

When serverless is the wrong shape

A steady, high-throughput service with strict single-digit-millisecond latency is often cheaper and simpler as a long-running container. Serverless shines for spiky, event-driven and low-duty-cycle workloads, and wherever the operations overhead of owning servers is the larger cost.

Takeaway

A cold start is five steps of setup you only pay for on a fresh environment. Platforms shrink the first two with microVMs and the middle ones with snapshots, and you shrink the rest with small packages, lazy and well-placed initialisation, adequate memory and, where justified, reserved warm capacity. Measure first, then spend.