Orchestration

Many machines from one site, without KVM, root or Docker: snowglobe daemon restores sites once as templates and forks machines from them in milliseconds, keeps snapshots of running ones, puts services behind ports that scale to zero, holds them all to a memory budget, and applies whole fleets from a file.

In a browser: snowglobe ui

snowglobe ui shows the daemon in your browser: the last hour on one time axis (memory against the budget, templates running, jobs and their latency, what happened), a lane per template with each machine's life as a bar, and under them whatever you select, with its terminal one click away. It also runs commands on fresh forks, builds sites from a directory, an image or a Dockerfile you write there, and manages templates, machines, services, snapshots and ports.

snowglobe ui                                     # opens http://127.0.0.1:7777/?token=…
snowglobe ui --start-daemon                      # starts a detached daemon first if none runs

It listens on 127.0.0.1 only. The link it prints carries a session token that becomes a cookie, and every request needs it, so other pages you visit can't reach the daemon through it. The terminal is ghostty-web, Ghostty's terminal compiled to WebAssembly, joined to the machine's console over a WebSocket.

Templates and machines: the daemon

snowglobe daemon keeps templates (a site restored once) and the machines forked from them. snowglobe up forks a machine from a template in tens of milliseconds. The machine shares the template's memory, lasts until it is stopped, keeps what it writes, and has its ports forwarded by the daemon (its site needs a network for that). sh, cp, exec, attach, ps and stop reach the daemon's machines as they reach instances.

snowglobe daemon --detach                        # in the background, its log in daemon.log beside the socket
snowglobe template add dist/api --name api       # the python example built with --network fetch
snowglobe up api --name web -p 8080:8000 --label app=web
snowglobe up api --name worker                   # another machine, its own state
snowglobe sh web 'python3 -m http.server 8000 &'
snowglobe ps
snowglobe stop worker
snowglobe template rm api                        # and its machines

The API is HTTP/JSON on a Unix socket in the user cache (curl --unix-socket works): /v1/templates, /v1/machines, /v1/machines/{name}/jobs and /v1/templates/{name}/jobs (a job on a fresh fork). --listen ADDR --token-file F also serves it over TCP, to requests bearing the token, and SNOWGLOBE_DAEMON=http://HOST:PORT with SNOWGLOBE_DAEMON_TOKEN points the CLI at it. A daemon started again takes back the templates and machines that are still running.

It also builds: POST /v1/builds takes a directory with a Dockerfile (by its full path), an image reference, or a Dockerfile's text, runs snowglobe build (Docker when it's there; an image builds natively without it), and makes the result a template. /v1/builds/{id}/log streams its output (?follow=1). A machine's terminal is a WebSocket at /v1/machines/{name}/attach?rows=R&cols=C. /v1/metrics keeps a sample every 5 s for the last hour (memory, machines, replicas, jobs and their latency), and /metrics serves the same in Prometheus's format.

Fire and forget: submit and runs

snowglobe submit runs a command once on a fresh fork of a template and returns at once with the run's id. The fork is thrown away when the command ends; its output and exit status are kept for snowglobe runs. --wait waits for the end and exits with the command's status. The API is POST /v1/runs and GET /v1/runs/{id}.

id=$(snowglobe submit py 'python3 report.py' --env DAY=monday --timeout 600)
snowglobe runs                                   # every run, newest first
snowglobe runs $id --wait                        # its output; exits with its status

Snapshots of running machines

snowglobe snapshot save saves a machine whole: its memory, its running processes and the files it wrote. Machines start from it again with up --snapshot, or it becomes a template of its own with template add --snapshot. Set a machine up once (dependencies installed, a server started, data loaded), then fork as many as you want, already set up.

snowglobe up api --name base
snowglobe sh base 'pip install httpx && python3 -m http.server 8000 &'
snowglobe snapshot save base --name ready        # tens of milliseconds
snowglobe template add --snapshot ready --name api-ready
snowglobe up api-ready -p 8000                    # installed, its server already listening
snowglobe snapshot ls
snowglobe snapshot rm ready

Services and scale to zero

A service puts a template's machines behind host ports that the daemon holds. Each connection (or snowglobe sh <service>) goes to one of its replicas, started on demand. With --min 0 it scales to zero: replicas with no connection or job for --idle stop, and the next request forks one again. Made of a snapshot whose server was already listening, a service at zero answers its first request in about 20 ms. --state keep saves the replica before it stops, and the next one carries on from that save (one replica); --state discard, the default, starts every replica from the template.

snowglobe service add web --template api-ready -p 8080:8000 --min 0 --max 3 --idle 5m --concurrency 10
snowglobe service add db --template pg --state keep --idle 10m
snowglobe service ls                              # replicas running now, of each
snowglobe service rm web

A replica takes --concurrency connections before another is started, up to --max; under --min, replicas that stop are replaced. The API is /v1/services and /v1/services/{name}/jobs.

Ports

snowglobe ports lists every host port the daemon holds and where it leads: the machine or service, the guest port and whether it is open now (a service's stays open at zero). snowglobe ports check says whether a host port is free, or which machine, service or other program holds it, and exits 1 when it is taken. The UI shows the same, and checks the ports you type as you type them. The API is /v1/ports and /v1/ports/check?port=N&ip=IP.

snowglobe ports
HOST             GUEST  KIND     NAME  TEMPLATE  STATE
127.0.0.1:18081  8000   machine  web   curl      open
127.0.0.1:18090  8000   service  api   curl      at zero
snowglobe ports check 8080                       # 127.0.0.1:8080 is free

Limits and lifecycle

snowglobe daemon --memory 8G caps what its template servers and machines take together. A machine that doesn't fit waits up to --wait (30 s) for room, then is refused with the reason. Linux measures each process's proportional share; macOS its resident memory, which counts pages shared with the template in full, so the cap is reached sooner. No cap unless asked. up --ttl 1h stops a machine after that long, whatever it is doing. template add --idle 30m stops a template's server after that long without a machine; the next up (or job) restores it first, in about a second.

snowglobe daemon --memory 8G --wait 30s &
snowglobe template add dist/python --name py --idle 30m
snowglobe up py --ttl 1h

Events and top

snowglobe top shows the daemon live: memory against its budget, templates (awake or asleep), services with their replicas and open connections, machines with what each takes and when it stops, and the latest events. snowglobe events streams those events (machines up and stopped, replicas started and stopped, templates asleep and awake, snapshots, admissions refused), the last 200 first; --json for programs. The API is /v1/events (server-sent events, ?after=SEQ to resume) and /v1/stats.

snowglobe top
snowglobe events --json | jq 'select(.kind == "admission.refused")'

Fleets: snowglobe apply

A snowglobe.toml says what the daemon should have, and snowglobe apply gets it there: templates (a built site, a URL, or a Dockerfile directory built first), snapshots (a machine of a template set up by commands, saved once a readiness command succeeds), templates of those snapshots, services and machines. snowglobe diff says what apply would do; snowglobe down removes all of it. An entry that changed is replaced, with what is made from it; one gone from the file is removed; names something else uses are refused. One file goes from source to running services (examples/fleet/snowglobe.toml):

[templates.py]
build = "../python"                # built first, for running natively
network = "fetch"

[snapshots.serving]
template = "py"
run = ["cd /srv && python3 -m http.server 8000 >/dev/null 2>&1 &"]
ready = "wget -q -O /dev/null http://127.0.0.1:8000/"

[templates.serving]
snapshot = "serving"

[services.web]
template = "serving"
ports = ["8080:8000"]
min = 0                            # scaled to zero, woken by a request
idle = "5m"

[machines.scratch]
template = "py"
ttl = "24h"