Design notes

For contributors: why devvm is built the way it is, what was tried and dropped, and what’s still open. How each part works is on its own page; the layout and rules are in AGENTS.md.

Emulated features

When devvm builds a backend feature from other parts of the backend, the feature is emulated, and it gets a note here before its review. The note says what the feature is built from, the guarantee it keeps, what could break it, and which end-to-end checks prove it. The threat check comes next, and only then is the feature marked reviewed. Every feature Lima has is native except its ports, which devvm’s forwarder emulates (below).

ssh into VMs

ssh for editors is native on Lima, so it isn’t an emulation, but its sessions keep a guarantee of devvm’s own: ssh and editors reach a VM by name through the entries devvm writes (security model), and a session gives the VM no way out it doesn’t already have, on a confined tier nothing past its proxy, whatever the VM runs as sshd.

What keeps a session from carrying the VM out. An ssh session can carry the VM’s traffic only through channels the host’s ssh asks for: remote forwards (-R, RemoteForward, a remote SOCKS forward), X11, a tunnel and the agent; OpenSSH’s client refuses a channel the server opens unasked. Root in the VM runs whatever sshd it likes, so the guarantee is the host’s ssh’s: devvm’s entry (first in ~/.ssh/config, so its options win over a Host *) sets ForwardX11 no, ForwardX11Trusted no, Tunnel no, PermitRemoteOpen none (so devvm needs OpenSSH 8.7 or newer; no IgnoreUnknown for older ones, as its first value wins and would replace a user’s Host * one, such as macOS’s for UseKeychain), ForwardAgent as the agent choice the VM’s current start was made with says, AddKeysToAgent no (no key of the host’s lands in an agent the session forwards), ProxyJump none and ProxyCommand none (no hop through something else), UpdateHostKeys no (the VM can’t add host keys; devvm reads them itself) and HashKnownHosts no. A RemoteForward can’t be turned off without refusing ssh -R on the command line too, so it isn’t (Not covered). VS Code’s Remote-SSH uses local and dynamic forwards (-L, -D) and is unaffected. Then sshd’s own settings, a defence for an honest VM (assets/layers/base.sh, /etc/ssh/sshd_config.d/00-devvm.conf, first so it wins; Lima’s own forwards are local; it applies only where the image’s sshd_config includes sshd_config.d, as Ubuntu’s and Debian’s do, so for some --from images not at all, which only an honest VM’s defence loses):

Local forwarding (-L, -D, -W) is allowed: the VM itself connects, from its own network, which its tier confines like any other connection of the VM’s (checked). VS Code’s Remote-SSH reaches its server this way.
Remote forwards bind only 127.77.0.0/16, the range devvm keeps for links between VMs, and remote socket forwards are refused. A remote forward makes the host connect on the VM’s behalf, so one the host’s ssh asked for by mistake could reach anything the host reaches, its loopback services and sockets (docker.sock, agents) included, around the VM’s confinement. GatewayPorts clientspecified binds the address a request names, and PermitListen 127.77.*:* matches that address as text, so a request for localhost (ssh’s default), 127.0.0.1, 0.0.0.0 or ::1 is refused. Only devvm on the host asks for a forward in that range, always by number. X11 and tunnels are refused too.
Agent forwarding is allowed: the entry decides it from the SSH agent choice the VM’s current start was made with (ForwardAgent, devvm’s own agent too); refusing it would break devvm ssh-agent on in editors.

These settings guard against the host’s ssh asking for a forward that devvm’s entry doesn’t pin (a RemoteForward under Host *, ssh -R), and only while root in the VM leaves sshd as it is. They’re a safeguard for an honest VM, and the boundary is the host’s ssh, which accepts a channel only for a forward it asked for.

What it relies on: OpenSSH’s client refusing unrequested channels, and devvm’s Include line first in ~/.ssh/config, so its entries’ options win.

What could break it: host ssh settings that come before devvm’s entries.

Not covered:

A remote forward you ask for (ssh -R, a RemoteForward of yours): remote forwards you add lists what it leaves open.
Sessions see the proxy’s password, as devvm’s own shells do, and VS Code relays what VS Code describes.

Ports on Lima

The guarantee: when a VM forwards every port, each TCP port from 1024 up that a server in it listens on, on its loopback or on all its addresses, is forwarded to the same port on the host’s loopback, and only to its loopback. When a VM lists its ports, only those are forwarded, however many it lists, by the same rules. The forwarder never takes a port that a program on the host has on any address when the forward is made, or a port devvm records for another VM. Forwarding gives the VM no way out that it doesn’t already have, because every connection starts on the host. Lima’s own forwards listen with SO_REUSEADDR, so on macOS, whenever a server in the VM starts, Lima takes 127.0.0.1:PORT while a program on the host has *:PORT, and that program’s loopback connections go with it. That’s why Lima forwards nothing and devvm’s forwarder forwards the VM’s ports instead. On macOS, a confined VM doesn’t depend on Lima for that, because its hostagent’s sandbox refuses it every port but its ssh port, so Lima can’t forward a port of the VM’s even if it ignored its rules. The emulation is reviewed, with the threat check’s findings fixed or listed below, which is why the host key read is capped, reports are paced and the log is limited, and the end-to-end checks prove the rest.

What it’s built from:

Lima’s forwards off. Each VM’s portForwards holds three ignore rules, each for every guest port and protocol (any, which is TCP and UDP). They’re on 127.0.0.1, which Lima also matches for ::1, on 0.0.0.0 and on ::, which together match servers on all addresses. They come ahead of Lima’s own last rule, which forwards everything. CreateVM sets them with limactl clone --set. Start sets them only where limactl list shows other rules, because an edit gives the VM new host keys at its next boot. Lima’s ssh is its VM driver’s forward (ssh.localPort), not a rule, so the ignore rules leave it alone, and they cover the guest’s own ssh port like any other.
The sandbox’s binds (macOS, confined tiers). The hostagent of a confined VM runs under a Seatbelt profile (confinement) that denies every network bind but two kinds. It may bind the VM’s ssh port over TCP, which devvm chooses and records in the VM’s state (the ports section’s host.ssh), and unix sockets in the VM’s Lima folder, which hold the hostagent’s own socket (ha.sock), the VM’s network (default_ep.sock, default_fd.sock) and ssh’s control socket. Seatbelt matches a bind by its port alone, on any address, even when the rule names localhost, and it can’t allow an ephemeral bind (port 0) without allowing every port. It matches a unix socket by its path with links resolved, so a link in the Lima folder doesn’t let the hostagent bind a socket elsewhere. Lima’s host resolver binds two ephemeral ports for its DNS server, so a confined VM starts with the resolver off and 127.0.0.1 as its DNS server, as on Linux, where nothing listens and lookups fail at once. The hostagent reaches the guest agent over vsock, which needs no bind. This list comes from starting a VM under a profile that reported every bind, then under one that denied the rest: Lima logs a forward it’s refused (failed to listen) and carries on, while a refused resolver ends the start.
Which ports. Which ports each VM forwards, all, none or a list, is in its state (ports.forward), and the backend never hears of them: neither Start’s Want nor any other call of the backend’s has them. vmssh.Forwards.Start, the one owner of the forwarder, reads them from the VM’s state each time it starts one, after a start of the VM and when devvm ports changes a running VM’s ports, without editing Lima’s config. A stopped VM’s change is its state’s alone, because its next start reads it. When the VM lists ports, the forwarder forwards those less the ports a program on the host has, which the owner leaves out with a ports_taken warning. When none is left, no forwarder runs.
A forwarder per VM, on the host. The forwarder is vmssh.Forwarder, which isn’t a backend’s. Its owner, vmssh.Forwards, runs it detached as devvm _ports, one per VM, for project VMs alone, because only a project VM’s start goes through it, and builds’ starts don’t. The owner starts it once the backend’s Start returns, and again on a running VM when its ports change, each after stopping any forwarder the VM had, which ends the connections open through it. devvm finds that one by its pid file, which it checks against the process’s arguments, as it does for the proxy. It sends that forwarder SIGTERM and kills it if it hasn’t ended within 2 s, so its ports are free before devvm asks which listed ports a host program has. devvm holds a lock of the VM’s own (ports.lock, beside its pid file ports.pid in the VM’s run folder) from stopping the old forwarder until the new one has started, so two devvm runs at once, such as a start and devvm ports, leave one forwarder, the one its pid file names. The owner waits up to 10 s for the forwarder to say it forwards while it runs. The forwarder says so in a status file of its own beside its pid file (ports.status), which holds its pid and which devvm removes before each start. If it doesn’t, devvm warns with ports_forward_failed, naming its log, and leaves it to keep trying while the VM runs on.
Reaching the VM. The forwarder makes one ssh connection, a master (ssh -F none -M -S <socket>), to the VM’s sshd on Lima’s ssh port on the host’s loopback. Lima’s SSH gives the port with the user and the key, the first IdentityFile of Lima’s ssh config, and devvm refuses that target unless the config names 127.0.0.1 (vmssh.FromTarget, the check links and the ssh config use too). It logs in as Lima’s user with Lima’s key alone (IdentitiesOnly, IdentityAgent=none), _config/user in devvm’s Lima home, which every VM there authorizes, and never with the user’s agent. The master reads none of the user’s configuration and forwards nothing either way, with ClearAllForwardings, PermitRemoteOpen none, ProxyCommand none, and no agent, X11 or tunnel. The VM’s sshd allows local forwarding, and remote forwards only to 127.77.0.0/16 for links (base.sh), and -W is local forwarding. Its control socket is in ~/.local/share/devvm/run (mode 700) and is named by a hash of the VM’s name, because a socket’s path is at most 104 characters. When the connection is lost, the forwarder closes its forwards and makes the connection again every 2 s while the lifeline holds.
What listens. The master’s session runs ports.sh (assets/guest/ports.sh) as the VM’s user (sh -s). It prints the local addresses of the VM’s listening TCP sockets, which are the state 0A lines of /proc/net/tcp and tcp6, whenever they change and every 10 s regardless, so it ends soon after the connection does. The forwarder treats the report as untrusted. A line over 128 bytes or a report over 4096 lines ends the connection, and anything that isn’t a hex address and port is skipped. Reports are applied at least 500 ms apart, and one that comes sooner waits, holding up the VM’s ssh channel with it, so the VM can’t make the forwarder bind, dial and read devvm’s state at its own pace. The VM can make the forwarder write to its log, with ports coming and going and with the session’s stderr, so the log takes 200 writes at once and then one every 10 s. Each write is at most 512 bytes, without control characters but newlines and tabs, C1 included, and a start empties the log once it’s over 1 MiB. A forwarded port’s connections go to 127.0.0.1 for a server on all IPv4 addresses or on that address, and to ::1 for a server on all IPv6 addresses or on that address. A server on another 127.x address alone gets its connections there. Servers on other addresses alone aren’t forwarded.
How many. With every port, the forwarder takes ports from 1024 up, because the ports below are the host’s system services. With listed ports, it takes those alone (-only), a listed port below 1024 included, because the user asked for it, and a bind the host refuses is logged. Either way the forwarder takes at most 256 ports at once. It reads the reserved ports (below) again with each report, and while it can’t read them, it takes no new port and keeps those it has. A port it has that has since become reserved, such as one another VM’s start has chosen, is given up at that report, though the VM still listens on it. A port the forwarder couldn’t take is tried again only once the VM’s server on it goes and comes back.
Listening. The forwarder listens on 127.0.0.1 and ::1, or on IPv4 alone on a host without IPv6. It binds without SO_REUSEADDR (platform.ListenExclusive). With that option, which Go and Lima set, macOS lets a socket take 127.0.0.1:PORT while a program has *:PORT, and that program’s loopback connections go with it. Without the option, the bind is refused when another program has the port on any address. This was checked on macOS for a program on 0.0.0.0, on :: (IPv4 and IPv6 at once, as Node’s and Go’s servers listen), on IPv6 alone and on 127.0.0.1, each with and without SO_REUSEADDR of its own. A port held only by ended connections (TIME_WAIT, with nothing answering on it) is taken as usual, so a server that restarts gets its forward back. When a bind is refused, the forwarder dials the port and takes it only if the dial is refused. A full backlog times out, which counts as taken. A port the forwarder took on 127.0.0.1 but finds in use on ::1 is given up, so localhost never reaches two programs. On Linux, SO_REUSEADDR stays on. Linux refuses that bind with the option too while another socket listens on the port, and without it, the forwarder’s own ended connections (TIME_WAIT) would keep anyone from listening on the port for a minute, its next forwarder included.
Each connection. Each forwarded connection is a channel of the master’s (ssh -S <socket> -W HOST:PORT), with one process per connection and at most 256 at once. The VM’s sshd opens it to the VM’s own loopback from inside the VM. The channel’s ssh has the master’s options. If the master is gone, ssh connects on its own, and it then reaches the VM’s sshd the same way, never the host’s.
Reserved ports. The forwarder never takes the VM’s own Lima ssh port, which Lima’s SSH names, or its proxy port, which its state records, and the owner passes both as -reserve. It also never takes the ssh, proxy and desktop ports devvm chooses and records in any VM’s state, or other VMs’ listed ports (ports.Used). Every Lima VM’s ssh port is one of those, whatever its tier: devvm chooses it when it makes the VM, sets it with limactl clone --set .ssh.localPort, and records it in the VM’s ports section before the clone (ports.Allocator.SSH). One allocator, ports.Allocator, chooses every host port devvm chooses, the ssh, proxy and desktop ports, and records it while it holds the VM’s lock and then the claims lock (run/claims.lock, state.VM.Claim) until the VM’s state is written, so two devvm commands making or starting VMs at once never choose the same one. A build VM has no state, so devvm chooses its ssh port too, on every host, through the same allocator (ports.Allocator.Build), under the claims lock alone (state.Claims), and on macOS the build’s Seatbelt profile names that port. It records the port by the build VM’s name (run/build-ports/<name>), where ports.Used reads it, until limactl start returns, and by then Lima has bound the port or the start has failed. The devvm that starts the build holds a file lock on that record until then, and the kernel lets go of the lock when that devvm ends, even when it’s killed. Each time devvm chooses a port, under the claims lock, it first deletes the records whose lock no process holds and that are over 15 minutes old. The age matters because a killed devvm leaves its limactl start running, which may still bind the port for up to Lima’s 10 minutes of waiting for the VM’s ssh. So a devvm killed while its build VM starts reserves that port for 15 minutes, and then only until the next port is chosen. A stopped VM gets a new one only when a program on the host has its port on any address, even one that accepts no connections, or when the VM lists it to forward, and its start then sets it with an edit. Lima binds the port only once the VM boots, a few seconds after its start, so a port Lima picked itself would be free for another VM’s forwarder in that window, and Lima’s own ssh, which checks no host key, would then reach the other VM (the threat check below).
The host key. devvm keeps one record of a VM’s host keys for each of its starts, vms/<hash>/run/known_hosts (vmssh.PinHostKeys). It removes the record before each start and reads the keys after it, through limactl shell, at most 64 KiB for at most 30 s, keeping only lines whose key decodes and names its own type (vmssh.ParseHostKeys). The record’s first line names the start it was read for (backend.Run.ID), so a record of another start, such as one before a limactl start outside devvm, is read again, and keys read while the VM started again aren’t recorded. The forwarder checks the VM against that record, with HostKeyAlias and StrictHostKeyChecking yes, as the ssh config, links’ sessions and a --clone VM’s git do. All of them reach the VM through one function, vmssh.Checked, which checks the backend’s ssh and adds the record: the forwarder and the ssh config pin the keys when the record is another start’s, and links’ sessions and git take only a record of the current start. The pin keeps the forwarder from talking to a program on the host that takes the ssh port. It doesn’t guard against the VM itself, which is already the other end of the connection.
Its lifeline. The forwarder follows the hostagent’s pid file (ha.pid, the lifeline Lima’s Running gives), as the proxy does, and exits once the file is gone, holds anything else, or its process is gone. devvm stop stops it with the VM, and devvm rm and devvm gc --orphans stop it before they delete its files with the VM’s folder. A VM that stops some other way, such as when its self-check fails or a restart stops it, ends its forwarder through the lifeline, and the next start stops any forwarder that’s left before it starts its own.

What keeps a forward from carrying the VM out: every connection starts on the host. A program connects to the forwarder’s port, and the forwarder asks the VM’s sshd to connect to the VM’s loopback. The VM opens nothing to the host through the forwarder, because the master asks for no remote forward, agent, X11 or tunnel, and OpenSSH’s client refuses a channel that the server opens unasked, whatever sshd root in the VM runs. The only remote forwards devvm asks a VM for are a client’s links, on a link session that is a connection of its own, never the forwarder’s master (links). What the VM sends back goes to the program that connected, as with any forward. On a confined tier, the VM’s own connections stay confined by Seatbelt or QEMU’s restrict=on, while the forwarder runs on the host as devvm, outside the hostagent’s sandbox.

What it relies on: OpenSSH’s client refusing unrequested channels and honoring -F none, the kernel refusing a bind without SO_REUSEADDR where another socket has the port (checked on macOS), and devvm’s state listing every VM’s ports. It also relies on Lima binding its ssh forward on the host’s loopback alone, on the host key that devvm pins at the start, and on Lima honoring the ignore rules ahead of its own rule. On macOS, a confined VM relies on Seatbelt refusing the hostagent’s binds instead of on the rules, and on the Virtualization framework’s own process, which runs outside the profile, binding nothing: the VM’s network goes through the hostagent’s unix sockets.

What could break it: a Lima release that forwards despite the ignore rules, or that forwards from somewhere other than portForwards. On macOS, a confined VM keeps the guarantee even then, because its sandbox refuses the binds. An open VM, and every VM on Linux, where QEMU’s restrict=on doesn’t cover the hostagent’s own binds, still relies on Lima honoring the rules. A Lima release that needs a new bind for a confined VM, such as a resolver that can’t be turned off, fails its start closed, and Lima’s log names the refused bind. The end-to-end checks look at the rules, and use lsof or ss to check that the hostagent has no TCP or UDP port of the VM’s and that a host program keeps its port. A program on the host could take Lima’s ssh port while the VM is stopped, and the pinned host key refuses it. Lima’s key logs in to every VM in devvm’s Lima home, as it does for Lima’s own ssh, but public key authentication never sends the key to the VM.

Threat check (the security-audit skill, guidance mode, its local IPC companion; the sandbox’s binds and the ssh port). Leads, each confirmed and fixed, ruled out, or listed:

Another VM answering for this one’s ssh: confirmed, fixed. An ssh port that Lima picks at a VM’s start, and binds once the VM boots, is one devvm doesn’t record, so no forwarder reserves it. Another VM forwarding every port, with a server on that port in the window, has its forward take it. Lima then carries on with its other ssh forward and waits on the port (seen with a host program holding it), and Lima’s own ssh checks no host key (StrictHostKeyChecking no), so limactl shell, the host keys devvm reads and pins after the start, and devvm’s setup and shell would reach the other VM. The fix: each VM’s ssh port is devvm’s, whatever its tier, recorded before the clone and reserved by every forwarder. It’s unit-tested and checked by the CLI tests, and end to end, the port a VM’s state records is the one Lima listens on.
A guest port bound around the rules: ruled out on macOS on a confined tier. Lima forwards a port only by binding it on the host, and the sandbox refuses every port but the ssh port. End to end, with Lima’s own rule put back for one start inside the sandbox, Lima tries the guest’s TCP and UDP ports, is refused (operation not permitted), and has no port but its ssh port, while devvm’s forwarder forwards the TCP port.
The ssh port on every address: listed. Seatbelt allows the ssh port on any address, so a Lima that bound it on 0.0.0.0 would put the VM’s sshd on the network, where it still takes only keys. Lima binds it on 127.0.0.1, and the end-to-end checks look at the hostagent’s listeners. The rule allows the port over TCP alone, so the guest’s UDP server on that port number can’t get a forward from it; checked end to end.
A program holding a stopped VM’s ssh port: ruled out, but for a race. A start looks for the recorded port with a bind on 127.0.0.1 and ::1 as well as a connection, so a program that has the port on any address, even one that accepts no connections, gives the VM a new port. Only a program that takes the port between that check and Lima’s bind, a few seconds later, gets Lima’s ssh, which is listed under not covered below. End to end, a VM whose port a host program holds starts on a new one, recorded, and the program keeps its own.
Two VMs choosing the same ssh port: confirmed, fixed. Without a lock, two devvm commands making VMs at once could each choose the same port from the ports other VMs record, and the second VM’s Lima would wait on a port that the first one’s has. devvm chooses and records a port while it holds the claims lock, run/claims.lock in its data folder, after the VM’s own lock (locks).
A build VM’s ssh port: confirmed, fixed. A port that Lima picked at a build’s start would be one that no VM’s state records, so another VM forwarding every port could take it in the window before Lima binds it, and Lima’s ssh would then run the build’s scripts in that VM. The fix: devvm chooses every build VM’s ssh port, on macOS and Linux, under the claims lock, and records it in run/build-ports/<name>, which every forwarder reserves, until the build VM’s start returns. A record that a killed devvm leaves is deleted the first time devvm chooses a port once the record is 15 minutes old, because no process holds its lock, and until then it keeps the port from any VM while the start that the killed devvm left may still bind it. Builds carry no credentials, an image is the build VM’s own disk, and a tool build writes its images through a folder that only its own VM mounts, so even a VM that reached a build’s ssh another way could break that build but couldn’t learn a secret from it or choose what an image or a tool image holds. It’s unit-tested on both systems, and the CLI tests check that a build’s start sets the port.
What a failed start or create leaves: ruled out. A VM whose create fails, other than by an interrupt, keeps its state, and with it its recorded port, until devvm rm deletes it, so the port stays reserved and nothing else takes it. devvm rm deletes the state, which frees the port.
Unix sockets: ruled out as a way out. The hostagent binds them only in the VM’s Lima folder, which only Lima’s own commands and devvm use.
DNS around the proxy: ruled out. With Lima’s resolver off, the guest’s DNS server is 127.0.0.1, where nothing listens, and Lima’s network’s DNS address and the host’s address on it answer nothing even when root asks them directly, because the sandbox denies the hostagent the host’s DNS service and every connection but the proxy’s; checked end to end.
A compromised hostagent: listed. The bind rules stop a Lima that misbehaves, not code an attacker runs in it, which the profile still lets read and write your files.
Starts without the sandbox: listed. Only confined starts on macOS run under it. A build keeps Lima’s host resolver, so its binds aren’t denied, and an open VM starts unconfined. Such a VM could run under a profile that denies binds alone, as its ssh port is fixed, but it would need Lima’s resolver off too, which changes how it resolves names (the user-mode network’s own DNS instead of the host’s resolver), so that’s proposed rather than done.

Threat check for listed ports (the security-audit skill, its local IPC companion). Listed ports use the same forwarder with -only, so these leads are what listed ports add, each confirmed or ruled out:

A port the VM chooses: ruled out. -only comes from the VM’s state, less the ports a host program has, both read on the host by the forwarder’s owner, and the VM’s reports can’t add to it. A listed port is forwarded only while a server in the VM listens on it, as with every port.
Port 22 listed: ruled out as a collision. Port 22 in the VM is its sshd, and Lima’s ssh forward comes from another host port, so a listed 22 forwards the VM’s sshd to the host’s port 22 as it would forward any server. The VM’s sshd still takes only keys. A host whose own sshd has port 22 keeps it, with a ports_taken warning.
A listed proxy port: ruled out. The forwarder reserves the VM’s own proxy port whether or not it’s listed, and every other VM’s as well.
A listed port below 1024: taken because the user asked for it. A host service that has the port keeps it, and a bind the host refuses is logged. One that isn’t running yet is under Not covered.
A forwarder left behind: ruled out. devvm stop stops it, devvm rm stops it and removes its files, any other stop ends it through its lifeline, and a start or change whose listed ports are all taken runs none. Two devvm runs that start a VM’s forwarder at once take turns on the VM’s lock, so the second stops the first one’s forwarder, and neither leaves one that its pid file doesn’t name, which nothing would stop.
A tier’s confinement: ruled out. Listed ports take fewer ports than every port, never more, and every connection still starts on the host.

Not covered:

A VM can take a free port from 1024 up, or a listed one, on the host’s loopback that a program there expects to be its own, such as one that isn’t running yet or is restarting. What the VM serves there reaches whatever connects, so a page it serves to the user’s browser can reach the internet from the host, past the VM’s tier.
A host program that starts on a port while the VM has it forwarded still binds it when it listens on all addresses with SO_REUSEADDR, as most servers do (checked on macOS). Its loopback connections then reach the VM, because the forward’s address is the narrower one. A program that binds without SO_REUSEADDR is refused.
The VM can make the forwarder dial a host loopback port that’s in use and then close the connection at once without sending anything, because that’s how the forwarder checks for a port held only by ended connections. This happens at most once per port every second, as the VM’s listener on the port goes and comes back.
Servers only on the VM’s other addresses, such as its network address or a container’s, aren’t forwarded.
Other users on the host reach the forwards, because they’re on the host’s loopback.
UDP isn’t forwarded. Ports shows how to tunnel it over TCP.
A server that keeps a port while its forward couldn’t be made, because the host had the port, isn’t forwarded until it stops and starts again.
Another user on the host can take a VM’s ssh port while the VM is stopped or booting. Lima’s own ssh would then reach that user’s program, as it checks no host key, while devvm’s forwarder and ssh config pin the VM’s.
A VM started outside devvm, such as with devvm limactl start, or one whose forwarder ended, runs without forwarded ports until its next start through devvm, or until devvm ports changes its ports, which starts a forwarder with the host key devvm pinned at its last start.

End-to-end checks (tests/e2e, project, and network for the sandbox’s binds):

Lima’s rules ignore everything, the forwarder runs with the host key devvm pinned for the VM, and Lima’s ssh listens on the port the VM’s state records.
A stopped VM whose ssh port a host program holds, on 127.0.0.1 alone and accepting no connections, starts on a new port that its state records, and the program keeps its own.
A VM server on a port answers at the host’s 127.0.0.1 through the forwarder, while Lima’s hostagent has neither that port nor a UDP server’s port. The forward listens on the host’s loopback alone. A port that a host program has on every address keeps that program’s localhost connections while a server in the VM listens on it, and the forwarder logs that port as not forwarded.
With listed ports, only those reach the host, whatever address the VM’s server is on. A listed port that a host program has is left out with a warning, and forwarded again once it’s free.
On a running VM, devvm ports changes the forwarder’s ports at once. A port that devvm ports add lists has its server reach the host at once, and once devvm ports rm takes it out, it doesn’t. The VM’s boot ID is the same throughout, so it never restarted.
devvm stop stops the forwarder, limactl stop outside devvm ends it (on macOS), and rm stops it and deletes its files.
On macOS, a confined VM has a profile that denies every bind but its ssh port’s over TCP and unix sockets in its Lima folder, and the same binds that work outside the profile, a TCP port and a UDP socket on the ssh port, fail under it. Its hostagent listens on its ssh port alone, with Lima’s resolver off. Started again with Lima’s own forwarding rule put back inside the sandbox, the hostagent tries a VM server’s TCP and UDP ports and is refused, and has neither. The VM still works: ssh by name, devvm’s forwarder (on the host’s loopback alone), the proxy and the self-check. Its next start puts the ignore rules back.
On a confined tier, DNS fails at once, and Lima’s network’s DNS address and the host’s address on it give no answer for a name.

devcontainer.json

A new VM gets what the project’s devcontainer.json says, as far as devvm does it (its page).

The guarantee: nothing from the file runs on or reaches this machine: it reads none of its environment, files, credentials, Docker images or registry logins, adds no mount, writes nothing to the project folder, and weakens no confinement (the network tier, .git protection). It shapes only what’s in a new VM and what runs in it, as the VM’s user. devvm’s own handling of it (reading, parsing, state, the environment file, commands, ports, the image lookup) can’t be turned against this machine or devvm. Reviewed: the threat check’s leads are fixed, ruled out or listed below.

The problem: the file is the project’s, so the VM’s: the VM writes the folder, and a cloned repo’s file is whoever wrote it. Dev Containers runs much of it on the host (initializeCommand, ${localEnv:…}, builds, mounts, runArgs, docker-outside-of-docker’s socket), pulls images with the user’s logins, and forwards ports at will.

How it’s kept:

Reading: only in the folder, through os.Root (a link out of it isn’t followed, .. included), opened without blocking (a FIFO the VM made in its place opens at once and is refused) and checked as a regular file on the open descriptor, at most 1 MiB. It’s read once, at plan; what the VM is made from is the parsed plan, recorded in its state, never the file read again. Later launches only hash it and warn.
Parsing: comments and trailing commas are blanked at the same offsets, strings untouched, then encoding/json; values of the wrong kind, a NUL, an environment variable’s name that isn’t one, and every variable devvm doesn’t fill in are refused before anything’s made, ${localEnv:…}/${env:…} above all. Nothing is filled in from this machine: ${containerWorkspaceFolder} is the folder’s path (the same in the VM), ${containerEnv:X} the VM’s own X, left to its shell.
Mapped, or refused, or ignored: build/Dockerfiles/Compose are refused; initializeCommand never runs; mounts, runArgs, privileged, capAdd, securityOpt, workspaceMount and the rest are ignored, named. Features map only to devvm’s own layers (docker-in-docker to the docker layer, four toolchains); no feature’s script is fetched, docker-outside-of-docker is refused, everything else too.
The image: --from’s path, but only an image its registry gives anyone: asked anonymously (from.Lookup.Public, never Docker’s store), pinned to that digest, which the build pulls by. A registry named localhost or by an address that isn’t the internet’s is refused by name, and the lookup’s client (from.NewPublicRegistry) connects only to the internet’s addresses, whatever a name resolves to, its token service’s included; offline, only a registry digest recorded for that reference stands in.
The docker layer: the backend’s own rules for it, as when it’s named; the file never changes the VM’s tier.
The environment: each variable’s value is built as a shell word (single-quoted text, "${X}" for the VM’s own), stored as JSON in the VM’s state, and written as export NAME=word into the VM’s environment file before devvm’s own (which win); a name is checked again when it’s written.
Commands: stored as argv in the VM’s state; run by the backend’s Exec (on Lima, one sh -c string, each word in single quotes), as the VM’s user, in the folder, after the self-check, .git’s locks and a --clone VM’s clone: confined like anything in the VM. Their output goes to the run log, shown only through printable.
Ports: only numbers and localhost:N/127.0.0.1:N, added to the user’s ports as devvm ports add adds them (filePorts, through AddPorts and RemovePorts; --ports wins when the VM is made): with every port forwarded, the default, they’re among them already. The ones the file added are recorded (devcontainer.ports_gave), so devvm update takes out only those the file no longer lists and never brings back one devvm ports rm took out. Forwarded on this machine’s loopback alone, as any listed port, by the one forwarder, never a host port of another program’s (ports_taken). A repo’s file can so widen a list the user narrowed, to more of this machine’s loopback, which is the user’s decision to accept by opening the folder; --no-devcontainer skips it.

What it relies on: os.Root and O_NONBLOCK; encoding/json; Lima’s Exec passing arguments unchanged; the registry’s digest naming what’s pulled (a pull checks it); the backend’s own confinement of the VM and its builds.

What could break it: a backend whose Exec runs its arguments through a host shell; a new property mapped without the same review; a host proxy that reaches this machine or its network for a public-looking name (Not covered).

Not covered:

The project’s own VM and the lookup through a host proxy (HTTPS_PROXY), where only names are checked, not what the proxy resolves them to: Attack vectors lists both, under its devcontainer.json entries. Its image is any public image, whose code runs as root in the build VM, as with --from.
A name that resolves to the internet for devvm and to this machine’s network for the backend’s pull (DNS rebinding) gets only content with the digest the public registry gave, which only someone who has that content can give.

Threat check (the security-audit skill; its local IPC, data isolation and supply chain companions). Leads, each confirmed and fixed, or ruled out:

An image only this machine or its network has (localhost:5000/…, a private address, a name that resolves to one, its token service on one): confirmed: a backend that pulls it from this machine reaches this machine’s loopback, and the lookup was made on this machine; fixed: refused by name and by address (publicHost, NewPublicRegistry), a recorded digest included; unit-, CLI- and end-to-end-tested.
forwardPorts over a user’s port list (added to devvm ports -g’s list, or to none, a limit the user set): accepted: the file’s ports are added to the user’s list, on this machine’s loopback alone, as the Ports entry above says, so opening the folder accepts them, and --no-devcontainer skips them; CLI- and end-to-end-tested.
The JSONC stripper and parser (comments in strings, an unterminated comment or string, a BOM, trailing commas): ruled out: blanking keeps offsets and leaves strings alone; anything else is invalid JSON; fuzzed (FuzzParse): no panic, every variable a shell word, nothing of this machine’s environment kept.
Reading (a link out, .., a FIFO, a device, a huge file, a swap between the check and the read): ruled out: os.Root, a non-blocking open, the descriptor’s own Stat, 1 MiB; one read per new VM; unit-tested (links, a FIFO).
Variables (${localEnv:…}, ${env:…}, nested or unterminated): ruled out: refused, or left as text for the VM’s shell; nothing is read from this machine.
The environment file and state (newlines, quotes, $(), a name like A;rm): ruled out: values are shell words, JSON-escaped in state (one line), names checked twice; the file is the VM’s own.
Command quoting (', newlines, an argument starting with -): ruled out: devvm runs bash -lc 'exec "$@"' with the argv after it, and Lima quotes each word.
Messages (the file’s values in this machine’s terminal): ruled out: every message value goes through msg.OneLine as it renders, the image through from.Valid, errors’ causes aren’t printed, the log tail goes through msg.Printable.
Commands before confinement: ruled out: they run after the self-check (which stops a VM whose policy fails), .git’s locks, and a clone’s seeding.
A digest the registry made up (Docker-Content-Digest): ruled out: sha256: and 64 hex digits, and every pull checks content against it.

End-to-end checks (tests/e2e, project): a VM made from a file: its variables in the shell, postCreateCommand once and postStartCommand at each start, in the VM and folder, its port forwarded and no other, nothing written to the folder, a changed file only warned about; a file whose image is on this machine’s registry refused, nothing made.

Network tiers

A VM’s network is one of four tiers, open, internet, allowlist and none, from the lowest to the highest (what each reaches). The design keeps two things: every VM’s requests go through its own proxy, whatever its tier, and only the open tier ever runs a VM unconfined.

What it’s built from: netpolicy.Tier, in the VM’s net section as tier. An unknown tier fails to read (Tier.UnmarshalJSON), and TierOf reads a section it can’t read as none, never as open; a section without a tier reads as allowlist. Tier.Net maps the tier to the backend’s confinement: NetProxy for every tier but open, and NetDirect with the proxy’s port and password for open (Confinement.GuestProxy). netpolicy.Policies gives each tier its policies (PolicySet), and Prepare writes them into the proxy’s config: builtin_policies for the built-in policies and policies for your files on the allowlist tier, deny_policies for the user’s files on internet and none, whose allows the proxy drops as it reads them, and open for the open tier, which allows every host and address with no floor and takes no policy at all (Config.check). Every tier’s proxy has the VM’s CAs and its bindings file, an open VM’s included, and every VM’s setup sends the same placeholders and redacted copies (stageSecrets), so no VM holds a real secret and the tier decides only the confinement and the rules. A running VM keeps the confinement it started with: one made open while it runs confined stays on NetProxy until its next start, with an open proxy. Every Prepare reads the tier from the VM’s state as it is then, so a command that read the state before a tier change never starts the proxy with the tier before.

Rules: ** covers addresses as well as names, and :* any port 1 to 65535. Neither lifts the floor, because only an address rule can (policy.addrAllowed), so the internet tier’s **:* reaches every public host and address and no internal one.

Lima: an open VM runs as a VM without a proxy would, with no Seatbelt profile and no QEMU wrapper, and its guest reaches its proxy at 192.168.5.2, the host’s loopback on Lima’s user-mode network, on macOS and on Linux. propagateProxyEnv is off for it, so the host’s proxy settings never reach the guest, and its NO_PROXY adds Lima’s names for the host.

Links: a client links only to a target on its tier or a stricter one (policyRule), and a group’s VMs are all on one tier. A tier change that would break either is refused (tierLinked, group_tier_fixed).

Threat check (the security-audit skill, guidance mode). Leads, each ruled out, fixed or listed:

The open flag reaching a confined VM’s proxy: ruled out. Open is set only for the open tier, a config with open and any policy is refused, and an unknown or unreadable tier reads as none.
A confined VM reaching the host or the LAN: ruled out. The confinement is the tier’s (Tier.Net), the floor still judges every address the proxy dials on a confined tier, and ** and :* lift no floor (TestAnyPort, TestFloor).
A tier change racing the proxy’s config: fixed. Prepare read the tier from the state its caller had read, so a command started before devvm network changed the tier could start the proxy again with the looser one; it now reads the tier from the VM’s state as it is then. Listed: two devvm commands at once can still interleave between that read and the proxy’s start, and the next launch puts the right config back.
A running confined VM made open: listed, by design. Until its next start it stays confined, but its proxy is open’s, so through the proxy it reaches this machine’s loopback and the LAN, as the open tier you asked for does. devvm network open asks first in a parent folder (widen), and the VM keeps its CAs, which its open proxy goes on signing with.
Other programs or VMs using an open VM’s proxy: ruled out. Every request needs the VM’s own random password, which is in no process’s arguments: the proxy’s config is mode 600, and the open self-check names $HTTPS_PROXY, not its value (TestSelfCheckOpen).
The company proxy’s password: improved. An open VM no longer gets the host’s proxy settings, so neither the shell’s password nor config.toml’s reaches its guest.
An open VM spending a secret past its proxy: ruled out. Its environment, its git credential helper and its agents’ copies hold only its placeholders, which end in its own salt, so a request it sends directly carries a placeholder that the secret’s host refuses, and only a request through its proxy, with its password, gets the real value.
An open VM’s proxy putting secrets into another host’s requests: ruled out. The open flag lifts the rules and the floor, not the bindings: the proxy intercepts and puts a secret in only for the binding’s own hosts on :443, and a request inside the tunnel that names another host gets 421, as on every tier (TestInterceptOpen; TestPrepareOpen and sync.txtar for the VM’s CAs, salt, bindings and placeholders).
An open VM’s CA: listed, as on every tier. The VM trusts its own CA, whose key stays on this machine, so its proxy can impersonate the secrets’ hosts to that VM alone.
** matching addresses on the allowlist tier: listed. A policy with ** now allows public address literals too, which it refused before.
Old VMs: listed. A VM made before tiers reads as allowlist, so one that was open starts confined, and a running one shows as unenforced until devvm restarts it.

End-to-end checks (tests/e2e): network: open from its next start, a direct connection works, its proxy is set, answers the self-check and carries the VM to this machine’s loopback and its LAN address, while the allowlist tier refuses this machine’s loopback; the open VM holds no real secret, its proxy puts the token in over TLS the VM verifies with its own CA, and a request around the proxy carries only the placeholder and gets a 401; open behind a company proxy, the guest gets its own proxy and never the shell’s settings, which go through the VM’s proxy alone; internet reaches a host off the allowlist and another port but not this machine’s LAN address, and a direct connection still fails; none refuses github.com, lets Claude Code’s API through, and keeps a deny. link: an allowlist client’s link to a none target goes on, and to an internet one is refused and warned of.

A company proxy

Behind a company proxy, nothing reaches the internet without going through it, so each VM’s proxy goes through it too (how), and each build goes through a proxy of devvm’s for the build alone, which goes through it. The design keeps two things: the proxy still decides every request by the name the VM asked for, and the company proxy’s password reaches nothing of a VM’s or a build’s.

What it’s built from: the host’s HTTPS_PROXY, HTTP_PROXY and NO_PROXY, read from devvm’s environment (backend.ReadHostProxy) when a VM’s proxy starts, or config.toml’s [proxy] when the environment sets neither proxy (settings.HostProxy), checked (HostProxy.Upstream: only http:// proxies), and written into the proxy’s config, proxy.json in the VM’s run folder, vms/<hash>/run/ (mode 600), as upstream (internal/proxy/upstream.go). config.toml is a file people copy and keep with their dotfiles, so it never holds the password: its URL names the user, and the password is the proxy secret (proxy.UpstreamSecret), in the Keychain or devvm’s secrets folder, which the proxy looks up through IO.Secret as it starts (upstreamPassword), so no file of devvm’s holds it either. It’s a host-only secret (secrets.Secret.HostOnly): no VM is ever bound to it. The proxy opens its own CONNECT through the company proxy, by name, with Proxy-Authorization: Basic from the URL, for tunnels and for the TLS connections it makes to put secrets in, which check the server against the system’s roots and the user’s CAs. Plain HTTP goes to the company proxy in absolute form. A change to the settings starts the VM’s proxy again at devvm’s next run, and an old proxy that won’t stop fails the start (proxy_stop_failed) rather than run on with its old settings.

What an upstream changes: the proxy resolves a name and dials the addresses it checked only when it connects directly. Through a company proxy, the company proxy resolves the name, so the floor can’t hold it to checked addresses:

An address the VM asks for is judged as before, and localhost, *.localhost and loopback and unspecified addresses never go upstream, so the floor refuses them as before unless a rule names them. A host whose last label is all digits or starts with 0x (127.1, 2130706433, 0x7f000001) is refused as no host name (hostOK), because a company proxy that reads it as inet_aton does would reach an address the floor never judged.
A name is refused when this machine’s resolver answers with any internal or denied address (floorUpstream). A name this machine can’t resolve within three seconds goes upstream unjudged only when the policy names it, as itself or under *.domain (policy.byName). When only ** allows it, the proxy refuses it (errUnresolved), because ** would otherwise send the company proxy any name the VM makes up, and the company proxy may resolve such a name to its own network’s hosts.
The company proxy may still resolve a name differently, or later, than this machine did (DNS rebinding between the two lookups). A VM that controls a name under an allowed domain can choose the unjudged path: its name server can fail, or answer after the proxy’s three seconds, and then answer the company proxy with an internal address.
So under a name the policy lists, a VM reaches whatever the company proxy reaches: its company’s internal hosts, and the company proxy’s own host, including this machine’s loopback when the company proxy runs on this machine (a local proxy that a VPN client runs, for example). Address denies apply only to the addresses this machine resolves. The allowlist stays the limit: a name the VM’s policy doesn’t allow never reaches the company proxy.
The proxy reads at most 64 KiB of the company proxy’s answer to CONNECT (maxConnectReply), so an answer whose headers never end fails the request instead of filling memory.
The company proxy sees every allowed host name and port, and with a CA in the user’s CA folder, it reads TLS too, as it does for this machine.

Credentials: the company proxy’s user and password are in no process’s arguments (only the file’s path is), in no log (the event log’s upstream field and the proxy’s own log name the company proxy as host:port, and the URL the proxy’s HTTP transport uses has no user), and in no request but the company proxy’s own: the proxy drops the VM’s Proxy-Authorization and sets the company proxy’s only for a request it sends there. The VM never sees them, not even in a refusal, which names the company proxy by host:port and its status code.

Builds: a build runs code that isn’t devvm’s: layer scripts run third-party installers, and with --from, the container image’s own shell and package manager run as root in the build VM. So a build never gets the company proxy’s settings. Behind a company proxy, each build goes through a proxy of devvm’s on the host for that build alone (backend.ProxyStarter). It’s devvm _proxy -config with the same config as a VM’s proxy (proxy.Config): a policy file of **, which allows any host on ports 80 and 443 but the host’s own and internal addresses, a new random password, the company proxy’s settings as its upstream, and its lifeline on the build VM’s hostagent, all in one file of mode 600. The build gets only http://devvm:<password>@<host>:<port>, in each proxy variable the host sets, and the host’s NO_PROXY. When the build ends, devvm stops that proxy and deletes its config, which holds the password and the company proxy’s, and a successful build’s folder goes too. The password is worthless once the proxy has stopped, and the build never sees the company proxy’s address, user or password. Without a company proxy, builds start no proxy and get no proxy settings.

Lima: the build VM (an image’s, the tools build’s or --from’s conversion VM) reaches the proxy at 192.168.5.2, the host’s address on Lima’s user-mode network, with vz and with QEMU, and on macOS its Seatbelt profile allows that one port of the host’s loopback alongside its ssh port. The proxy runs on the host, so a company proxy on this machine’s loopback, such as one a VPN client runs, works for builds on macOS too. Lima’s propagateProxyEnv would write the host’s proxy settings into the guest’s /etc/environment between #LIMA-START and #LIMA-END at each boot (Lima’s 07-etc-environment.sh), and a stopped build VM’s disk would keep that block, so build VMs start with propagateProxyEnv: false, and only a VM on Lima’s own network whose guest doesn’t name devvm’s proxy (NetDirect without Confinement.GuestProxy) starts with it on. Every project VM names its proxy, an open one’s included, so every project VM starts with it off, and its first start makes that change in the edit that adds its proxy to its environment. Lima also logs the proxy settings it reads, password and all, into devvm’s run log, so devvm runs limactl without them, and only a start with propagateProxyEnv on gets them back. Each build script gets the build proxy’s variables exported at its start, on its input with the script (internal/backend/lima/buildproxy.go), and /etc/sudoers.d/devvm-build-proxy keeps them for sudo until the scripts end. The build proxy’s files and logs are in the build VM’s folder, in run/builds/<name>/run/build-proxy/ in devvm’s data folder.
--from’s conversion: oci/build.sh runs as root with the build proxy’s variables for crane. The container image’s own code runs only in a chroot under env -i: the /etc/os-release check gets PATH alone, and chroot_run (bootify.sh, the CAs’ ca.sh) gets PATH, HOME, LC_ALL and the build proxy’s variables, for bootify.sh’s package downloads.

The build’s policy is ** alone, so on its own the build’s proxy would refuse every name this machine can’t resolve, and where only the company proxy resolves outside names, a build would reach nothing by name. Lima’s build proxies start with upstream_unresolved in their config, which sends such names on to the company proxy unjudged. The refusal would protect little there, because without a company proxy those builds are on Lima’s user-mode network and reach internal addresses directly.

End-to-end checks (Lima):

images: a layer built with a fake company proxy’s password in HTTPS_PROXY and HTTP_PROXY (a closed port, NO_PROXY=*): its script and sudo see the build’s proxy at 192.168.5.2 with user devvm, never the password, the build’s proxy is gone afterwards, and neither the image’s whole disk, nor Lima’s files for it, nor devvm’s logs hold the password;
from: a company proxy of the run’s own with a password, and a --from alpine:3.21 base, with every feature, and a layer on it: the registry, Alpine’s mirror and the layer’s own download reach the company proxy through the builds’ proxies; what the layer’s commands and sudo have is the build’s proxy with a password of its own; and the password is in no converted disk, image, log, build file or process’s arguments;
network: a company proxy of the run’s own (a devvm _proxy with a password, allowing any host): the VM’s next launch with it in devvm’s environment starts the VM’s proxy again with -upstream; an allowed host goes through it by name (its event log) and the VM’s event log names it; a host the VM’s policy doesn’t allow gets 403 and never reaches it; its password is in no process’s arguments, no log and nothing of the VM’s; the launch after, without it, starts the VM’s proxy again going directly.

Not covered: https:// and socks5:// company proxies, which devvm refuses (upstream_proxy_scheme); proxy auto-configuration (PAC) files and the macOS system proxy, which devvm doesn’t read; Kerberos or NTLM authentication. An open VM goes through its proxy as every VM does, so no project VM gets the host’s settings in its /etc/environment. What devvm’s redaction of Lima’s logs leaves open is under your company proxy’s password.

The desktop

devvm desktop shows a VM’s desktop on this machine with VNC, and keeps two things: no one on this machine or the network reaches the desktop without the VM’s password, and the desktop gives the VM no new way out. Both hold whatever the VM’s own server does, because root in the VM can replace it (how).

What it’s built from: the built-in desktop layer (assets/layers/desktop.sh): Xfce and TigerVNC’s Xvnc from the distribution’s archive, and /usr/local/bin/devvm-desktop, which starts Xvnc on a socket of the VM’s user’s own (/run/devvm-desktop/vnc.sock, mode 600 in a 700 folder), with no TCP port and no password, then an Xfce session. On this machine, internal/desktop’s Relay, which devvm desktop runs (internal/vm/desktop.go): a listener on 127.0.0.1 alone (platform.ListenLoopback, without address reuse), on the port the VM’s state records (the ports section’s host.desktop, chosen by ports.Allocator and among ports.Used), and for each viewer that passes its check, an ssh -W channel to the socket through devvm’s ssh config’s entry for the VM. Only a VM made with the layer has the desktop (image.DesktopLayer), because it adds about 370 MB to an image, so base and the VMs without it stay that much smaller.

Why devvm’s own relay, not the port forwarder: the forwarder forwards every port a VM listens on from its start, on the same port number. A VNC server on a port would have been on this machine’s loopback at every start, without anyone running devvm desktop, on a port another VM’s desktop or macOS’s own Screen Sharing server (5900) can have. The relay is open only while devvm desktop runs, on a port of the VM’s own, and checks the password itself, which the forwarder, a byte copier, can’t.

The password: VNC’s own authentication (RFB security type 2: DES of a 16-byte challenge, desktop.Response) is what macOS’s Screen Sharing speaks to a server that isn’t a Mac, and it takes 8 characters. devvm’s are random, from 31 characters (about 40 bits), checked one at a time with a second’s wait after a wrong one, so an online guess takes thousands of years. The relay checks it before it starts any ssh, so a wrong viewer costs the VM nothing. It’s kept in the VM’s state (the desktop section’s made.password, mode 600), shown only with the address (desktop_ready, a message, not the run log), and never sent to the VM, so a VM can’t use it to reach its own desktop’s port. Screen Sharing gets the address alone (open vnc://127.0.0.1:PORT), not the password, which would be in open’s arguments where any process on this machine reads them.

What keeps the VNC connection from carrying the VM out. Every connection starts on this machine: a viewer connects to the relay, and the relay asks devvm’s ssh for a local channel (-W) that the VM’s sshd connects to the VM’s own socket. The ssh asks for no session, agent, X11, tunnel or remote forward, and OpenSSH’s client refuses a channel the server opens unasked, as for the forwarder. What the VM sends back goes only to the viewer that connected. The VM’s server reaches nothing on this machine, so the VM’s tier holds as it is: the VM’s own connections still reach only its proxy, and devvm allows no new host for the desktop.

The clipboard: off unless --clipboard. Without it, Xvnc starts with AcceptCutText, SendCutText, SetPrimary and SendPrimary off, and the relay drops every ClientCutText from the viewer, extended clipboard ones too (internal/desktop/filter.go). It reads the viewer’s messages by their lengths (RFC 6143’s and the extensions TigerVNC offers: fences, continuous updates, desktop size, xvp, GII, QEMU’s key and audio events) and ends the connection on a type it doesn’t know, so nothing it can’t read reaches the VM. The other way, from the VM to the viewer, isn’t read: following the server’s ServerCutText would mean decoding every screen update’s encoding. A VM whose server was changed can send its clipboard to the viewer, so Screen Sharing’s Use Shared Clipboard setting is what keeps it off this machine then.

The local proxy, for browsers: Chrome, Chromium and Firefox ignore the password in the VM’s HTTP_PROXY and ask for it on every site, so the desktop’s session uses devvm-local-proxy instead (how), which the guest setup writes (assets/guest/sync.sh) and socat serves. It gives no program a power it lacked, because the password it adds is the one the VM’s environment already holds, and every program in the VM can read it there. What it must not do is hand the password to anything outside the VM, or send it anywhere but the VM’s proxy:

Where it listens: 127.0.77.1:3128 (proxy.LocalProxyAddr, proxy.LocalProxyPort), on the VM’s loopback alone (bind=), taking connections from 127.0.0.0/8 alone (range=). The port forwarder never forwards a server on that address (vmssh’s dialAddr, proxy.IsLocalProxyAddr), so a VM that forwards every port doesn’t put a proxy that needs no password on this machine’s localhost, where other users and programs would reach the VM’s proxy, and its secrets, without the password.
Where the password goes: only to the VM’s proxy, whose address is written into the script with the password, never to an address a request names. It’s in the one Proxy-Authorization header the local proxy adds, after it drops the client’s, and devvm’s proxy removes that header before it forwards a request. A CONNECT’s tunnel carries nothing of the local proxy’s after its head.
Web pages: a page can reach the loopback as a server, but its requests are in origin form (GET /), which the local proxy refuses with 400 before it connects anywhere. A page can’t send a CONNECT or an absolute-form request line, and DNS rebinding changes only the Host header, so a page can’t use the local proxy to reach the VM’s proxy.
One request per connection: a request that isn’t a CONNECT goes with Connection: close, because the local proxy reads only a connection’s first head, so no later request on the connection reaches the VM’s proxy without the password, as it would otherwise get 407 and the browser’s prompt.
The file: /usr/local/bin/devvm-local-proxy holds the password, mode 700 and the VM’s user’s, which is stricter than the VM’s /etc/environment, where the same password already is.

The tier holds as it is: the VM’s proxy decides each request the local proxy sends, and a host the tier doesn’t allow gets its 403.

End-to-end checks (tests/e2e):

desktop (Lima): a VM of base without the desktop’s programs, which devvm desktop refuses (desktop_no_layer); then a VM with the desktop layer on the allowlist tier: the relay on 127.0.0.1 alone, its port and password in the VM’s state; a wrong password refused, the password let in, with the VM’s ServerInit and a screen update (TestE2EDesktop); Xvnc on its user’s socket (700, 600) and no TCP port, the clipboard off; the password in no file of the VM’s, no run log and no host process’s arguments while devvm desktop started; the self-check passed and no direct connection out, nor to the desktop’s port on this machine; the Xfce session’s proxy variables set to the local proxy, which listens on 127.0.77.1:3128 alone, takes curl without a password to an allowed host, gives a blocked one the proxy’s 403 (CONNECT and plain HTTP), refuses an origin-form request with 400, and isn’t forwarded to this machine; root’s own server in place of Xvnc (no password, its socket anyone’s, recording what reaches it): a wrong viewer never reaches it, and the viewer’s clipboard never does while its key event does; Ctrl+C closing the port; --clipboard on the same port, the clipboard on.

Not covered: the VM’s clipboard sent to a viewer that takes it, by a server root changed; Screen Sharing itself, which no check drives, so its messages are the relay’s reading of the protocol; a program on this machine with the password, which reaches the desktop as you do; a program on this machine, or a VM that reaches this machine’s loopback, that keeps the relay’s 8 connections busy or keeps failing its check, which keeps viewers out for as long as it does, and gets no further; the browsers themselves, which no check drives, so that they read the session’s proxy variables rests on Chrome’s and Firefox’s documented behavior on Xfce, and the local proxy’s handling of requests on its unit test (netpolicy’s TestLocalProxy) and curl.

Base’s image and packages

base starts from the newest Ubuntu cloud image on each machine, and its packages are the newest when each key of base first builds, so devvm’s own files pin neither and need no bumps (how, packages). What it trusts, and where:

The image: Ubuntu’s cloud image key alone, pinned in devvm’s files. devvm checks the release’s SHA256SUMS signature itself, in Go, so a host without gpgv checks it too, and a bad signature is refused, never passed over for an older release. The listing that names the newest release is unsigned, so a server or network that can change it can only hold devvm back to an older signed release, over HTTPS. The signed SHA256SUMS names files, not its release, so such a server could also serve an older release’s files under a newer release’s name, which devvm then records as the pin; that holds base back the same way, and it makes the package check’s date stricter, not looser. The pinned SHA-256 then decides every later download.
The packages: Ubuntu’s archive key, through apt in the build VM, which holds nothing but Ubuntu’s image and devvm’s own scripts while it records. A mirror only serves files, and apt checks each against the archive key, so ubuntu-mirror can’t install a package Ubuntu didn’t publish. Ubuntu’s InRelease files carry no Valid-Until, and the build VM has no lists before, so a stale or hostile mirror, or anyone on an http:// path to one, can serve an older signed state of the archive. devvm refuses one older than the archive state the pinned cloud image was built from, naming the mirror (packages_stale_mirror, packages_stale without one): every pocket but the release’s must be dated no more than a day before the image’s release, and the release pocket no earlier than Ubuntu’s release day (Release.Stale). So a recorded state is never older than the pinned image. A mirror that fell behind after the pinned release still gets recorded, so freshness past the pin’s date rests on the mirror, and on HTTPS for Ubuntu’s own.
The cache: a first build records what its own apt checked, and devvm checks it again on this machine before it keeps anything. The build hands back the InRelease files and package lists apt used, and devvm verifies each InRelease with Ubuntu’s archive key, pinned in its files, in Go (pgp.go’s clearsigned messages). Each list must be one a verified InRelease gives the SHA-256 and size of, and each package one a list gives the SHA-256 of, so the recording VM can’t add a package Ubuntu didn’t publish, even with root, and the set keeps the archive’s own entry for each package. devvm keeps each file under its own SHA-256 and each key’s set once. What comes back from the VM is checked for shape, hashes and size (at most 20,000 entries and 16 GiB of packages, 2 GiB of lists) before anything is kept, so a build VM can’t put a file under another file’s name, change another key’s set, or write outside the cache. A key’s set is never written again, so a later build can’t change it either. The recording VM can’t record an older version than apt would install either: the build must hand back resolute-updates’ and resolute-security’s InRelease files with lists for the recorded packages’ architectures, and every non-empty list that each InRelease it hands back gives for the recorded packages’ architectures, and devvm refuses a package that the release pocket, -updates or -security lists a newer version of, ordered as dpkg --compare-versions orders them (packages_lists_missing, packages_not_newest). apt in the build takes phased updates too, so the newest version is what it installs. A replay checks each file’s SHA-256 on this machine before it sends it, and apt in the VM checks it again against the set’s entry. Replays trust apt’s entries as recorded (trusted=yes on a local repository of files devvm checked), because the archive’s signature covers the archive’s lists, not a set of packages taken from them.
What it doesn’t stop: anything with root in the recording VM before apt.sh end (a feature’s script, a compromised pinned download) chooses which of Ubuntu’s signed packages get recorded, because the host checks that each package is the archive’s and the newest that the lists handed back have, not that apt chose it. It can record a package base doesn’t ask for, or one from -backports, but never one from -proposed. It can also hand back an older signed state of -updates and -security than the archive’s newest, dated as early as a day before the pinned image’s release, which the stale check allows, and record the newest versions of that state. Every later build of that key replays what it recorded, as the same image would. It can also change the image itself, which the image’s key doesn’t cover. devvm rebuild base records anew. Packages for --from bases and for layers come from their sources as they are, unpinned.
Breakage from upstream: users now build from images and packages that e2e hasn’t run. If the newest image doesn’t boot, a first build tries the release before it (how), which covers a release whose kernel Lima can’t start, but not packages that break base.sh or a VM later. A scheduled build of base from the newest image and packages, on CI or the test machines, should catch a break before users do; it isn’t built yet.

Shared tools

Each tool version is one compressed EROFS image, loop-mounted read-only in each VM from a read-only share (details).

Measured (macOS): ten tools take 0.85 GB as images (1.5 GB installed), once for every VM and mix. Start times from an image match a local disk warm (node -e 0 13 ms vs 14 ms, claude --version 3 ms), and are 10–30 ms slower cold. Mounting five images takes 0.15 s; VM start is unchanged.
Why images, not a shared folder: tools run from a virtiofs folder were about 3× slower warm (node -e 0 38 ms, claude --version 30 ms), the Mac’s case-insensitive disk collided 33 file names in Python’s terminfo, and mise’s version links failed in a read-only folder.
Why not per-tool layers: an image chain for every mix of languages and agents repeats each agent’s files in every mix.
Where they build: a tool image is shared by every VM whose tools need its key, and its build runs whatever runs as root in its build VM, so the key alone can’t keep one VM’s base out of another VM’s tools. A glibc tool image therefore always builds in a clone of devvm’s own Ubuntu base, on every backend, even for a VM on a --from Debian or Ubuntu image, whose code would otherwise run as root in the build of images that Ubuntu VMs then mount. --from VMs mount those images, because the tools are their projects’ own prebuilt releases, which run on any glibc new enough, and devvm still refuses a tool that needs a newer glibc than the image has (shared tools and glibc). A musl tool image can only build on Alpine, so it builds in the VM’s own --from base, and its key hashes that base’s digest, so it’s only ever another VM’s on the same container image. Before it builds, devvm checks the base’s record against what the images are for, and refuses any other base (tools_wrong_base, building tools). The cost is a build of devvm’s own base for a --from VM’s first glibc tools when it isn’t built yet.
A VM for each tool: a tool’s installer runs as root in its build VM, and a tool image’s key hashes only its own tool’s inputs, so tools built together in one VM could write into each other’s images: through a shared work folder, a process left running until the next tool’s mkfs.erofs, or root replacing mise or mkfs.erofs itself. A fresh folder for each tool and killing its processes wouldn’t stop root, so each tool builds in a VM of its own, with an output folder of its own, and only that tool’s image leaves it (building tools). Measured on Fedora 43 (bare-metal x86_64, Lima with QEMU and KVM), building node, pnpm and bun took about 16 s in one VM and about 34 s in three, the same in two runs each, so each extra tool costs about 9 s, once for each tool version.
Why a tool that doesn’t mount fails the launch: mise would otherwise download the same version into the VM on first use, which gives that VM a private copy, a download nobody asked for, and one that the VM’s tier may block, all without a word (how).
Instead of Nix: Nix and NixOS were weighed as the way to deliver the shared tools and rejected, because NixOS breaks prebuilt binaries, and a Nix hidden under the tools shows through in read-only-store errors. The nix layer is separate, and you choose it. It gives a project that uses Nix a Nix of its own in its VM, with its own store and daemon, and the shared tools stay as they are beside it.

The shared Nix cache

VMs with the nix layer share their downloads from cache.nixos.org through a cache that only the host fills (how). It’s the first thing that VMs share, so the design keeps two things: each VM’s Nix store stays its own, and no VM can change what another VM gets.

Why a cache the host fills, not a shared store: one store, or one daemon, shared between VMs is what Nix’s own store vulnerabilities break. CVE-2026-110450 (the daemon’s store corrupted through repair), CVE-2026-39860 and the CVE-2024-27297 family (a fixed-output derivation that changes its output after Nix registered it) each let a client of the daemon change paths in its store, and in a shared store, every other VM would then trust those paths. A VM’s own builds are worse: a path a VM built has no signature that another VM could check, short of a key that trusts that VM, and then every VM trusts every other. A binary cache keeps the trust where Nix already has it. Each VM’s Nix fetches from the cache as from cache.nixos.org, checks cache.nixos.org’s signature and the NarHash itself, and adds the path to its own store, so the cache only saves downloads.

What the host checks, and what each VM checks:

The proxy (internal/proxy/nixcache.go) takes only GET and HEAD of nix-cache-info, <32 nix32>.narinfo and nar/<52 nix32>.nar with no suffix or .xz, .zst or .bz2, with no query and nothing escaped (nixPath, fuzzed by FuzzNixPath), and 400 for anything else. It makes up nix-cache-info itself.
It fetches from https://cache.nixos.org alone, only when the VM’s policy allows cache.nixos.org:443, through the floor or the company proxy (dialOut), with the system’s roots and the user’s CAs, following no redirect, and sending only its own GET and User-Agent.
A narinfo is kept when its StorePath is the path asked for, its URL is a NAR path of the pattern above, its hashes are SHA-256 in Nix’s base 32, each reference is a store path’s base name, and a Sig by cache.nixos.org-1 (the Ed25519 key pinned in nixKey) verifies over 1;StorePath;NarHash;NarSize;References, the references as full store paths, sorted and joined by commas (parseNarinfo, against a narinfo of cache.nixos.org’s own in internal/proxy/testdata/nix/). The bytes kept are cache.nixos.org’s, as they came.
A narinfo’s URL must be named by its own NarHash or FileHash, since the URL isn’t signed. A NAR is fetched only after a narinfo this proxy checked names its URL (nixKnown, at most 65536 entries), and kept only when its SHA-256 and size are that narinfo’s FileHash and FileSize, read through a limit of FileSize plus one byte. A HEAD never fetches one.
Each VM’s Nix checks the signature again and the NarHash, which the host can’t: the compression is xz or zstd, which aren’t in Go’s standard library, and internal/proxy takes nothing else. The spike showed a VM’s Nix refuses both a narinfo with an added reference and a corrupted NAR.
Files go into a temporary file in the folder (.tmp-*, 600) and are renamed into place, in folders of 700. Not-found answers and errors are never kept, and the VM gets 404, so its Nix goes to cache.nixos.org itself.

The cap: nix-cache.size, 10 GiB when it’s off, passed in the proxy’s config as nix_cache_max. Before a proxy writes a file, it takes the file’s room under an exclusive flock on .lock (reserve), in whole 4 KiB blocks. When .size, the running total, is missing or the new file would pass the cap, it counts the folder again, each file by the larger of its size and its blocks, and deletes the least recently used files, by modification time, which a hit sets, down to nine tenths of the cap. A NAR being fetched is made its full size at once and counts, but is never deleted, so a fetch that would pass the cap with them is refused. A fetch that fails gives its room back. One proxy fetches each NAR once however many requests ask for it, and at most four at once. A NAR whose FileSize is over the cap isn’t fetched. devvm gc, its --nix-cache and the daily retention run trim under the same lock (TrimNixCache). A temporary file over an hour old is a dead proxy’s, and the next count deletes it.

What stays open:

Timing and filling the cache: Attack vectors lists what one VM learns of another by how fast the cache answers, and how a VM that fills the cache to its cap pushes other VMs’ paths out. Hiding the timing would mean waiting as long as a fetch on every hit, which is the saving.
Private caches (Cachix, a netrc) are never shared: only root in the VM can add a substituter, and Nix reaches it through the allowlist directly.

Options weighed:

Nix’s local overlay store, the VM’s store over a read-only lower store shared from the host. It works on Lima, but it’s still experimental in Nix 2.35, the lower store can’t change while it’s mounted, so sharing new paths means remounting, and a garbage collection by a daemon not set up for it writes whiteouts that hide lower paths from that VM for good.
Packages fetched into an image, by a custom layer that runs nix build while the image builds (the recipe). Clones share them on disk at no cost, but only what’s in the image, and a changed flake.lock needs a rebuild.
Open VMs: every VM has its own proxy, an open one’s included, so open VMs share the cache through it, with the VM’s password, as confined ones do. Ruled out: a shared loopback listener without a password, because anything on the host or any VM that reaches the host’s loopback could use it; and the cache folder mounted read-only into VMs, which leaves the VM’s Nix to read the files and the host with nothing to fill it.

Review (the security-audit skill, on nixcache.go, its callers and the VM’s setup):

What a VM sends (headers, a body, a query, an escaped path, another method): ruled out: nixGet makes a request of its own from the checked path, the VM’s User-Agent and path go only to its own event log, and what’s kept is cache.nixos.org’s body under a name of the pattern.
A path out of the folder (.., %2e, a symlink): ruled out: the names hold only Nix’s base 32, which has no /, . or %, and can’t be .lock, .size or .tmp-*; the folder is in devvm’s data folder, which no backend mounts, so nothing in a VM can put a file or link there.
Upstream elsewhere (a redirect, DNS rebinding, a company proxy): ruled out: every dial is dialOut(cache.nixos.org, 443), through the floor or the company proxy as any request, ServerName is fixed and no redirect is followed; the client is made after the user’s CAs are read.
A NAR kept under another NAR’s name: confirmed, fixed. A narinfo’s URL, FileHash and FileSize aren’t signed, so a bad upstream (or a CA in the user’s CA folder) could pair a real narinfo with another NAR’s URL and a file of its own, and every VM would then get that file for the other NAR, which its Nix refuses, until a trim. parseNarinfo takes a URL only when it’s named by the narinfo’s own NarHash or FileHash, and a hit whose size isn’t that of the narinfo this proxy checked is dropped and fetched again; unit-tested.
The cap under many proxies: confirmed, fixed. Files being fetched didn’t count and were written before any room was taken, so one VM’s parallel requests could put many NARs’ worth on the host’s disk past the cap, and a small narinfo takes a whole block. Now a fetch takes its room under the lock first, makes its file its full size at once, and is refused when the files being fetched leave no room (reserve); sizes count whole blocks; one proxy fetches each NAR once, however many ask, and at most four at once (fetchOnce); a narinfo is written under the lock; unit-tested (40 writers at once, a full cache, eight requests for one NAR).
A HEAD that downloads: confirmed, fixed: a HEAD for a NAR the cache doesn’t have answers 404 without fetching it.
Parser differences (spaces, \r, a key twice, references twice): ruled out: they only refuse, and each VM’s Nix parses and checks the bytes again, so no narinfo the host keeps means something else to Nix.
One VM learning of another: timing only: hits and fetches answer with the same bytes and headers (no Last-Modified or ETag), events go to the VM’s own log, /blocked holds none of them, and errors in the body say nothing of the host.
The VM’s own settings: ruled out: devvm-cache.conf and the !include line are root’s, in that VM; only root is a trusted user, and root in a VM changes only its own Nix, while the cache takes nothing from any VM.
Memory: ruled out: a narinfo is read up to 1 MiB, and nixKnown holds at most 65536 entries per proxy.

End-to-end checks (tests/e2e, nix): the VM’s Nix has the cache first and cache.nixos.org’s key alone; nix run nixpkgs#hello comes through the cache, fetched by the host into the folder (files 600, folders 700); a VM’s PUT and POST, a URL in the path, a query and .. get 400, a NAR no checked narinfo names and a path cache.nixos.org doesn’t have get 404, and the host keeps none of them; a second VM gets the first’s paths as hits and fetches none; after devvm network open, the open VM’s nix-daemon still goes through its proxy and keeps the cache first.

A CA per VM

To put a secret in, a VM’s proxy shows the VM certificates from a CA that the VM trusts, and inside the VM that CA vouches for any site. So each VM has CAs of its own, and the design keeps two things: a CA’s key never leaves this machine, and no VM’s proxy can make a certificate that another VM accepts (how).

What it’s built from: trust.InitCA, at each launch, whatever the VM’s tier, readies the VM’s current CA and its next one in vms/<hash>/run/ca/ (paths.VM.CA): ECDSA P-256, 30 days and an hour each, the next one starting a week and an hour before the current one ends. The proxy’s config names the four files (ca, ca_key, ca_next, ca_next_key) and their certificates’ SHA-256 (ca_sha256, ca_next_sha256), so a replaced CA starts the proxy again. The proxy signs with the next CA from an hour after its start (ca.signer), and the VM’s setup installs both certificates, devvm-proxy.crt and devvm-proxy-next.crt, so the switch needs neither a launch nor a new proxy. The guest never fetches a CA: its trust comes only from the setup that devvm sends over ssh at each launch.

Threat check (the security-audit skill, guidance mode, its data isolation and lifecycle companion). Leads, each ruled out, fixed or listed:

One VM’s CA signing for another: ruled out. Each proxy’s config is devvm’s, mode 600 in the VM’s own run folder, and names only that VM’s files, and a VM reaches no other VM’s proxy without that VM’s password. A VM trusts only the certificates its own setup sends, and the setup removes any other devvm-*.crt.
The keys reaching a VM: ruled out. The setup sends the two certificates and nothing else from the folder, /ca.pem serves certificates only, and no VM mounts devvm’s data folder. The CLI tests check the setup’s files for a private key, and e2e searches the VM’s /etc, /home, /root and /usr/local for each key.
Key files: the folder is mode 700 and each key mode 600, written whole through a temporary file. Fixed: the proxy read the CA’s files through tls.LoadX509KeyPair, which follows links and takes any mode, and it now reads them as the user’s own regular files, not links, with a key no one else may read or write, of at most 64 KiB (loadCA, readPrivate). On macOS each VM’s CA folder is kept out of Time Machine at each launch.
Expiry and rotation: a CA lasts 30 days and an hour, and each certificate it makes a day, never past the CA’s end. Listed: the next CA’s key is on disk for up to 23 days before that CA starts, so a copy of it taken then works until that CA ends, up to 53 days after the copy. A VM that isn’t opened again stays verifiable for at least 30 days less an hour after its last launch, which is the trade the successor buys.
Two launches of one VM at once: fixed. InitCA holds the CA folder’s lock, run/ca/lock, while it reads and writes the CAs, so two launches at once take turns and never leave a key and a certificate from different runs (TestInitCATogether).
Lifecycle: a remake keeps the VM’s folder and its CAs. devvm rm, devvm gc --orphans and the deletion of a VM whose creation was interrupted delete the folder with the keys. A change of tier keeps the CAs, because every tier’s proxy signs with them.
Issuer confusion: fixed before it could arise: each CA’s name holds its serial, so a VM’s two CAs never share a subject, which some verifiers look issuers up by.
Clock skew: the switch waits an hour past the next CA’s start, so a VM whose clock is up to an hour behind still accepts the certificates.

End-to-end checks (tests/e2e, network): two confined VMs have different CAs; each VM trusts its own two and none of the other’s, and its /etc/ssl/devvm.pem holds the next one; no key is anywhere in the VM, and the keys are mode 600; each proxy serves its own CAs at /ca.pem, and a certificate it presents for a bound host verifies with its own VM’s CAs and not with the other’s; devvm rm deletes one VM’s CAs and leaves the other’s. The switch itself is checked by unit tests (TestNextCASigns, TestInitCASwitch), because it happens weeks after a launch.

Links between VMs

A link lets a program in one VM, the client, reach a TCP service in another VM, the target, by name, as in postgres://db:5432/app. Links work on every tier, within the tier rule below. You run devvm link publish 5432 in the target’s folder to offer the port to any of your VMs, and devvm link add db 5432 in the client’s folder to reach it as db:5432 and db.devvm.internal:5432. A link carries its connections over ssh through the host, so it uses neither VM’s network, and the design keeps every guarantee that the tiers’ confinement and port forwarding give. The review of the code checks it against the threat check below.

The rules:

A link is reached only by its name. The client reaches the target’s port at the link’s short name and at NAME.devvm.internal, and never at its own 127.0.0.1 or ::1, so localhost:PORT in the client stays the client’s own. The short name is the target’s folder part, or the name given with --as NAME, and the target’s full VM name is never a hostname. .internal is reserved for private use, so NAME.devvm.internal names no host on the internet.
The target publishes. A port is reached only once the target’s folder has published it with devvm link publish, which offers it to any of your VMs. Each link is still made from the client’s folder, with devvm link add.
A client links only to a target on its own tier or a stricter one. A link joins the two VMs’ trust (below), so a client linked to a target on a lower tier would reach more through the target than its own tier allows: an allowlist client linked to an internet or open target would reach the internet through it. devvm checks the rule when the link is made, at each launch of the client, on every connection, and when devvm network gives the target a tier lower than the client’s or the client one higher than a target’s, both of which it refuses while such a link exists (link_tier_mismatch). For a client on a confined tier, on every connection the target’s tier must also be enforced as the target runs (netpolicy.Listed), not only recorded in its state: devvm network allowlist on a running open target takes effect at its next start, and a target whose self-check failed may reach the internet directly, so until then the connection is refused as unenforced. devvm link add, devvm link publish of a port such a client links to, and the client’s launch warn of a running target in this state with the same check (link_target_unenforced), refusing nothing, and a stopped target isn’t warned of, because its tier applies when it next starts. An open client may link to a target on any tier.
A connection never starts a VM. A connection to a stopped target fails, and devvm logs why in the client’s event log, but the target stays stopped.
devvm records links on the host, in each VM’s state under ~/.local/share/devvm, never in a project folder or a VM. The client’s VM gets only the names and addresses it needs, which devvm checks before it sends them.

A link joins the two VMs’ trust. Code in the client can often run code in the target through the target’s own service, such as Postgres’s COPY … TO PROGRAM, and it then has whatever the target has: its network, the secrets its proxy puts into requests, and its project folder. devvm doesn’t try to stop that, because it’s the service’s own behavior. So a link is safe only between VMs you’d trust with each other’s reach, and devvm link add says so when it makes one. The tier rule above keeps a link from widening the client’s tier.

What it’s built from:

A VM’s identity. Every VM gets a random ID when devvm makes it, recorded in its state. A link records the target’s VM name and its ID when it’s made, so a VM made again in the target’s folder, which has the same name and a new ID, isn’t the linked target, and a second VM with the same folder name that appears later changes nothing.
Which VM a link names. devvm link add TARGET PORT takes a VM’s full name, its ssh alias (devvm-<folder>, as its state records it), a folder’s path, or a folder’s name, which is matched against the folder that each VM’s state records, so a VM whose backend cut its name still matches its whole folder name. A folder name that matches several VMs is refused, with each VM and its folder listed.
The published ports are in the target’s state, a list of ports and ranges as devvm ports takes them, without all. devvm link unpublish takes ports out of the list, and a link to a port that isn’t published is refused on its next connection.
An address per target. Each target a client links to gets an address of its own in 127.77.0.0/16, chosen when the first link to it is made and kept for every port of that target in that client, so two targets can share a port. The client listens on the target’s own port, so devvm link add db 80 listens on port 80. The addresses are internal and appear only on these pages.
Names in the client. The setup that devvm sends at each launch writes a block marked # devvm links begin and # devvm links end into the client’s /etc/hosts, with a line 127.77.x.y NAME NAME.devvm.internal for each target, and replaces the block whole, writing the file in place rather than renaming over it. devvm sends the same block over ssh as root when links change on a running client, and it applies at once. No link name goes into NO_PROXY or no_proxy, which name the VM alone, because a running VM can’t take a new environment: HTTP clients with the VM’s proxy settings reach a link through the proxy (below). So a change of links changes no VM’s environment, restarts no daemon, and on Lima runs no limactl edit, which would give the VM new ssh host keys.
The client’s sshd. Every VM’s sshd allows remote forwards to 127.77.0.0/16 alone, with one rule written when the image is built, so no link rewrites or reloads it. That’s 00-devvm.conf from assets/layers/base.sh, which sets AllowTcpForwarding all, GatewayPorts clientspecified and PermitListen 127.77.*:*, and keep AllowStreamLocalForwarding local, PermitTunnel no and X11Forwarding no. Without clientspecified, sshd would bind every remote forward on the VM’s loopback, 127.0.0.1. sshd matches the address a request names against the rule as text, so 0.0.0.0, 127.0.0.1, ::1, localhost and * are refused, and only devvm on the host asks for these forwards, always with a numeric address. The rule is a safeguard for an honest VM, because root in the VM can change sshd, and the boundary is the host’s ssh client (below).
Low ports. At each start, the setup sets net.ipv4.ip_unprivileged_port_start to 0 where it’s 1024, which is on Lima’s Alpine VMs, so sshd can listen on a link’s port below 1024 for the VM’s user. Lima’s Ubuntu and Debian VMs already have 0, so devvm leaves it there.
The link session. For each running client with links, a hidden command, devvm _links, runs on the host as one detached process. It keeps an ssh connection of its own to the client, never the forwarder’s master, with the options of internal/vmssh: -F none, BatchMode, the VM’s key alone (IdentitiesOnly, IdentityAgent=none), its pinned host key, ForwardAgent=no, ForwardX11=no, Tunnel=no, PermitRemoteOpen=none, PermitLocalCommand=no and ProxyJump=none. It starts that connection as a master with ClearAllForwardings=yes, so the master asks for nothing at first, and adds each link with ssh -O forward -R 127.77.x.y:PORT:<socket> and removes it with ssh -O cancel, at its start and while the client runs. A cancel keeps the connections that are open, and the master ending drops them. Each link is its own -O forward, never ExitOnForwardFailure for all of them, so a link whose address a program in the client already holds fails alone, with an error naming the link (link_port_taken).
The sockets. Each link’s forward ends at a unix socket on the host, mode 600 in devvm’s run folder ~/.local/share/devvm/run (mode 700), named by a hash of the client, the address and the port, because a socket’s path is at most 104 characters. The masters’ control sockets are in the same folder, named by a hash of the client, and for a master to a target, of the target and its start too. A stale socket at the path is replaced, anything else there is refused, and the socket goes when its link or the process ends.
Each connection. When a program in the client connects to a link’s address, the client’s sshd opens a channel to the host for the forward, and the host’s ssh connects it to that link’s socket. devvm _links then reads the links state again and checks that the link still exists, that the target’s ID matches, that the port is published, that the tier rule holds, that the target runs as its backend says, and, for a client on a confined tier, that the target’s tier is enforced as it runs. The connections to one target share what its backend says of its run, which is whether it runs, its start and its ssh: one lookup is in flight for each target, and its answer is reused for one second (linkRunFor), so a burst of 256 connections asks the backend once rather than once for each. A connection that closes while its lookup runs finds the target stopped, but that lookup isn’t kept: the connections waiting on it look again, so they get the target’s real answer. A target that stops is noticed within that second, and the connection’s other checks, the tier’s as the target runs included, are made for each connection. That tier check asks the backend again, and a stopped VM never runs confined, so for a client on a confined tier it notices at once that the target stopped. When the backend then says the target no longer runs on the start that was looked up, the connection is refused as link_target_stopped, not as unenforced. It then carries the connection with ssh -S <master> -o ProxyCommand=/usr/bin/false -W 127.0.0.1:PORT through a master to the target that it keeps, one per target, with the target’s key and pinned host key. The target’s sshd connects to the target’s own loopback. When the master is gone, -S alone would make a connection of its own, and ProxyCommand=/usr/bin/false makes that connection fail, so a channel goes through the master or nowhere. A target’s master ends when the target stops, and the next one reads the target’s known hosts again, because a VM whose Lima config changed gets new host keys when it boots. devvm never starts the target. At most 256 connections are carried at once, and the next one is refused and logged. A link’s socket holds 128 connections waiting to be accepted on macOS, and the process accepts nothing while it starts an ssh, so its accept loop only accepts, and hands a burst on whole once 5 ms pass with nothing more waiting, the first has waited 50 ms, or 257 wait. Another goroutine then carries or refuses each connection in it, logging each refusal before it closes the connection, while the loop goes on accepting the next burst, and the loop waits for that goroutine only once 257 of the next burst wait. So every connection in a burst is carried or refused with link_limit, and none is refused by the socket unlogged.
The proxy’s entrance. In a client on any tier, an HTTP client that honors HTTP_PROXY sends db:5432 to the VM’s proxy, and the proxy is a second entrance to the same socket, so a link added while the client runs works at once without a restart. devvm _links writes every link the client has, carried or not, to links.json in the client’s run folder, vms/<hash>/run/ (mode 600, written whole) whenever they change or the file is gone, with each link’s names, address, port, target and socket. A link it doesn’t carry has no socket, and a session that ended leaves its list, so the proxy refuses those links (502, link_refused) instead of asking its rules, which could send db to a host the host’s resolver or a company proxy finds by that name. devvm removes the list with the VM. devvm puts the path in the proxy’s config as links, and its run folder, where the sockets are, as link_sockets, so internal/proxy stays standard library only, and the proxy reads the list for every request, after the VM’s password and before any rule, as it does for devvm.proxy. It matches CONNECT NAME:PORT and absolute-form GET http://NAME:PORT/ exactly, after one normalization: lower case, one trailing dot dropped, and NAME.devvm.internal as the list names it. It takes an entry only when the entry’s socket is the one that proxy.LinkSocket names from devvm’s run folder, this VM, the entry’s address and its port, which is also the name vmssh.Links.Socket gives the socket. It then connects to the socket and passes the bytes as they are: no SNI check, no interception, no secret put in, even when a secret’s hosts would match, and no rule asked. An absolute-form request goes over a new connection to the socket for each request, without the proxy’s headers, and HTTPS in absolute form gets 400. devvm _links’s checks of each connection apply to both entrances unchanged, and it logs the connection once, as link. A stale entry fails there, or finds no socket, which the proxy answers with 502 and logs as link_refused (not carried). The proxy also refuses any other name under devvm.internal, logged as a refused block, so a long name never reaches the rules. An absolute-form request to a link ends when the client gives it up, so a target that never answers can’t hold the client’s 256 connections. Anything else not on the list meets the floor and the rules as before, so a short name the client has no link for, before its first session lists it, is the rules’ to decide.
Events. devvm _links writes link, link_refused (with why), link_target_stopped and link_limit events to the client’s event log, in the proxy’s JSON-lines format, with the link’s name as the host and the target VM as target, so devvm network log shows them. Every client gets an event log for them, whatever its tier. devvm _links writes it with the proxy’s own writer, which rotates it past 10 MiB by the proxy’s rule and follows the file to its new name when the other writer rotated it, so the two writers never lose each other’s lines.
The forwarder skips links. The port forwarder (vmssh.Forwarder) forwards a server on any loopback address of the VM’s, so it would put a link’s listener on the host’s 127.0.0.1 and ::1. It never forwards a listener whose address is in 127.77.0.0/16, on any VM, so a server of the client’s own on 127.0.0.1 with the same port is forwarded as usual. Lima’s own forwards don’t take it either, because devvm’s rules ignore every port first, and Lima’s fallback rule matches 127.0.0.1 and the bind addresses 0.0.0.0, :: and ::1 alone.
In VMs’ lives. A launch starts devvm _links once the client runs and its setup is done, warns once for each stopped target (link_target_stopped) without starting it, and skips a link whose tier rule no longer holds, with link_tier_mismatch. For a client on a confined tier, it also warns once for each running target whose tier isn’t enforced (link_target_unenforced). devvm link add and devvm link rm on a running client change its setup and its running devvm _links at once, and on a stopped client they apply at its next start. devvm stop stops devvm _links with the client, which ends its connections and removes its sockets, and the process also ends once the client no longer runs the start it was made for, as the forwarder does, or has no links left. devvm rm of a client removes its links with its state, and devvm rm of a target removes every link to it from every client’s state and names the clients that lost one, deleting nothing else of theirs. devvm link unpublish of a port with links names the clients that link to it. Both backends must have the link feature, which needs ssh on both VMs, and devvm checks it with Review.Require when a link is made and at each launch.

What keeps a link from carrying a VM out. Every connection starts on the host, and neither VM’s network carries it:

A confined VM reaches only its proxy and the links you made. The link’s bytes travel inside ssh connections that the host opens, so no backend’s confinement changes, and the client’s own connections, including sshd’s, stay confined as before. Through its proxy, a VM reaches only the sockets of its own links: a request names a host and a port, never a socket, the proxy’s list is in the VM’s run folder, which no VM can write, and the proxy takes an entry only when its socket is the one devvm names for that VM’s link at that address and port.
The VM can’t choose where its connections go on the host. OpenSSH’s client accepts a forwarded channel only for a listener it asked for, and connects it only where it chose, which is the link’s socket. A channel the client’s sshd opens for any other address or port, or of any other kind, is refused, whatever sshd root in the client runs. The forwarder’s master asks for no remote forward, and the link session is a connection of its own, so links don’t change what the ports note says.
Other users and VMs can’t use a link. Its end on the host is a unix socket of yours, mode 600 in a folder of mode 700, so no other user on the host reaches it, and neither does a VM that reaches the host’s loopback. Programs of yours on the host can connect to it, as they can already ssh into the target by the name devvm host shows.
A target’s service isn’t put on the host’s localhost. -W reaches the target’s loopback through its sshd, and the forwarder never forwards 127.77.0.0/16, so a link listens nowhere on the host.
Each VM’s own ssh rules hold. Host keys are pinned, each VM has its own key, and no agent is forwarded (ForwardAgent=no, IdentityAgent=none), as for the forwarder. devvm takes a client’s or a target’s ssh only as its backend gives it on the host’s loopback, 127.0.0.1, with a user and a key, and it refuses any other address, as the ssh config’s entries do.

Why not a shared network, the proxy, or a host port:

A network shared between VMs would bypass the proxy and the VMs’ confinement. On Lima it needs a root helper, and QEMU can’t make one.
CONNECT through the proxy as the only way in would work only for programs that use the VM’s proxy settings, and a raw TCP client in every VM would need a tool of devvm’s to make the connection. The proxy is a second entrance instead, for HTTP clients, beside the ssh forward that carries every link on every VM.
The target’s port forwarded to the host’s localhost is refused by the proxy’s floor, is reached by any open VM and any program on the host, and is a port that a host program can take first.

What the prototype showed. devvm’s prototype ran in the e2e sandbox, and nothing needed the link feature off:

macOS’s ssh 10.3p1 and Linux hosts’ ssh 9.6p1 and 10.2p1 accept -R 127.77.x.y:PORT:<unix socket>, and -O forward and -O cancel work on a master started with ClearAllForwardings=yes. A cancel keeps open connections, ending the master drops them, and a session keeps the PermitListen it connected with.
sshd 9.2p1 (Debian 12, --from), 10.2p1 (Ubuntu 26.04) and 10.3p1 (Alpine), with GatewayPorts clientspecified and PermitListen naming one address, bind that address alone and refuse 0.0.0.0, 127.0.0.1, ::1, localhost, *, other ports and port 0. The glob 127.77.*:* works the same way and matches the requested address as text.
All of them honor ip_unprivileged_port_start, which is 0 by default on Lima’s Ubuntu and Debian VMs, and 1024 on Lima’s Alpine VMs.
SIGHUP keeps sessions and open connections under systemd and OpenRC. Lima’s Ubuntu VMs (an sshd per connection over vsock) read the config at every connection. sshd’s rule is written once, so links don’t depend on either.
ssh -W through a kept master works over Lima’s ssh, and a new master after the target restarts reads its rewritten known hosts, while Lima keeps its key. Without a master, -S connects on its own unless ProxyCommand=/usr/bin/false.
glibc 2.36 and 2.43 and musl 1.2.6 resolve /etc/hosts names in 127.77.0.0/16, and Go, Node 18, 22 and 24, curl, Python and libpq 15 and 18 reach db and db.devvm.internal. HTTP clients honor NO_PROXY with db,.devvm.internal, Node 22 needs the leading dot, and Node 18’s fetch ignores proxies.
devvm’s port forwarder put a link’s listener on the host’s 127.0.0.1 and ::1, which is why it skips 127.77.0.0/16.
The prototype didn’t run confined VMs, so the end-to-end checks do.

What it relies on: OpenSSH’s client accepting forwarded channels only for listeners it asked for, and connecting each one only where it chose. It also relies on -O forward and -O cancel changing one master’s forwards, on ProxyCommand=/usr/bin/false keeping -S from connecting without its master, and on the host’s unix socket permissions. The pinned host keys are the ones the forwarder and ssh rely on. Lima’s rules must ignore every port ahead of its fallback rule, and devvm’s state on the host must hold each link’s target, ID and port. The proxy’s entrance relies on devvm’s run folder and the VM’s being yours alone, and on the proxy checking each entry’s socket against the one it names itself from the VM, the address and the port.

What could break it: an OpenSSH release whose client accepts a channel for a listener it didn’t ask for, which the end-to-end checks would see as a host connection devvm didn’t make. A Lima release that forwards a listener on a 127.77 address despite the rules would put a link on the host’s loopback, which the forwarder’s own skip doesn’t cover, and the end-to-end checks look for it on the host. A backend that started a target for a connection would break the rule that a connection never starts a VM, but devvm never asks a backend to start one then.

Not covered:

What the client does through the target’s service. A link joins the two VMs’ trust, as above, and what a VM does through a link lists what that leaves open.
Programs of yours on the host can connect to a link’s socket and the masters’ control sockets, as they can already reach the VMs.
Open connections outlive their link. devvm link rm and devvm link unpublish refuse the next connection, while those that are open stay until they close, the client stops or the target does. A target can’t get a tier lower than a client’s that links to it, because devvm network refuses it then, so no open connection breaks the tier rule.
A program in the client can take a link’s address and port first, and that link then fails with link_port_taken until the program lets it go. Root in the client can also edit its own /etc/hosts, which changes only what the client reaches.
A link name hides a name the client would otherwise resolve, such as db under its search domain. devvm refuses localhost, the client’s own hostname, another link’s name and anything that isn’t one DNS label.
A remote forward you ask for into a VM, with ssh -R through devvm’s ssh config, may bind an address in 127.77.0.0/16 in that VM, as remote forwards you add says. sshd matches the rule as text and then resolves what it matched, which is why a host name that begins with 127.77. binds whatever address it resolves to in the VM. devvm’s own forwards name addresses only.
Many connections: a client can keep 256 connections busy, which refuses its own further connections and makes the host run that many ssh processes.
The network rules don’t apply to links, through the proxy or not, so devvm network deny db doesn’t stop the link db. The published ports and the policy rule between the two VMs decide a link’s connections, and devvm link rm removes one.
A link’s short name before its first session lists it. Until devvm _links has written a client’s list once, the proxy doesn’t know db is a link, so a request for db:5432 meets the rules, and a rule such as ** sends it to whatever the host’s resolver or a company proxy finds by that name. After that, the list stays until devvm removes the VM, and NAME.devvm.internal is never the rules'.
Linux hosts are untested.

Threat check (the security-audit skill, guidance mode, its local IPC, data isolation and resource exhaustion companions). One row per lead, with what holds it and the end-to-end check that proves it:

Lead	What holds it	End-to-end check
A compromised client attacking the target through a link	Accepted, bounded by the rules: only a published port, only on the target’s loopback, and a client only to a target on its own tier or a stricter one	The link reaches the published port, while an unpublished port and a port that isn’t linked are refused, and after devvm link unpublish the next connection is refused
A client reaching a target on a lower tier	Refused when the link is made, at the client’s launch, on every connection, and by `devvm network` giving the target a lower tier than the client’s or the client a higher one than the target’s. For a client on a confined tier, on every connection the target’s tier must also be enforced as it runs, so a target given a confined tier while running, outside its proxy until its next start, or one whose self-check failed, is refused (`unenforced`)	An allowlist client and an open target are refused by devvm link add, at launch, after `devvm network open` on the target, and by `devvm network allowlist` on an open client linked to such a target. A target given the allowlist tier while it runs, which devvm ls shows as unenforced, has the link’s next connection refused and logged as `unenforced`, while its server runs, devvm link add and the client’s launch warn of it, and once restarted behind its proxy it’s reached again
Root in the client opening a channel to an address or port the host didn’t ask for	OpenSSH’s client refuses it and connects only to the link’s socket	With the client’s sshd loosened by root (below), no host listener or connection appears that devvm didn’t make, and only the link’s socket is reached
Root in the client loosening sshd (`PermitListen any`, `GatewayPorts yes`, a restart)	The host asks only for its links’ numeric `127.77` addresses, and what sshd binds in the client is the client’s own: root can already put a relay of its own on any of its addresses. With `GatewayPorts yes`, sshd binds each forward on every address of the client instead, so a second link on the same port can’t bind, and a client forwarding every port would forward it, as it would root’s relay. Nothing on the host changes	After root rewrites the client’s sshd config and restarts it, the link session asks only for the links’ addresses and the link still works, and with `PermitListen any` alone nothing listens on the client’s `0.0.0.0`, `127.0.0.1` or `::1` for it. With `GatewayPorts yes` too, the link session still listens on no host TCP port and connects only to the VMs’ sshd, and the check prints what the client binds
A VM making its proxy reach a socket that isn’t one of its own links, or another VM’s link	A request names a host and a port, never a path. The list is in the VM’s run folder (mode 700), which no VM can write, at a path devvm gives the proxy in its config, and the proxy reads it only as a regular file, not a symbolic link, that only you can write. An entry counts only when its socket is the one devvm names for this VM’s link at that address and port in devvm’s run folder (`proxy.LinkSocket`), and each proxy needs its own VM’s password	The client’s proxy refuses, by `GET` and by `CONNECT`, a name or port it hasn’t linked, a published port it hasn’t linked, `NAME.devvm.internal.devvm.internal`, `devvm.internal` and a link’s `127.77` address, and none of them reaches a link, while its own link answers. The target, on the allowlist tier and with no links, gets a refusal from its own proxy for the client’s link name. Unit tests (`internal/proxy/links_test.go`) refuse an entry naming another VM’s socket, a socket outside the sockets’ folder (one in the list’s own folder among them) or another port’s, no link without the sockets’ folder, a list that’s a symbolic link, writable by others, of another version or not JSON, and a host that is a socket’s path
An HTTP client in a confined client reaching a link added while it runs	The proxy reads the list for every request, before any rule, and NAME stays out of `NO_PROXY`	A link added to the running client is reached at once by curl through the VM’s proxy, by `GET` and by `CONNECT`, `NO_PROXY` holds only `localhost,127.0.0.1,::1`, and the client’s boot, its init’s start and its pinned host keys are the same after
The proxy changing a link’s bytes, or putting a secret into them	The link’s path tunnels the bytes as they are, before the SNI check, interception and secrets	None of its own end to end. Unit tests send a ClientHello for another host and a placeholder through a link whose name and port a secret’s hosts and an allow rule match, and the socket gets them as they were sent, and an absolute-form request reaches it without `Proxy-Authorization`
An honest VM’s sshd binding a link where it shouldn’t	`PermitListen 127.77.:` with `GatewayPorts clientspecified`, written once in every VM	`sshd -T` in every VM shows the settings, and `ssh -R` from inside the client to `0.0.0.0`, `127.0.0.1`, `::1` or another range fails
Another user’s program, or a VM, using a link	A unix socket, mode 600, in a folder of mode 700, and no host TCP port	The socket is mode 600 in a 700 folder, and no host TCP port carries the link
A link listener forwarded to the host’s localhost	The forwarder skips `127.77.0.0/16`, and Lima’s rules ignore every port	With the client forwarding every port, nothing from a link listens on the host, while a server of the client’s own on `127.0.0.1` with the same port is forwarded as usual (its control)
A link reached on the client’s localhost	Names only, numeric `127.77` addresses	Nothing from a link listens on the client’s `127.0.0.1` or `::1`, and a server of the client’s own there runs
A VM made again in the target’s folder inheriting a link	The target’s ID, recorded with the link	After devvm rm of the target, the client’s link is gone and the client is named, and a VM made again in that folder isn’t linked
Many connections	At most 256 at once, then refused and logged (`link_limit`), and an accept loop that takes a burst whole before it starts any relay and goes on accepting while the burst is carried or refused, so the socket’s backlog never refuses one unlogged	256 connections at once, past the socket’s backlog of 128 on macOS, are all carried, and a 257th is refused and `devvm network log` shows `link_limit`
A program in the client taking a link’s address first	Each link’s own `-O forward`, failing alone with `link_port_taken`	A server on a link’s address and port before the link is added: that link fails with `link_port_taken`, and another link of the client’s works
A malicious target’s answers	As any server’s: they reach only the connection that asked	None of its own: the target’s bytes go back over the same connection
Root in the client, or a malicious target, writing a line into the session’s log that reads as the session’s own, such as a false `link_port_taken` or a hidden real one	OpenSSH prints what the remote side sends, such as a disconnect’s reason, with its newlines, at the `LogLevel=ERROR` the session’s ssh uses. So `devvm _links` writes its ssh processes’ output to its log a whole line at a time, each line behind `ssh:`, so only the session’s own lines start with its prefix, which devvm link add and the launch read. A line in an error the session logs is indented, and the output of `ssh -O` is joined into one line	None of its own. A unit test (`internal/vmssh/links_test.go`) runs a stub ssh that prints a disconnect’s reason holding forged lines, one of them in pieces around a line of the session’s, and the log holds only the session’s own lines
A connection starting a stopped target	Never: `devvm _links` checks the target runs and never starts it	A connection to a stopped target fails, `devvm network log` shows `link_target_stopped`, and the target stays stopped. Restarted, it works again with its new host key
Lima’s own ssh asking for remote forwards, as every VM’s sshd allows them	Lima’s own ssh never asks for one, because devvm sets no reverse rule and its Lima home refuses settings it didn’t write. Every VM’s sshd allows remote forwards only on `127.77.:`, so no other address binds, and on macOS a confined VM also has the hostagent’s sandbox	Lima’s rules ignore every port, `sshd -T` shows `permitlisten 127.77.:`, and on a running Lima client, sshd listens on nothing but its own port and the link’s addresses
A link name shadowing a real host	One DNS label, never `localhost`, the client’s hostname or another link’s, plus `NAME.devvm.internal` under a reserved name	The CLI tests refuse each, and end to end, `--as NAME` reaches the target by that name alone
Links changing on a running client	`-O forward` and `-O cancel`, the setup over ssh, and the proxy’s list	devvm link add and devvm link rm on a running client apply within seconds, through the proxy too, without a restart
Two targets on one port	An address per target	Two targets publishing the same port are each reached by their own name
A link left behind	`devvm stop` and `devvm rm` end the process and its sockets, and `gc --orphans` finds leftovers	Removing either VM leaves no `devvm _links` process, socket or link state
Links on open and confined tiers	The same mechanism on every tier	The section runs with both VMs on the allowlist tier and both open, a port below 1024 such as 80 on Lima’s Alpine VMs, and `devvm network log` shows link events on both tiers

Groups

A group is a named set of VMs that reach each other’s published ports by name: you run devvm group join web in each VM’s folder, and every member gets a link to each port that every other member publishes. Groups are built on links alone. A group adds nothing below the link session, so every connection of a group’s link goes through the same session, sockets, proxy entrance and checks as a link made with devvm link add, and everything above holds for it.

The rules:

A VM is in one group at a time. devvm group join refuses a VM that’s in another group (group_other), naming devvm group leave, and devvm group leave takes no name.
One tier. Every member of a group is on the same tier. devvm group join refuses a VM whose tier differs from the members’ (group_tier_mismatch), and a group with a member whose network section can’t be read, and devvm network is refused for a member (group_tier_fixed), naming the group and devvm group leave. So every link a group makes keeps the links’ tier rule, and each of its connections is still checked against the states, with the target’s tier enforced as it runs, and warned of with link_target_unenforced when the group makes the link.
Membership is recorded on the host, in each member’s own state under ~/.local/share/devvm, as group and group_as in its links section, never in a project folder or a VM. A VM made again after devvm rm gets new state with a new ID, so it’s in no group, and the links to the old VM, which record its ID, don’t apply to it. devvm layer add, devvm layer rm and devvm update keep the VM’s state, so the VM stays in its group (a VM made again).
A group’s links are links, recorded in each member’s links state with a mark of their own, so devvm group leave removes only those, and devvm link rm refuses them (link_from_group) while they still apply. Every connection of one is also checked against membership: a group’s link applies only while its client and its target are in the same group (ungrouped otherwise), so a link that a leave cut short, or a race between two devvm commands, left behind carries nothing, and devvm link rm removes it. devvm group leave removes the leaver’s own group links before it takes it out of the group, and running devvm group join again works out every member’s links again. Each takes the client’s address for that member and the member’s name in the group: its folder part, or the name it joined with (--as NAME). devvm works out a member’s group links again whenever a member joins or leaves, publishes or unpublishes a port, or adds or removes a link of its own, and it records and applies only the members whose links changed, naming the links each gains or loses (group_links_added, group_links_removed). A running member gets them at once, as devvm link add gives them, and a stopped one at its next start.
The user’s links take precedence. A link made with devvm link add stays as it is: a port it links gets no group link, its name names that member’s group links too, and when its name is another member’s name in the group, that member’s group links aren’t made in that client, which is warned of (group_link_name_clash).
Names are unique in a group. devvm group join refuses a name another member has, as its name or its folder part (group_name_taken), and localhost (group_name_host). It also refuses a VM whose own hostname, its folder part, is another member’s name, whether that’s the member’s folder part or the name it joined with (group_name_hostname), because the name would resolve to the joining VM itself, whatever name it joins as. The refusal names the fix: the other member leaves and joins again with another name, or the project goes in a folder with another name. So no member’s name is another member’s hostname, and groupLinks makes no link by such a name, or by localhost, all the same.
Joins take turns. Each devvm group join and devvm group leave, each publish and unpublish that works the members’ links out again, and devvm rm of a VM hold a lock on the host, ~/.local/share/devvm/groups.lock (lockGroups), from reading the members’ states to writing them. A join checks the group once before it asks to widen a parent folder’s VM, and again once it holds the lock, with the VM’s state read again, so two VMs joining at once are checked one after the other, and the second sees the first as a member: the one tier and the names hold however the joins overlap. A devvm that waits for the lock says so (group_waiting) after 200 ms. The lock is an OS lock (flock) on an open file, which the system lets go of when the process holding it ends, however it ends.
devvm rm of a member takes it out of its group, after devvm rm’s removal of every link to it, and the group goes with its last member.

What a group adds to the link threat check. A group joins the trust of every member: code in any member can often run code in the others through the ports they publish, and then has their network, secrets and project folder. A member joining later gets that reach into every member, and they into it, without a command in their folders, so devvm group join says so on every join (group_trust). Nothing reaches more than the published ports on each member’s own loopback, and the one tier keeps every member’s tier whole.

Not covered:

Every member trusts every other, which what a VM does through a link lists.
A published range gives a link for each port in it, in every other member. A wide range means as many forwards in each member and as many sockets on the host, so publish only the ports you mean the group to reach.
devvm link add or devvm link rm at the same moment as a group’s change can each write the same member’s links whole, because those two don’t take the groups lock, so the later write wins. A group link left that way is refused as ungrouped, and the next change works the links out again from the states.

Threat check (the security-audit skill, guidance mode, its local IPC, data isolation and resource exhaustion companions), what a group adds to the links’ rows above:

Lead	What holds it	End-to-end check
A member reaching every other member’s published ports	Accepted, bounded by the links’ rules: only published ports, only on each member’s loopback, and said on every join (`group_trust`)	Three members reach each other’s published ports by name within seconds of the last join, and a port published by a member reaches the others at once
A member joining later	It needs `devvm group join` in its own folder, on the host; membership is in each member’s state on the host, which no VM writes. Every join says the group joins every member’s trust	The third VM’s join gives the other two links to its published port and it links to theirs, and `group_trust` is said
A member reaching one on a lower tier	One tier at `devvm group join`, checked again under the groups lock, `devvm network` refused for a member, and every connection checked as for any link	An allowlist VM is refused by an open group, and an open one by an allowlist group; `devvm network allowlist` for a member is refused. Unit tests (`TestGroupJoinsAtOnce`) and the CLI tests join an allowlist VM and an open one at once, and only one gets in, every time
A member’s name taking over a name the user chose	The user’s links take precedence, a name is unique in the group, `localhost` is refused, and so is a VM whose own hostname is a member’s name	The CLI tests refuse a taken name, another member’s folder part, `localhost`, and a VM whose folder part is a member’s folder part or the name it joined with, and drop a member’s group links whose name one of the user’s links has
Leaving or removing a member leaving links behind	`devvm group leave` and `devvm rm` remove the group’s links in every member, and the user’s stay	After `devvm group leave`, the leaver’s and the others’ group links are gone and their names don’t resolve, while a link made with `devvm link add` still answers
A group’s link outliving its group, from a leave cut short or a race	Each connection checks that the client and the target are in the same group (`ungrouped`), and a leave removes the leaver’s group links first	Unit tests (`TestCheckGroupLink`) refuse a group’s link once either VM is in another group or none, and keep the user’s own
A VM made again inheriting membership	Membership is in the VM’s own state, which `devvm rm` deletes, so a VM made after it starts in no group. `devvm layer add`, `devvm layer rm` and `devvm update` keep the state, because they make the same VM again (below)	The CLI tests: devvm rm of a member takes it out, and the last one’s leaving removes the group

A VM made again

devvm layer add and devvm layer rm make a VM again from a new image, and devvm update from its own image’s new build, while keeping devvm’s state for it and its home folder (making a VM again). All three use the one remake, and three choices shape it.

Option B, a new disk. The layer’s script never runs in the existing VM. Running it there would run a third-party installer next to the project’s files, as root, on a disk the VM’s own code has changed, and the image’s key would then describe a disk that no longer matches it. A new disk from the image keeps every VM a copy of an image whose key hashes everything in it. Taking a layer out works the same way, because a layer has no script that undoes it, and only a new disk from an image without the layer leaves nothing of it behind.

The old disk is kept until the new VM has started. devvm stops the VM and renames its machine to devvm-<hash>-old (backend.Backend’s Rename, limactl rename on Lima, which keeps the disk whole) before it makes the new one by the VM’s name, rather than deleting it first. So a failed create, start or home copy puts the old VM back as it was, and only a remake that got its new VM started with its home folder deletes the old disk. Every backend renames a stopped VM, so none deletes the old disk early. The aside name is no project VM’s name (vmname.IsVM), which keeps it out of devvm ls, the ssh config, the mounts devvm’s SSH agent checks and devvm space’s strays. It’s 24 characters for every VM, two more than the shortest name a project VM can have, so in a Lima home whose path leaves room for fewer, the rename is refused (lima_home_too_long) before anything else changes, and the VM stays as it was, stopped.

The VM keeps its ID. A link records its target’s id so that a different VM in the target’s folder, made after devvm rm, never inherits the link (links between VMs). A remake isn’t that case. The user runs it in the folder, on the host, and it keeps the VM’s name, its state, its folder and its project, so its links, its group and its published ports are the user’s choices for this project, and none of them is worth losing to a layer. A new ID would make every client’s link to the VM replaced, and the user would have to add each again in every client’s folder, which gives nothing a check needs: the clients reach the same project’s published ports, on a VM the same person made again from an image devvm built. What changes is the VM’s disk and its host keys. devvm reads the new host keys from the new disk’s system files, which the home folder’s copy never reaches, as for any VM that starts, and a link session’s master to the target ends when the target stops, so the next one reads the new keys. A remake of a confined VM keeps it, with the same proxy password and salt, so the links’ policy rule holds throughout.

The home folder’s copy

The old VM’s home folder is the VM’s own data, which nothing on the host trusts. devvm runs tar --one-file-system --ignore-failed-read in the old VM as its user, so the stream holds nothing from a mount, such as the project folder or an agent’s folder, and devvm reads it on the host with Go’s archive/tar and compress/gzip (copyHome). Nothing unpacks it on the host. Each entry is kept or dropped there, and only what’s kept is written, gzipped again, to home.tar.gz in the VM’s folder, 600 in folders that are 700:

Types: regular files, folders and symlinks, and a hard link only to a regular file kept before it. Devices, FIFOs and anything else are dropped, so the new VM’s tar creates none.
Paths: each is cleaned, and one that’s absolute, has .. above the home folder or holds a control character is dropped, and so is the home folder’s own entry, so its mode stays the new image’s. So is any entry below a symlink kept before it, which tar would follow out of the home folder.
Modes and owners: the setuid, setgid and sticky bits are dropped, and so are the entry’s owner and group, so the new VM’s tar makes everything its user’s.
What devvm writes itself: ~/.config/devvm (its environment, which holds the placeholders of the VM’s secrets), ~/.ssh/authorized_keys, the agents’ logins, the agents’ folders that devvm links to their mounts and the files it copies from the host, all of which the new VM’s setup writes again, and ~/.cache.

devvm puts the file into the new VM once it has started confined, as the VM’s user, with tar --skip-old-files, so a file or folder the new image put in the home folder, ~/.bashrc among them, stays the image’s, its metadata included, and the shell that runs tar there reads only the image’s files. devvm’s setup runs after it and writes its own files again. The file is deleted once the new disk has it, when the remake is undone, and with the VM’s folder.

Not covered:

The home folder is as trusted as the old VM was. What its code wrote there, such as a shell snippet in ~/.bashrc.d, a ~/.ssh/config entry or an agent’s settings, runs in the new VM as it would have in the old one. The new VM is the same project’s VM, with the same reach, so this is no new access, and --clean-home starts from the image’s home folder instead.
Its size isn’t bounded. A VM can make its home folder as large as its disk, and the copy then takes as much room on the host, besides the old disk and the new one. A VM can already fill the host’s disk through the project folder, which it writes, so this adds no new way to do it, and devvm checks for a VM’s free space before the remake.
It’s on the host’s disk for the remake’s minutes, with whatever the VM keeps in its home, such as keys or logins it made itself. The VM’s disk holds the same data in devvm’s Lima home, with the same permissions, so a backup of the data folder sees it either way.

Threat check (the security-audit skill, guidance mode, with its data isolation and lifecycle and resource exhaustion companions):

Lead	What holds it	Check
A hostile home folder writing outside the new VM’s home through `..`, an absolute path or a symlink it planted	`copyHome` drops each, on the host, before the new VM’s `tar` sees the stream, and that `tar` runs as the VM’s user	Unit tests (`TestCopyHome`) keep files, folders, symlinks and hard links to kept files, and drop `..`, absolute paths, control characters, devices, FIFOs, setuid bits, owners, hard links out of the home folder and entries below a symlink
A hostile stream attacking the host	It’s parsed by memory-safe stdlib readers and never unpacked on the host, and it’s written only into the VM’s own folder, which only devvm writes	`TestCopyHome` refuses a stream that isn’t gzip
devvm’s own files or the user’s secrets carried across	`~/.config/devvm`, `~/.ssh/authorized_keys`, the agents’ logins and the files and folders devvm copies or links from the host are left out, and the new VM’s setup writes them again	`TestCopyHome` leaves them out, and the CLI tests check that a saved home holds `.bash_history` without `~/.config/devvm`
The old VM’s code influencing the new VM’s setup or the restore itself	The restore keeps what the image put in the home folder, its folders’ metadata included (`--skip-old-files`), so the shell that runs it reads the image’s `~/.bashrc`, and devvm’s setup runs after it	The CLI tests check the restore’s command and order; e2e checks that a file in `~` survives a remake and that the VM opens a shell afterwards
A remake halfway losing the VM	The old machine is kept aside until the new one has started with its home folder, the cleanup puts it back on any failure or interrupt before then and never deletes the new one after, and a killed devvm leaves the `remake` section, which the next devvm finishes, while `devvm gc --orphans` skips the VM and `devvm rm` deletes both machines. The saved home folder goes only after the section does	Unit tests (`TestRemakeUndo`, `TestRemakePending`, `TestRemakeDone`, `TestRemakePendingChecked`) and the CLI tests: a failed create and an interrupt put the old VM back as it was, a remake left with its old machine aside or gone is finished by the next devvm, checked again first, and one whose old machine wasn’t deleted keeps its new machine
Two devvms on one remake	The remake lock (`run/remake.lock`), held from finding the VM until the remake ends, with the state read again under it: a second devvm refuses (`remake_running`) rather than finishing a remake that’s still running	Unit test (`TestRemakeHeld`)
Links reaching a different VM	The VM keeps its ID only when its state is kept, by the user’s own remake in its folder, and a link session reads the new host keys once the target starts again	Unit tests (`TestAddLayers`) check every state key but the image survives, the ID included, and e2e’s `project` section checks that a VM made again keeps its settings
A parent folder’s VM made again from a subfolder	Asked only in a terminal (`remake_parent_vm_ask`), which `--force` never answers, and refused without one	The CLI tests refuse it without a terminal and keep it when the answer is no

opencode’s database in history exports

opencode 2 keeps its sessions and its logins in one SQLite database, ~/.local/share/opencode/opencode.db (opencode-<channel>.db for another channel’s build), in WAL mode. An export of agents’ history goes to this machine and can go into another VM of the folder, so it must carry the sessions without the logins. Leaving the database out would lose opencode’s history, and opencode’s own opencode session export runs opencode and its server once per session, which an export before devvm rm can’t count on. devvm copies the database in the VM with the sqlite3 that base installs (sqlite3 on Debian and Ubuntu, sqlite on Alpine), as the VM’s user, in a temporary folder in ~/.cache (opencodeClean):

VACUUM INTO copies the database as one snapshot, with what its -wal file holds, so a login that opencode saved last, which is still only in the WAL, is in the copy and taken out with the rest.
In the copy, devvm drops every trigger, so none can put a row back, and empties the tables that hold logins: opencode 2.2’s credential (its logins and API keys), account and control_account (its console’s access and refresh tokens) and account_state, the session_share table of an earlier opencode (each share’s secret), and any other table whose name has credential, account, auth, secret, token or share in it, or that has a column such as secret, api_key, access_token or password, so a table that a later opencode adds for its logins is emptied too. It also deletes the rows of the kv table whose key starts with wellknown:config:, because each holds an organization’s remote config, which opencode fetched with that organization’s well-known login and which can hold the organization’s keys. opencode fetches the config again once it has the login.
It takes out the record of opencode’s migration that imports auth.json (20260805200742_import_legacy_credentials), so the opencode that next opens the copy imports the logins that devvm’s setup wrote in its own VM’s auth.json, as it does when it first makes its database.
It turns WAL off, with secure_delete on, and runs VACUUM, which writes the file again, so no free page keeps a deleted row and no -wal or -journal file is left.
It counts the rows of each table it emptied and the triggers, and when any is left, or any step failed, the database is left out of the export and the run log says why.

The export’s tar leaves out the database and its -wal, -shm and -journal files as they are, matched within opencode’s folder, and adds the copy at the database’s path (--transform, which leaves symlinks’ targets alone). The import does the same to an export it unpacks, in the VM, before anything reaches the home folder: it deletes the -wal, -shm and -journal files and any database that isn’t a regular file, and puts in each database’s copy, or deletes the database when the VM has no sqlite3 or the copy fails. Each sqlite3 runs with trusted_schema off, so an imported database’s schema can’t call the CLI’s functions that write files.

Not covered:

A secret that’s in a session itself, such as a key you pasted into a prompt, stays in the session, as it does in Claude Code’s and Codex’s transcripts.
A plugin’s own storage and a remote workspace’s binding stay. opencode keeps each plugin’s storage in its kv table, under the plugin’s own prefix, and each remote workspace’s binding, the record its provider needs to reconnect, in its workspace table. opencode’s built-in plugins keep model catalogs and session records there, but a plugin you install, or a workspace provider, that keeps a token there takes it into the export.
A VM without sqlite3, made from an image built without it, exports and imports no opencode database, and opencode’s sessions are lost from that export.
A remake’s home folder copy keeps opencode’s database with its logins, because it’s the same VM’s data going into the same folder’s new VM (the home folder’s copy).

Threat check (the security-audit skill, guidance mode, with its data isolation companion):

Lead	What holds it	Check
A key typed into opencode in the VM reaching this machine’s export	The export holds only the copy, whose login tables are empty and whose file is written again by `VACUUM`	Unit tests (`TestOpencodeClean`, `TestHistoryExportRuns`) plant a credential, an account’s tokens, a share’s secret and a later table’s key, the credential only in the WAL of an open connection, and find none of them in any byte of the copy or the export; e2e’s `network` section plants a key with the VM’s `sqlite3` and finds neither it nor the login’s placeholders in the export
The WAL or journal file carrying the rows	They never go into an export, and an import deletes them	`TestHistoryExportRuns` and `TestHistoryImportCleansOpencode`
An export with the logins in it reaching another VM	The import makes the same copy in the VM before anything reaches its home folder, or leaves the database out	`TestHistoryImportCleansOpencode`, with and without `sqlite3`
A trigger putting a row back as it’s deleted	Every trigger is dropped first, and the rows are counted after	`TestOpencodeClean` plants one
A hostile database’s schema running code in the VM	`trusted_schema` is off, and it runs as the VM’s user on the VM’s own files, which the export’s code could reach anyway	Reviewed

Linux hosts: overlays instead of copies

Lima on QEMU keeps one qcow2 disk per VM, and a clone copies it (2.0 GB for base). A project VM’s disk is a qcow2 overlay of its image’s instead (details): 196 KB new, 12 MB after booting, the same speed except fsync writes (1.8–2.5 s vs 1.2–2.1 s per GiB). Flattening one (qemu-img rebase -b "") took 8 s, if it’s ever needed.

One binary

devvm’s own files (assets/ in the repo: base.yaml, layers/, features/, shared-tools/, guest/, oci/) are embedded in the binary (assets.FS), so devvm installs, moves and upgrades as one file and can’t find a mismatched copy beside it. Code reads them from that file system; scripts reach VMs on their standard input, never by path or file mode. The binary is the only copy: no shared folder on disk holds devvm’s files, because a copy there could be edited and still be used. A VM’s proxy is devvm itself, so it reads its built-in policies from the binary (its config’s builtin_policies, by name), once, and its policies carry only your files. limactl create reads base.yaml on its standard input (-). Only --from’s build on Lima needs a path, because its build VM mounts the conversion’s scripts from oci/ read-only (root there reads them), and none of the folder’s other files, with guest/ca.sh beside it when you have company CAs. It writes them into its own temporary folder, devvm-oci.<random>, beside vm/ and never inside it, files 600 and folders 700, and the build’s cleanup deletes the folder however the build ends. Image keys hash the embedded files, so what goes into an image is always what its key describes.

devvm’s own Lima home

Lima merges _config/default.yaml and override.yaml of its home into every VM. In ~/.lima, a networks: [{vzNAT: true}] there gave a confined VM a second network card with a default route around its proxy (the self-check caught it), and an extra mount went unnoticed. Hence a Lima home of devvm’s own, which refuses those files and any network devvm didn’t give a VM (details). The costs: plain limactl list doesn’t show devvm’s VMs, and VM names are shorter, as Lima’s ssh socket path is at most 104 characters.

The state layout

devvm keeps its state in plain files in two folders, which State files describes. The layout follows a few rules:

One folder per VM, keyed by identity. A VM’s folder, vms/<hash>/, is named by the hash that ends the VM’s name, the hash of its project folder’s physical path when devvm made it. The VM’s name, its backend and its ssh alias are chosen once, when devvm makes the VM, and recorded in its state, so devvm finds a folder’s VM by the state that records the folder, first by the path’s hash, then reads the name and the backend from the state. A folder that moves keeps its VM, whose state records the new path while its name and folder stay (a moved folder). A backend’s limit on names, or another VM taking the same folder name, never changes which VM a folder finds. Every file devvm keeps for a VM is in its folder, so deleting a VM stops what runs for it, removes what it has outside the folder, and then deletes the folder whole, and devvm space and devvm gc count folders. A list of a VM’s files, kept in each place that deletes or counts them, would drift, and one of those places would miss a file. Only sockets stay outside, in run/ and named by hashes, because a socket’s path is at most 104 characters.
The folder says who writes a file. ~/.config/devvm holds what you write, and devvm never rewrites a file of yours there: it changes single lines of policy and keeps the rest, comments included, and it writes the folder’s version. Everything devvm writes is in ~/.local/share/devvm. A default comes from a flag, then config.toml, then settings.json, then devvm’s own, and a command that would save a default that config.toml sets refuses, naming the file and the key, because the saved value would never apply.
Two formats. What you edit is TOML, read strictly, so a typo stops the command and devvm names the file and the line, rather than the key being ignored. What only devvm writes is JSON, through encoding/json, and devvm ignores fields it doesn’t know, so a newer devvm’s additions don’t stop an older one. A change that an older devvm would read wrongly goes through the layout version instead.
One section per package. vm.json has the core fields every VM has, and each package that keeps something of a VM’s owns one section of it, with its own type, so internal/state imports none of them. Each section keeps what you asked for apart from what devvm made, which sits under made, so a value is never stored twice under two names.
The lock rule. Each change to a VM’s state holds that VM’s lock, reads the file again, and writes it once, so a command’s changes land together, and reads take no lock, because each write renames a whole file over the old one. A command holds at most one VM’s lock at a time. A change across VMs, such as removing every link to a VM being deleted, lets go of the first VM’s lock, then takes each other VM’s lock in turn and reads its state again before writing it. The claims lock is taken only after a VM’s own lock, never before, so no two devvms can each wait for the lock the other holds.
A layout version. Each folder records its layout version in version. A newer devvm that changes the layout raises it and adds a migration from the version before, an older layout is migrated before a command runs, and a newer one is refused, because this devvm would read it wrongly. testdata/script/layout.txtar lists every file and folder a confined VM, a port, a link and a secret leaves, so a change to the layout changes a listing and fails the test until someone decides whether it needs a new version (changing the state layout).
devvm’s own files stay in the binary, as one binary describes, so neither folder holds a copy of assets/.

Why not a database. SQLite or bbolt would give transactions, but either one is a new dependency, and devvm builds without cgo, so SQLite would mean a large pure-Go port. State would also stop being files that you can read with cat or jq, back up, and check by hand. Files written whole and renamed into place, with an OS file lock for each VM, give what devvm needs: one writer to a VM at a time, and readers that never see half a file.

Failed attempts

Claude’s host login in VMs: on macOS it’s in the Keychain, which can’t be shared. VMs get a login made in a VM instead, which the VM’s proxy captures. The first token saved by hand was the code the browser shows, so devvm secret set claude refuses one without the sk-ant-oat01- prefix.
krunkit (Lima on libkrun) to return freed memory: it didn’t visibly return any.
Debian’s cloud kernel for --from: on arm64 it has no gpio_keys, so it ignored Lima’s stop request; Lima hard-killed the VM after 30 s, and node, bun and dockerd in the image came out corrupted. On x86_64 it has no 9p, so on a Linux host every mount of the VM’s was an empty folder: Lima mounts them with nofail, and the shared tools’ build failed at its first cd. --from uses the generic kernel on both. With it, a Debian VM on QEMU first booted with its network card named enp0s4, which cloud-init couldn’t rename to Lima’s eth0 once the card was up, so the VM got no ssh host keys. Debian’s kernel command line has net.ifnames=0, which keeps the name eth0 from the start.
Every layer as a Docker build stage, flattened: it worked, but each image became its own full disk (3.7 GB for node).
Shared read-only layers: each Docker layer as an EROFS file, stacked as / with overlayfs over each VM’s own disk by an initramfs of devvm’s own. It worked (node took 1.0 GB of layers instead of a 1.8 GB disk), but devvm owned the boot path (the initramfs, an exit initramfs for clean shutdowns, kernel extraction), and cold program starts were 10–50 ms slower. Shared tools, and overlays on Linux, give the same savings without it.
A read-only mount for .git/hooks: root in the VM can unmount it and write hooks through the project mount underneath. Locking .git itself (uappnd) breaks git, and a locked placeholder .git/commondir makes git fail. Hence chflags uchg on hooks and config, plus detection (details).
Kernel extraction trying gzip first: Ubuntu’s arm64 kernel is an EFI executable with a zstd payload.
mke2fs -d from a tarball: it needed libarchive and a UTF-8 locale. A directory works.

Things learned

Lima vz never returns freed guest memory to the Mac. sbx returns it with a VM monitor of its own; libkrun has MADV_FREE, with a stronger fix still an open PR.
APFS clones cost nothing until written, and du overcounts them: real use was 5.5 GB where du said 12.7 GB.
Lima locks each additional disk to one running VM, so images shared across VMs go through a shared folder (virtiofs on macOS, 9p on Linux) and are loop-mounted from it.
Read-only mounts are enforced by the host: even root in the VM can’t write to them, but root can unmount a nested one and reach what’s underneath.
Codex and opencode refreshing their OAuth logins in a VM can rotate the token and sign the host out. On every tier the VM holds only placeholders, and devvm refreshes the logins on the host when the proxy asks it for their values (internal/agentlogin). Codex decodes id_token’s claims and the access token’s exp without checking signatures, so placeholders are unsigned JWTs with the real account claims and a far-off exp.
A process passes the files it was given on to the processes it starts, unless they’re marked close-on-exec. Go marks only the files it opens itself, so a terminal, an editor or a test harness that gives devvm another one (a pty on descriptor 3, the end of a pipe) would have it held by devvm’s proxy, its port forwarder and Lima’s processes, which keep that terminal or pipe open for as long as they run. So devvm marks every file it was given past standard input, output and error close-on-exec as it starts, and keeps them open itself.
No existing tool gives Lima Dockerfile-style layers with sharing at run time; nerdbox and Kata do it on VM runtimes of their own.
Where devvm fits, next to sbx and Colima: sbx puts agents first; devvm gives a full Linux VM per project, with agents optional.

Open work and risks

User-facing limits are in Known limits, and what devvm leaves open is in Attack vectors.

--from: a moved tag is noticed only at the next build, not when listing images; devvm doesn’t log in to registries, so private images work only through Docker; registries over plain HTTP (localhost:5000) aren’t supported.
Alpine: rc-status as a normal user shows acpid and syslog as crashed, as their pid files are root’s (sudo rc-status is clean).
Free-space floors: the free space devvm wants before each build and new VM (free space) is set by Lima’s own package, and for shared tool images in internal/backend/disk.go, where the comments give what each number was measured from. The measurements are free space, read with statfs before, during and after each step, in e2e runs on macOS with Lima vz, and for shared tool images also with devvm’s data folder on a tmpfs of its own on Linux. On Linux hosts, Lima’s numbers add a copy of base’s disk to an image’s and a tools build’s, but they haven’t been measured there yet. A --from base converts its image in the host’s temporary folder, which devvm doesn’t check, because on Linux that’s often a small tmpfs.
x86_64 hosts are untested on real hardware; on Intel Macs, --from’s kernel is untested with vz.
Linux hosts: the QEMU wrapper depends on the shape of Lima’s -netdev user,id=net0,… argument. If Lima changes it, the wrapper refuses to start QEMU rather than run it unrestricted.
Network tiers, not built:
- tokens for services devvm doesn’t know: a host, a header, a format and an environment variable;
- method and path rules on decrypted hosts, such as read-only GitHub, or no git-receive-pack unless approved;
- approving blocked hosts as they’re hit.
Claude Code’s login is captured on a path Anthropic doesn’t document. devvm reads Claude Code’s token endpoint, its client ID, its requests and replies and the .credentials.json format from Claude Code 2.1’s bundle, and refreshes the login on the host as Claude Code would (logins made in a VM). A release that changes any of them can break the login in VMs, or the host’s refresh, without warning. The network end-to-end section drives the real Claude Code’s claude auth login against a fake endpoint, so a change to its requests fails that section, and a Claude Code token from claude setup-token, which Anthropic documents, stays the fallback.
Network tiers, risks of the mechanism:
- sandbox-exec is deprecated. If Apple removes it, macOS needs another way to confine the hostagent: an egress filter upstream (gvisor-tap-vsock#609 and #631; Lima’s hostLoopback: false, lima#5581), a user-v2 daemon of devvm’s own (lima#5464), or QEMU with the Linux mechanism on macOS too (it works there, slower than vz). The self-check catches it breaking silently.
- VMs started outside devvm (limactl start, limactl shell starting a stopped VM, Lima’s start at login) aren’t confined: devvm reports them unenforced. An override.yaml setting a dead HTTPS_PROXY might harden this (untested).
- A Lima upgrade that adds a hostagent connection fails closed: the start hangs until Lima gives up (10 minutes in the experiment). devvm should time-box the start and say it’s the sandbox. That includes unix sockets: the profile allows only those in the VM’s Lima folder and the forwarded agent’s. Allowing every unix socket would let a compromised hostagent reach docker.sock or another ssh agent.
- Tunnels the proxy doesn’t read are decided by the CONNECT host and checked against the TLS server name, so domain fronting through a shared CDN may still reach other hosts (attack vectors).

Last updated on November 6, 2026

Lima