# Virtual Machines reference **Page:** kit/vm/reference [Download Raw Markdown](./kit/vm/reference.md) --- # Virtual Machines reference Lookup tables for [Virtual Machines](/kit/vm/). Every route, field and response is in the [VM API reference](/api/vm/). One VM service runs per computer, so its URL label is normally `vm-1`. ## Hardware fixed at create | Field | Values | |---|---| | `firmware` | `uefi` (default) or `bios` | | `secure_boot`, `tpm` | Secure Boot and a TPM | | `bus` | System disk bus: `virtio` or `sata` | | `cpu` | CPU model | | `agent`, `console` | Guest agent channel and serial console, both on by default | | `audio` | Sound output | | `share` | The shared folder | | `os` | Preset: `windows-11`, `windows-server` or `linux`. `windows-11` turns on UEFI, Secure Boot and a TPM. A Windows preset picks `virtio`, or `sata` on a host without the virtio-win driver ISO. | A field you set explicitly wins over the preset. `GET /api/v1/vm/capabilities` says which of UEFI, TPM, Secure Boot, audio, the shared folder and templates new VMs can use, plus the memory ceiling, memory running VMs already hold and free storage. ## Reconfigure a stopped VM `PATCH /api/v1/vm/vms/{name}` (`hoody vm update`) changes a stopped VM for its next start: memory, vCPUs, disk size, display, attached CD-ROMs, autostart, disk throughput limits, the shared-folder quota, the stop timeout (`stop_grace_seconds`, 5 to 900 seconds, default 30) and port forwards. Disk size only grows, and is sent alone. `ports` replaces the whole list. Every field is in the [VMs API](/api/vm/vms/#reconfigure-a-stopped-vm). ## Refused starts `POST /api/v1/vm/vms/{name}/start` answers `200` even when the start is refused. Only these come back as an error status: | Status | Cause | |---|---| | `400` | Bad name or query string | | `404` | Unknown VM | | `409` | Wrong state, another operation in progress, or an orphaned VM | | `507 InsufficientSpace` | Shared folder over its quota | Every other refusal is in the VM's `error` field: no KVM device, not enough memory, a KVM resource limit, TPM, audio and display checks, and failures of the launch itself. The field is a whole error envelope, so the code is at `error.error.code`. 1. `error` set in the `200` body: the start was refused, even while `state` reads `Starting`. 2. Otherwise poll `GET /api/v1/vm/vms/{name}` until `Running` or `Failed`. A failure after the launch began shows as `state: Failed` with the same nested `error` and a prose `reason`. 3. `Failed` with no `error` carries only `reason`: a refusal with no code, or a guest that reached `Running` and later exited. ```json { "name": "dev-vm", "state": "Starting", "error": { "error": { "code": "KvmUnavailable", "message": "...", "retryable": false } } } ``` ## States A VM is `Creating`, `Stopped`, `Starting`, `Running`, `Paused`, `Stopping`, `Failed` or `Orphaned`. - `stop` presses the guest's power button and forces it off only if it has not stopped after the configured timeout (`stop_grace_seconds`, default 30 seconds), unless the stop request overrides it with `grace_seconds`. The VM reads `Stopping` until its process exits. - `force stop` skips the guest, which can lose unsaved data, like pulling the power cord. - `pause` freezes the CPUs and frees nothing. Stop a VM to release its memory. - `autostart` starts a VM when the VM service starts, the only thing that brings it back after the computer restarts. ## Displays ### Finding the address - `GET /api/v1/vm/vms/{name}/display` on the `vm` URL redirects to the display's address; `?format=json` returns the name, display number and address. `GET /api/v1/vm/vms/{name}` has it as `display_url`. - Both build the address from the hostname the request arrived on, so call them through the `vm` URL. A proxy alias or custom domain that targets the `vm` service returns the same address. Through any other service's hostname, and for display 0, there is no address. - A headless VM is refused with `409 DisplayNotConfigured`. The route only reads: it starts neither the VM nor its display. ### Numbers | Rule | Detail | |---|---| | `"auto"` | The lowest free number from 7001 to 8000 (the default range). Free: no other VM has it, no terminal session uses it and nothing draws on it. Named in the create response's `display`, then in `spec.display`. `PATCH` accepts it too. | | Keeping a number | A VM already in the range keeps its number while it is still usable by this VM: nothing else runs on it or what runs is this VM's own display, no other VM is set to it, and, when nothing runs, no terminal session holds it. Otherwise it gets the lowest free number. | | Range full | `409 DisplayRangeExhausted` | | Explicit number in use | `409 DisplayInUse` on create and update. At start, HTTP `200` with `error.error.code: "DisplayInUse"`, since the number can be taken in between. This includes a number another VM is set to, running or not, and a display started for another VM; the message names that VM. | | Out of range | Outside 1 to 9999: `400 SpecInvalid`, since no display can be started for it. | | `display_id` | Another name for `display`. Sending both with different values is `400 SpecInvalid`. | | Headless | Without `display`. The service never picks a number unasked. `PATCH` can add one to a stopped VM later. | A VM never runs on a display something else uses: the desktop on `:1`, a terminal's or browser's display, or another VM's. An existing VM on such a display, or on one an older VM service started without recording the terminal session, is refused at every start until its display changes, with `DisplayInUse` in `error`; for the older-service case the message says the display was started for this VM. Stop it, `PATCH` its display to `"auto"` or a free number, and start it again. ### How a VM's display runs - When nothing runs on the display yet, the start creates a [terminal](/kit/terminals/) session with the same number (terminal 7001 for display 7001) and records it for this VM. The terminal service creates it only if nothing uses that display. - An already running display is used only if the terminal service still reports that same session, started for this VM, as on a restart. Before every start the service connects to the display the way the VM will, and starts only if that works and the session is unchanged. - A display used by something else gives `DisplayInUse` in `error`. Any other failure, or settings changed during the start, gives `DisplayUnavailable`, whose message says which step failed and what to do. - The window has no frame, fills the display and follows its size. The window manager takes no keys, so Alt+F4, Alt+Tab and Ctrl+Alt with an arrow key reach the VM. A left click gives the VM the keyboard. Closing the window does not stop the VM. - New VMs get the `virtio-vga` adapter (`spec.display_adapter`), so a guest with a virtio-gpu driver resizes to fill the display: built into Linux, on the virtio-win disc for Windows. Older VMs keep `vga`, scaled to fill the display at the resolution the guest chose. The adapter is fixed at create; an export does not record it, so an imported VM gets `virtio-vga`. ### Lifetime - The display keeps running after the VM stops, and the next start reuses it. Which VM it belongs to survives a VM service restart and is cleared, with the display, when the computer restarts. Stopping a display while its VM runs stops the VM. - Deleting a VM stops the display created for it and deletes that terminal session, freeing the number. If terminal N still has the recorded session, the stop is the same as `POST /api/v1/system/displays/{N}/stop` ([Displays](/kit/displays/)), and also stops an X server someone else started on N meanwhile. The session is checked again just before any signal, so a replaced session keeps its display. - If terminal N was replaced or deleted, the stop did not complete (for example a screenshot held the display), or the terminal service was unreachable, the record stays and nothing more is sent. What already happened stands: a display can be stopped with its session left, and a stop whose answer was lost can still finish. - A number stays taken while a display runs on it or a terminal session holds it; the record alone does not. To free it, stop display N and delete terminal N if it exists. No other terminal session is ever deleted. ## Uploading in parts Declare the image by SHA-256 and total size, send the parts in order, then finalize. Every part but the last is exactly the `part_size` the begin answer gives. A replayed part is refused rather than applied twice, so retrying an unanswered request is safe. Finalize hashes the stored bytes and publishes the image if they match. ```bash SHA=$(sha256sum installer.iso | cut -d' ' -f1) BEGIN=$(hoody -o json vm isos create "$SHA" --bytes "$(stat -c %s installer.iso)") if [ "$(echo "$BEGIN" | jq -r '.phase')" != ready ]; then # ready: already published, nothing to send PART_SIZE=$(echo "$BEGIN" | jq -r '.part_size') NEXT=$(echo "$BEGIN" | jq -r '.next_part // 9223372036854775807') # 0 for a new upload, the resume point otherwise; none or past the last part once every byte is stored split -b "$PART_SIZE" -d -a 5 installer.iso part- i=0 for f in part-*; do if [ "$i" -ge "$NEXT" ]; then hoody vm isos upload "$SHA" --part "$i" --input "$f" \ || { echo "part $i failed; hoody vm isos get $SHA gives next_part" >&2; exit 1; } fi i=$((i+1)) done hoody vm isos finalize "$SHA" fi ``` This is a part upload, shown in the CLI only. The SDK calls are `box.vm.isos.create`, `uploadPart` and `finalize`. An interrupted upload resumes from `next_part` in `hoody vm isos get`. Unfinished uploads keep their space and are never removed automatically; published images keep theirs until you delete them. An image attached to any VM cannot be deleted. An `iso` source with a `url` is refused: download the image into the store first. ## Snapshots and imports - A restore keeps running if your client stops waiting, and nothing reads its outcome afterwards: only a `snapshot` event with `action: "restored"`, on a stream opened before the restore call, reports it. - The export answer gives the artifact's `bytes`, `part_size`, number of `parts` and `sha256`. Parts can be fetched in any order, and again. Join them in order, check against `sha256`, then release the export. - The import store takes the artifact with the same begin, upload and finalize steps as an image. Create from it with a `snapshot-artifact` source. - The artifact supplies the disk, firmware, TPM and other hardware, but not `memory_mib`, `vcpus` and `bus`, which are required. Give the original `bus`: a guest installed on one does not boot from the other. - The artifact does not include shared-folder files. The imported VM starts with an empty shared folder; copy those files separately and verify the destination before deleting the original VM. - An artifact with TPM state is refused unless the create sets `duplicate_tpm_identity` (`--source-duplicate-tpm-identity`). The copy then holds the same TPM secrets as the original. A `vm-disk` clone gets a fresh TPM. - Plan for twice the artifact's size: it and the rebuilt VM exist together. It stays after the create, so it can build more VMs, and takes space until deleted. ## Console, logs and guest agent | Call | Detail | |---|---| | `GET .../console` | Up to 65536 bytes of raw output per read. `x-hoody-console-cursor` continues where it stopped; `x-hoody-console-gap` says whether output was lost since the last read. On by default. | | `POST .../console` | 1 to 4096 bytes typed into the serial input exactly as sent. Requires `console` and `console_input`, both fixed at create; otherwise returns `409 BadState`. | | `GET .../log` (`hoody vm logs get NAME --source S --tail N`) | The last lines of the current run, or the latest if stopped. `source`: `console` (escape sequences and control characters removed), `qemu`, or the helpers `passt` (network), `swtpm` (TPM) and `virtiofsd` (shared folder). Read a helper when `sidecars` reports it `degraded`. | | `GET .../agent` (`hoody vm guestagent get NAME`) | The QEMU guest agent's version and commands. | | `GET .../agent/inventory` (`hoody vm guestagent inventory get NAME`) | The guest's OS, hostname, network interfaces with IP addresses, filesystems and logged-in users. | For the agent calls, `409` is normal for a VM that is stopped, still booting or has no agent. What the agent reports comes from the guest and is not verified. ## Event streams | | `GET /api/v1/vm/vms/{name}/events` | `GET /api/v1/vm/events` | |---|---|---| | Follows | One VM | Every VM, including ones created later | | First frame | `state` with the current state, sent even when filtered out | None: read `hoody vm list` | | `deleted` | Ends the stream | Does not end it | | `resync` | Carries the VM's detail in `data.detail`, or `data.error` and `data.hint` when the state could not be read: re-read the VM | `seq: 0`, no `name`, `scope: "stream"`: re-read the list | - Kinds: `state`, `progress`, `sidecar`, `qmp-error`, `snapshot`, `share-quota` and `monitor-stalled`, plus `deleted` and `resync`, always sent. - Frames carry no `id:` and nothing is replayed. A comment line every 15 seconds keeps the connection open. - A slow reader loses the oldest events and normally gets `resync` with the number lost. On the per-VM stream, if the VM was deleted or its name now belongs to another VM, a final `deleted` frame carries `data.lost` and ends the stream instead. Otherwise the `resync` frame's `seq` is the detail's `seq`; when the detail has none (during create), or the frame carries `data.error`, it repeats the last known value, which proves nothing. Read the VM again and process every `resync`. ### A view of every VM 1. Open `GET /api/v1/vm/events`, then read the VM list. The view holds what the list returned. 2. For any frame naming a VM, `deleted` included, read it. On `200` replace its row, `vm_id` included; on `404` remove it; on another error keep the row, mark it stale and read again. 3. Keep at most one read in flight per VM. If another frame for it arrives meanwhile, read once more afterwards. 4. On a `resync` with no `name`, read the whole list again. 5. Number each read of a name and each list refresh. Apply a read only if no newer read of that name and no list refresh started after it. During a refresh, note names that frames arrive for and read them again after installing the list. A `resync` during a refresh starts it over. A `state` frame can update the screen at once, but the read is what the view keeps: other kinds carry no detail, and a sampled value such as the shared-folder measurement can be older in a frame. A replacement VM holds the name, so a late `deleted` for an earlier VM of that name cannot remove it. Reads triggered by events do not refresh sampled values such as `share`: poll the VM when they must be fresh, and show `share.sampled_unix`. ## Recovery - `Orphaned`: the service cannot verify the VM's run, and the guest may still be running. Read `reason`, check the guest process yourself, then `hoody vm force adopt` resumes managing it or `hoody vm force markdead` records it dead. - `hoody vm unidentifiedruns list` shows memory held by runs no VM name accounts for; `hoody vm unidentifiedruns force markdead` clears one. See the [recovery API](/api/vm/recovery/). ## Health and metrics `GET /api/v1/vm/health` (`hoody vm health`) reports the service up with its build and start time, memory use, open file descriptors and process ID. `GET /api/v1/vm/metrics` (`hoody vm metrics`) returns Prometheus text, also at `/metrics`. `hoody_vm_vms_by_state{state}` has a series per state; the other families count starts, stops, failures, admitted guest memory and shared-folder usage. ## Destructive operations | Operation | CLI | What is lost | |---|---|---| | Delete a VM | `hoody vm delete` | The VM, its disk, firmware and TPM state, its snapshots and their available exports, and its shared-folder files. `--force` first stops a running VM without asking the guest | | Force stop | `hoody vm stop --force` | Unsaved data inside the guest | | Restore a snapshot | `hoody vm snapshots restore` | The current disk, varstore and TPM state | | Delete a snapshot | `hoody vm snapshots delete` | That snapshot | | Delete an image or artifact | `hoody vm isos delete`, `hoody vm imports delete` | The stored image or artifact | | Recover an orphaned VM | `hoody vm force adopt`, `hoody vm force markdead` | Mark-dead on a run still alive lets its disk be reused while the old guest writes to it | | Clear an unidentified run | `hoody vm unidentifiedruns force markdead` | Its memory is released; if the guest still runs, new VMs can be admitted against memory it uses | On a terminal these ask for confirmation; `-y` confirms non-interactively. Unfinished uploads, published images and imported artifacts keep their space until you delete them. ## Troubleshooting | Symptom | Cause | Fix | |---|---|---| | Every start refused with `KvmUnavailable` | The service cannot use KVM: `/dev/kvm` is missing or inaccessible, or QEMU reports KVM disabled | Check `kvm_present` in `hoody vm capabilities get`. VMs start only once the computer has a usable KVM device. | | Start refused with `MemoryExceedsCgroup` | Running VMs plus this one exceed the computer's memory limit | Stop a VM in `running_set`, or lower this one with `hoody vm update dev-vm --memory-mib 2048` while stopped. `hoody vm unidentifiedruns list` shows memory no VM name accounts for. | | `DisplayInUse` on start, create or update | The number is used by the desktop, a terminal, a browser or another VM; at create and update, also one another VM is set to. A display started before VM displays had their own window manager is refused too: the terminal was not created in VM mode. | At create, resend with another number or `"auto"`. For an existing VM, `PATCH` it while stopped, then start it. | | Start refused with `DisplayUnavailable` | No display usable: the service could not start it, could not tell what runs there or whose it is, could not connect, or settings changed during the start. The message names the step: terminal service unreachable, slow or too old, ownership record unreadable, or connection refused. | Do what the message says and start again. If it repeats, use another number or run headless. | | `DisplayRangeExhausted` on create or update | `"auto"`, and every number from `first` to `last` is taken | Delete a VM, make one headless with `display: null`, close what draws on a number, or give an explicit number. | | Display route refused with `DisplayNotConfigured` | The VM is headless | Stop it, `PATCH` `display` to a number or `"auto"`, start it. | | VM is `Orphaned` | The run cannot be verified | See [recovery](#recovery). | | Pause and resume refused while `Running` | QEMU stopped answering control commands; the VM shows `monitor_stalled: true` | Stop the VM: a stop completes without the control connection. | | Nothing can connect to the VM | No port forward for that port | Stop it and read `spec.ports` with `hoody vm get dev-vm`. `--ports` replaces the list, so resend all: `hoody vm update dev-vm --ports proto=tcp,host=8080,guest=80 --ports proto=tcp,host=2222,guest=22`. |