# VM: Recovery **Page:** api/vm/recovery [Download Raw Markdown](./api/vm/recovery.md) --- # VM recovery The VM recovery endpoints of a computer's VM service handle two failure shapes: a run that lost its VM name, and a VM left `Orphaned`. Use them to list the runs that hold memory under no VM name, to take over an orphaned VM's run, to record an orphaned VM's run as dead, and to release an unidentified run. All three write operations on this page are destructive and are meant for a person who has checked the guest process. Read the VM's `reason` before acting. Who may call these routes is set by the computer's proxy permission rules. ## List unidentified runs `GET /api/v1/vm/unidentified-runs` Lists the runs that hold memory against this computer under no VM name. The list is normally empty. Each entry holds memory against this computer, so a non-empty list can explain why VM starts are refused for lack of memory. This endpoint takes no parameters. ### Response The unidentified runs, in no particular order. Normally empty. Media type: application/json ```json [ { "run": "4f1c9a2e7b3d5a60", "reason": "unreadable-journal", "liveness": "unknown", "admitted_bytes": 536870912, "estimated": true, "clear_route": "/api/v1/vm/unidentified-runs/4f1c9a2e7b3d5a60/force-mark-dead" } ] ``` The request is refused because it carries query parameters, which this operation does not accept. Media type: application/json ```json { "error": { "code": "SpecInvalid", "message": "This operation takes no query parameters", "retryable": false } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `SpecInvalid` | Invalid request | The operation takes no query parameters, and a request carrying any is refused. | Remove every query parameter and resend the request. | ### Response fields Each entry describes one unidentified run. | Field | Type | Required | Description | |---|---|---|---| | `run` | string | Yes | The run ID: 16 lowercase hexadecimal digits, never all zeros. Pass it when marking this run as dead, which releases the memory it holds. | | `reason` | string | Yes | Why this run has no VM name. Always `unreadable-journal`. | | `liveness` | string | Yes | Always `unknown`: no process is recorded for the run, so the service cannot tell whether it is alive. Clearing it therefore needs a human's confirmation. | | `admitted_bytes` | integer or null | Yes | Bytes of memory held against this run. This is the whole memory ceiling, because the run's real size is unknown. `null` when the computer has no memory limit. The key is always present. | | `estimated` | boolean | Yes | Always `true`: `admitted_bytes` is an estimate, not a measurement. | | `clear_route` | string | Yes | The route that releases this run's memory. It names the same run ID as `run`. | ### Example ```bash curl -X GET "https://{projectId}-{computerId}-vm-1.{server}.computers.hoody.com/api/v1/vm/unidentified-runs" ``` ```typescript import { HoodyClient } from 'hoody-sdk'; const client = new HoodyClient({ baseURL: 'https://{projectId}-{computerId}-vm-1.{server}.computers.hoody.com' }); const runs = await client.vm.unidentifiedRuns.list(); ``` ## Mark an unidentified run dead `POST /api/v1/vm/unidentified-runs/{run}/force-mark-dead` Records an unidentified run as dead, releases the memory it holds and moves the run's files aside instead of deleting them. The run ID is named twice, in the path and as `confirm_run` in the request body, so a body that names a different run is refused. The service cannot verify that the run is dead. If the guest still runs, new VMs can be admitted against memory the run still uses. Read the entry's `reason` and `liveness`, check the guest process, and only then clear the run. ### Parameters | Name | In | Type | Required | Description | |---|---|---|---|---| | `run` | path | string | Yes | The run ID from `GET /unidentified-runs`: 16 lowercase hexadecimal digits, never all zeros. | ### Request body | Field | Type | Required | Description | |---|---|---|---| | `confirm_run` | string | Yes | Must equal `{run}` in the path. Naming the run twice guards against clearing the wrong one. | ```json { "confirm_run": "4f1c9a2e7b3d5a60" } ``` ### Response The memory was released and the run's files were moved aside, not deleted. Media type: none. The response has no body. The run ID is malformed, or the body does not confirm the same run, or the request carries query parameters. Media type: application/json ```json { "error": { "code": "NameInvalid", "message": "The run ID must be 16 lowercase hexadecimal digits and not all zeros", "retryable": false } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `NameInvalid` | Invalid run ID | The run ID in the path is not 16 lowercase hexadecimal digits, or it is all zeros. | Read the run ID from `GET /unidentified-runs` and resend it unchanged. Not retryable as written. | | `SpecInvalid` | Invalid request | `confirm_run` is missing or does not match the run ID in the path, or the request carries query parameters. | Send `confirm_run` with the same run ID as the path and remove every query parameter. Not retryable as written. | No unidentified run has that ID. Media type: application/json ```json { "error": { "code": "NotFound", "message": "No unidentified run has that ID", "retryable": false } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `NotFound` | Run not found | No unidentified run has that ID. | List the runs with `GET /unidentified-runs` and use an ID from that list. Not retryable as written. | The request body stopped arriving. Media type: application/json ```json { "error": { "code": "RequestBodyStalled", "message": "The request body stopped arriving", "retryable": true } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `RequestBodyStalled` | Request body stalled | The request body stopped arriving. | Resend the request with the complete body. Retryable. | The run's files could not be moved aside. The memory stays held. Media type: application/json ```json { "error": { "code": "StorageFailed", "message": "The run's files could not be moved aside", "retryable": false } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `StorageFailed` | Storage failure | The run's files could not be moved aside. The memory stays held. | Fix the storage problem, then retry the request. Not retryable as written. | ### Example ```bash curl -X POST "https://{projectId}-{computerId}-vm-1.{server}.computers.hoody.com/api/v1/vm/unidentified-runs/4f1c9a2e7b3d5a60/force-mark-dead" \ -H "Content-Type: application/json" \ -d '{"confirm_run": "4f1c9a2e7b3d5a60"}' ``` ```typescript import { HoodyClient } from 'hoody-sdk'; const client = new HoodyClient({ baseURL: 'https://{projectId}-{computerId}-vm-1.{server}.computers.hoody.com' }); await client.vm.unidentifiedRuns.forceMarkDead('4f1c9a2e7b3d5a60', { confirm_run: '4f1c9a2e7b3d5a60' }); ``` ## Adopt an orphaned VM's run `POST /api/v1/vm/vms/{name}/force-adopt` Resumes managing an orphaned VM's run. The service takes the run back under management and the VM becomes managed again. Adopting takes over a run that may still be executing. Read the VM's `reason`, check the guest process, and confirm that the VM is `Orphaned` before adopting. ### Parameters | Name | In | Type | Required | Description | |---|---|---|---|---| | `name` | path | string | Yes | The VM name: lowercase letters, digits and `-`, starting with a letter or digit, at most 63 characters. | This endpoint takes no request body. ### Response The run was adopted and the VM is managed again. The body is a VM detail object, whose fields are listed under VM detail fields. Media type: application/json ```json { "name": "dev-vm", "vm_id": "0000019a3f5c2e11", "state": "Running", "run": "4f1c9a2e7b3d5a60", "progress": { "phase": "committed", "bytes_done": 10737418240, "bytes_total": 10737418240 }, "sidecars": [ { "kind": "passt", "state": "ready" }, { "kind": "virtiofsd", "state": "ready" }, { "kind": "swtpm", "state": "ready" } ], "spec": { "memory_mib": 2048, "vcpus": 2, "disk_bytes": 10737418240, "firmware": "uefi", "bus": "virtio", "display_adapter": "virtio-vga", "cpu": "qemu-default", "tpm": true, "file_sharing": true, "agent": true, "console": true, "console_input": true, "audio": "none", "machine_type": "q35", "display": 1, "autostart": false, "share_quota_bytes": 1073741824, "ports": [ { "proto": "tcp", "host": 2222, "guest": 22 } ] }, "share": { "bytes": 1048576, "partial": false, "sampled_unix": 1760000000, "quota_bytes": 1073741824, "over_quota": false }, "seq": 42 } ``` The VM name is invalid, or the request carries query parameters, which this operation does not accept. Media type: application/json ```json { "error": { "code": "NameInvalid", "message": "The VM name is invalid", "retryable": false } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `NameInvalid` | Invalid VM name | The VM name is invalid. | Use a name of lowercase letters, digits and `-`, starting with a letter or digit, at most 63 characters. Not retryable as written. | | `SpecInvalid` | Invalid request | The operation takes no query parameters, and a request carrying any is refused. | Remove every query parameter and resend the request. Not retryable as written. | The VM does not exist. Media type: application/json ```json { "error": { "code": "NotFound", "message": "The VM does not exist", "retryable": false } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `NotFound` | VM not found | The VM does not exist. | Check the VM name. Not retryable as written. | The VM is in the wrong state for adoption, its create is still in progress, or the service has blocked the VM. Media type: application/json ```json { "error": { "code": "BadState", "message": "The VM is not Orphaned, there is no run to adopt, or its create failed", "retryable": true } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `BadState` | Wrong VM state | The VM is not `Orphaned`, there is no run to adopt, or its create failed. | Read the VM, check its `state`, and retry once it is `Orphaned`. Retryable. | | `OperationInProgress` | Create in progress | The VM is still being created, or has just been created and is not ready to be managed yet. | Retry shortly. Retryable. | | `Orphaned` | VM blocked | The service has blocked this VM and refuses both force commands. | Follow the error message, then restart `hoody-vm`. Not retryable as written. | ### Example ```bash curl -X POST "https://{projectId}-{computerId}-vm-1.{server}.computers.hoody.com/api/v1/vm/vms/dev-vm/force-adopt" ``` ```typescript import { HoodyClient } from 'hoody-sdk'; const client = new HoodyClient({ baseURL: 'https://{projectId}-{computerId}-vm-1.{server}.computers.hoody.com' }); const vm = await client.vm.vms.forceAdopt('dev-vm'); ``` ## Mark an orphaned VM's run as dead `POST /api/v1/vm/vms/{name}/force-mark-dead` Records an orphaned VM's run as dead. The VM is no longer `Orphaned` and can be started or deleted. The operation is accepted only when the VM's disk is not locked. This records the run as dead while the guest may still be running. Read the VM's `reason`, check the guest process, and only then mark the run dead. While the VM's disk is still locked by a running process, the request is refused. ### Parameters | Name | In | Type | Required | Description | |---|---|---|---|---| | `name` | path | string | Yes | The VM name: lowercase letters, digits and `-`, starting with a letter or digit, at most 63 characters. | This endpoint takes no request body. ### Response The run was marked dead. The VM is no longer `Orphaned` and can be started or deleted. The body is a VM detail object, whose fields are listed under VM detail fields. Media type: application/json ```json { "name": "dev-vm", "vm_id": "0000019a3f5c2e11", "state": "Stopped", "run": "4f1c9a2e7b3d5a60", "progress": { "phase": "committed", "bytes_done": 10737418240, "bytes_total": 10737418240 }, "sidecars": [ { "kind": "passt", "state": "ready" }, { "kind": "virtiofsd", "state": "ready" }, { "kind": "swtpm", "state": "ready" } ], "spec": { "memory_mib": 2048, "vcpus": 2, "disk_bytes": 10737418240, "firmware": "uefi", "bus": "virtio", "display_adapter": "virtio-vga", "cpu": "qemu-default", "tpm": true, "file_sharing": true, "agent": true, "console": true, "console_input": true, "audio": "none", "machine_type": "q35", "display": 1, "autostart": false, "share_quota_bytes": 1073741824, "ports": [ { "proto": "tcp", "host": 2222, "guest": 22 } ] }, "share": { "bytes": 1048576, "partial": false, "sampled_unix": 1760000000, "quota_bytes": 1073741824, "over_quota": false }, "seq": 43 } ``` The VM name is invalid, or the request carries query parameters, which this operation does not accept. Media type: application/json ```json { "error": { "code": "NameInvalid", "message": "The VM name is invalid", "retryable": false } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `NameInvalid` | Invalid VM name | The VM name is invalid. | Use a name of lowercase letters, digits and `-`, starting with a letter or digit, at most 63 characters. Not retryable as written. | | `SpecInvalid` | Invalid request | The operation takes no query parameters, and a request carrying any is refused. | Remove every query parameter and resend the request. Not retryable as written. | The VM does not exist. Media type: application/json ```json { "error": { "code": "NotFound", "message": "The VM does not exist", "retryable": false } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `NotFound` | VM not found | The VM does not exist. | Check the VM name. Not retryable as written. | The VM is in the wrong state, its create is still in progress, or the run is not dead because the VM's disk is still locked. Media type: application/json ```json { "error": { "code": "BadState", "message": "The VM is not Orphaned, or its create failed", "retryable": true } } ``` | Error Code | Title | Description | Resolution | |------------|-------|-------------|------------| | `BadState` | Wrong VM state | The VM is not `Orphaned`, or its create failed. | Read the VM, check its `state`, and retry once it is `Orphaned`. Retryable. | | `OperationInProgress` | Create in progress | The VM is still being created, or has just been created and is not ready to be managed yet. | Retry shortly. Retryable. | | `Orphaned` | Disk locked or VM blocked | The VM's disk is still locked by a running process, so the run is not dead; or the service has blocked this VM and refuses both force commands. | Check the guest process and stop it so the disk lock clears, or follow the error message and then restart `hoody-vm`. Not retryable as written. | ### Example ```bash curl -X POST "https://{projectId}-{computerId}-vm-1.{server}.computers.hoody.com/api/v1/vm/vms/dev-vm/force-mark-dead" ``` ```typescript import { HoodyClient } from 'hoody-sdk'; const client = new HoodyClient({ baseURL: 'https://{projectId}-{computerId}-vm-1.{server}.computers.hoody.com' }); const vm = await client.vm.vms.forceMarkDead('dev-vm'); ``` ## VM detail fields The 200 responses of force-adopt and force-mark-dead return the same VM detail object. | Field | Type | Required | Description | |---|---|---|---| | `name` | string | Yes | The VM name. | | `vm_id` | string | No | The VM's identifier: 16 lowercase hexadecimal digits. | | `state` | string | Yes | One of `Creating`, `Stopped`, `Starting`, `Running`, `Paused`, `Stopping`, `Failed`, `Orphaned`. | | `reason` | string | No | Human-readable reason for the current state. Present on `Failed` and `Orphaned`, absent otherwise. | | `run` | string | No | The run ID: 16 lowercase hexadecimal digits, never all zeros. | | `progress.phase` | string | Yes | One of `reserving`, `importing`, `committing`, `committed`, `failed`. | | `progress.bytes_done` | integer | Yes | Bytes written so far. | | `progress.bytes_total` | integer | Yes | Bytes expected in total. | | `progress.fell_back` | string | No | Set when the disk could not be cloned by reflink and was fully copied instead, which takes longer and uses more space. | | `progress.warnings` | array of string | No | Things you should know about a create that succeeded. Omitted when there are no warnings. | | `progress.error.error.code` | string | Yes | The error code for a failed create. | | `progress.error.error.message` | string | Yes | Human-readable message for the failed create. | | `progress.error.error.retryable` | boolean | Yes | Whether repeating the identical request could succeed without anything else changing. | | `sidecars[].kind` | string | Yes | Which helper: `passt` for the guest's network, `virtiofsd` for its shared folder, `swtpm` for its TPM. | | `sidecars[].state` | string | Yes | One of `absent`, `starting`, `ready`, `degraded`, `stopped`. | | `spec.memory_mib` | integer | Yes | Guest RAM in MiB. | | `spec.vcpus` | integer | Yes | Number of virtual CPUs. | | `spec.disk_bytes` | integer | No | Size of the system disk in bytes. Absent when the size could not be read, which does not mean zero. | | `spec.firmware` | string | Yes | `uefi`, `uefi-secure-boot` or `bios`. | | `spec.bus` | string | Yes | `virtio` or `sata`. | | `spec.display_adapter` | string | Yes | `vga` or `virtio-vga`. | | `spec.cpu` | string | Yes | `qemu-default`, `westmere` or `host`. | | `spec.tpm` | boolean | Yes | Whether the VM has a TPM 2.0. | | `spec.file_sharing` | boolean | Yes | Whether the VM's `share/` folder is shared into the guest. | | `spec.agent` | boolean | Yes | Whether the VM has the guest-agent channel, the device rather than whether an agent runs in the guest. | | `spec.console` | boolean | Yes | Whether the VM has a serial console. | | `spec.console_input` | boolean | Yes | Whether the console may be written to. Always `false` when `spec.console` is `false`. | | `spec.audio` | string | Yes | `none` or `output`. | | `spec.machine_type` | string | Yes | The QEMU machine type the VM is pinned to. | | `spec.display` | integer | No | X display number. Absent when the VM is headless. | | `spec.cdroms` | array of string or null | No | Attached read-only media by slot, in boot order. `null` is an empty slot that keeps its position. Omitted when the VM has no CD-ROM slots. | | `spec.autostart` | boolean | Yes | Whether the VM starts automatically when the service starts. | | `spec.disk_bps` | integer | No | Disk throughput limit in bytes per second. Absent when unlimited. | | `spec.disk_iops` | integer | No | Disk operations per second limit. Absent when unlimited. | | `spec.share_quota_bytes` | integer | No | Size limit in bytes for the VM's `share/` folder. Absent when unlimited. | | `spec.ports[].proto` | string | No | `tcp` or `udp`. Default: `tcp`. | | `spec.ports[].host` | integer | Yes | Port opened on every address of the computer, from 1 to 65535. | | `spec.ports[].guest` | integer | Yes | Port the guest listens on, from 1 to 65535. | | `spec.stop_grace_seconds` | integer | No | How long a graceful stop waits for the guest to power itself down, in seconds. Absent when the default of 30 applies. | | `display_url` | string | No | The address of the VM's screen in the browser. Present only in the answer to `GET /vms/{name}`, and only when the VM has a display numbered above 0. | | `monitor_stalled` | boolean | No | The QMP monitor is connected and has stopped answering. Omitted when false, so absence means healthy. | | `seq` | integer | No | The VM's event counter as of this response. See the note below. | | `error.error.code` | string | Yes | The error code for the current run's start failure. | | `error.error.message` | string | Yes | Human-readable message for the start failure. | | `error.error.retryable` | boolean | Yes | Whether repeating the identical request could succeed without anything else changing. | | `share.bytes` | integer | Yes | Allocated size in bytes of the VM's `share/` folder. | | `share.partial` | boolean | Yes | `true` when the measurement stopped early, so `bytes` is a lower bound rather than a total. | | `share.sampled_unix` | integer | Yes | When the measurement was taken, in Unix seconds. | | `share.quota_bytes` | integer | No | The VM's `share_quota_bytes`. Absent when the share has no size limit. | | `share.over_quota` | boolean | Yes | The latched over-quota state of the share. See the note below. | `seq` is the VM's event counter as of the response. To combine a stream with a read of this resource, open the stream first, then read. A `state` event whose non-zero `seq` is not above this value describes a change the response already reflects and can be skipped. Process every other event whatever its `seq`: events with `seq: 0` are not counted here, and a `resync` always reports a gap. A per-VM `resync` uses `data.detail.seq` when the detail has one; otherwise, for example a detail with no `seq` while the VM is being created, or a `resync` that carries `data.error`, it repeats the stream's last known `seq`, which does not prove the current counter, so re-read the VM. The all-VMs stream's `resync` carries `seq` 0, as do the uncounted kinds (`progress`, `sidecar`, `qmp-error`, `snapshot`, `share-quota`, `monitor-stalled`, `deleted`) and the `state` frame that ends a create. Process every `resync` whatever its `seq`. `share.over_quota` is the latched state left by the last quota crossing, not a comparison with the latest measurement. It turns `true` when a measurement exceeds the quota, and a partial measurement counts, because its size is a lower bound. It turns `false` only when a complete measurement is at or below the return level, `quota - quota / 16` (integer division, rounded down), or when the quota is removed, so it can stay `true` while the measured size is below the quota but above the return level. Changing the quota re-checks the last measurement and can change the state. Each change publishes one `share-quota` event, and nothing else publishes one. It is always `false` when there is no quota. ## Error envelope Every error response carries the same envelope. | Field | Type | Required | Description | |---|---|---|---| | `error` | object | Yes | The error object. | | `error.code` | string | Yes | The error code, from a fixed set. Branch on this, not on `message`: the message is human-readable text and may change. | | `error.message` | string | Yes | Human-readable description of the failure. | | `error.retryable` | boolean | Yes | Whether repeating the identical request could succeed without anything else changing. |