VM Management

Compute providers manage virtual machines through the vm.Manager interface. The production implementation uses Lima (Linux virtual machines via QEMU) to create isolated Ubuntu 24.04 VMs with SSH access. VMs are provisioned on lease acceptance and torn down on settlement. Boot time is approximately 21 seconds from lease acceptance to a running VM with SSH.

#Manager interface

1type Manager interface {2    Provision(leaseHash string, res Resources, accessPubKey string) (*Info, error)3    Teardown(leaseHash string) error4    Exec(leaseHash string, command string) (*ExecResult, error)5    DialSSH(leaseHash string) (net.Conn, error)6    Get(leaseHash string) (*Info, error)7    List() []*Info8}

#Lima implementation

The LimaManager (vm/lima_manager.go) manages QEMU-based VMs using the limactl CLI tool from the Lima project.

#Architecture

xe-node process (provider)    │    ├── LimaManager    │     ├── limactl create (from YAML template)    │     ├── limactl start (boots QEMU VM)    │     ├── limactl shell (exec commands)    │     └── limactl delete --force (teardown)    │    └── QEMU processes (one per active lease)          ├── xe-<leaseHash[:12]> → Ubuntu 24.04 VM          └── xe-<leaseHash[:12]> → Ubuntu 24.04 VM

#Initialisation

When a provider node starts with --provide, the LimaManager is created:

mgr, err := vm.NewLimaManager(cfg.DataDir, cfg.LimactlPath)

• LIMA_HOME is set to {dataDir}/lima — all VM state, disk images, and sockets live here
• Template directory is {dataDir}/lima-templates — YAML templates are written here before limactl create
• Images directory is {dataDir}/images — the Ubuntu cloud image is cached here
• On startup, limactl --version is executed to verify the tool is available
• If the Ubuntu cloud image is not present locally, it is downloaded automatically from the upstream Ubuntu cloud images server (~600 MB)
• The manager logs the Lima version and advertised resource capacity

#VM naming convention

VMs are named xe-{leaseHash[:12]} — the first 12 hex characters of the lease block hash. This keeps names short while avoiding collisions. For example, lease hash 6226fb52501a9052b533... creates a VM named xe-6226fb52501a.

#Provision flow

When Provision(leaseHash, resources, accessPubKey) is called:

1. Convert access key — the accessPubKey (ed25519 public key as 64-char hex string) is converted to SSH authorized_keys format. This is the consumer's key for SSH access to the VM.

2. Render template — a Lima YAML template is generated referencing the local Ubuntu 24.04 cloud image with the requested resources and SSH key:

   vmType: "qemu"images:  - location: "file:///var/lib/xe-node/images/ubuntu-24.04-x86_64.img"    arch: "x86_64"cpus: 1memory: "512MiB"disk: "5GiB"mounts: []containerd:  system: false  user: falseprovision:  - mode: system    script: |      #!/bin/bash      mkdir -p /root/.ssh && chmod 700 /root/.ssh      echo '<ssh-ed25519 key>' >> /root/.ssh/authorized_keys      chmod 600 /root/.ssh/authorized_keys      sed -i 's/#PermitRootLogin.*/PermitRootLogin prohibit-password/' /etc/ssh/sshd_config      sed -i 's/#PubkeyAuthentication.*/PubkeyAuthentication yes/' /etc/ssh/sshd_config      sed -i 's/#PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config      systemctl restart sshd

   The image is referenced via file:// from the local cache at {dataDir}/images/. SSH is pre-installed in the Ubuntu cloud image — the provision script only injects the consumer's key and restarts sshd.

3. Write template — saved to {dataDir}/lima-templates/{vmName}.yaml

4. Create VM — limactl create --name={vmName} {templatePath} uses the cached Ubuntu cloud image as a copy-on-write backing file (instant, no download)

5. Start VM — limactl start {vmName} boots QEMU with KVM acceleration. Ubuntu boots in ~15 seconds, cloud-init injects the SSH key, and Lima confirms SSH readiness. Total: ~21 seconds.

6. Wait for SSH — polls limactl list --json every 2 seconds for up to 2 minutes until the VM reports an SSH port. This port is on 127.0.0.1 and is the tunnel target.

7. Store state — the VM info (lease hash, SSH port, status, resources) is stored in the in-memory vms map

#Resource constraints

If the lease requests less than the minimum, the minimum is used.

#Teardown

When Teardown(leaseHash) is called (triggered by lease settlement):

1. Execute limactl delete --force {vmName} — this stops the QEMU process and removes all VM artifacts (disk, sockets, logs)
2. Delete the template file from {dataDir}/lima-templates/
3. Remove the entry from the in-memory vms map

#Command execution

Exec(leaseHash, command) runs commands inside the VM via Lima's shell mechanism:

limactl shell {vmName} -- sh -c "{command}"

Returns an ExecResult with: - ExitCode — the command's exit code - Stdout — captured standard output - Stderr — captured standard error

#SSH connection

DialSSH(leaseHash) opens a raw TCP connection to the VM's SSH port:

net.Dial("tcp", fmt.Sprintf("127.0.0.1:%d", vm.sshPort))

This is used by the tunnel protocol to proxy SSH sessions from remote consumers through libp2p streams to the VM's local SSH server.

#Data types

#Resources

1type Resources struct {2    VCPUs    uint64 `json:"vcpus"`3    MemoryMB uint64 `json:"memory_mb"`4    DiskGB   uint64 `json:"disk_gb"`5}

#Info

1type Info struct {2    LeaseHash   string     `json:"lease_hash"`3    Status      string     `json:"status"`4    Resources   *Resources `json:"resources,omitempty"`5    CreatedAt   int64      `json:"created_at"`6    Error       string     `json:"error,omitempty"`7}

#VM status values

#ExecResult

1type ExecResult struct {2    ExitCode int    `json:"exit_code"`3    Stdout   string `json:"stdout"`4    Stderr   string `json:"stderr"`5}

#VM lifecycle

lease_accept confirmed        │        ▼  ┌──────────────┐    vm_credentials     ┌──────────┐  │ Provision    │ ──────────────────────►│ Consumer │  │ (limactl     │    (direct msg)       └──────────┘  │  create+start)│  └──────────────┘        │        ▼  ┌──────────────┐  │ Running      │ ◄── SSH tunnel from consumer (via libp2p)  │ (Ubuntu VM)  │ ◄── Exec commands via API or messaging  └──────────────┘        │        │  lease expires + settle        ▼  ┌──────────────┐  │ Teardown     │  │ (limactl     │  │  delete)     │  └──────────────┘

1. Provision — triggered by autoAcceptLease() after the lease_accept block is confirmed (~21 seconds). Lima creates an Ubuntu 24.04 VM from the cached cloud image with the consumer's SSH key injected via cloud-init.
2. Credential delivery — the provider sends a vm_credentials direct message to the consumer with the VM's SSH connection details.
3. Running — the consumer can access the VM via:
   • SSH tunnel through the SSH gateway (remote access via libp2p)
   • Exec API via POST /vms/{lease}/exec (HTTP API)
   • Direct messaging via the vm_exec protocol (libp2p)
4. Teardown — triggered by settleLease() after the lease_settle block is confirmed. The VM is deleted and its resources freed.

#VM access methods

#SSH gateway (recommended)

The SSH gateway provides transparent remote access to VMs. The consumer connects via standard SSH:

$ ssh -i ~/.ssh/lease_key -p 2222 <leaseHash>@<gateway-host>

The gateway authenticates using the AccessPubKey from the lease block, then tunnels the session through libp2p to the provider node, which proxies to the VM's local SSH server. See SSH Gateway & Tunnel Protocol for full details.

#HTTP exec API

For programmatic access, the provider exposes an exec endpoint:

POST /vms/{leaseHash}/execContent-Type: application/json{"command": "uname -a"}

Response:

1{2    "exit_code": 0,3    "stdout": "Linux lima-xe-6226fb52501a 6.8.0-106-generic #106-Ubuntu SMP ...\n",4    "stderr": ""5}

#Direct messaging (vm_exec)

Consumers can also execute commands via the libp2p messaging protocol:

Consumer                    Provider   │                           │   │  vm_exec {lease, cmd}     │   │──────────────────────────►│   │                           │  verify consumer identity   │                           │  limactl shell {vm} -- sh -c {cmd}   │  ExecResult {code, out}   │   │◄──────────────────────────│

The provider node verifies the requesting peer owns the consumer account before executing.

#Message handlers

The node registers VM-related message handlers:

#Provider node flags

Providers enable compute leasing with the following CLI flags:

$ xe-node -provide -vcpus 4 -memory 8192 -disk 100 -ssh-port 2222

When -provide is set, the node:

1. Initialises the LimaManager with the configured limactl path.
2. Sets up the tunnel protocol handler for incoming SSH tunnel requests.
3. Advertises available resources via marketplace gossip every 5 minutes.
4. Watches for incoming lease blocks every 5 seconds and automatically accepts them.
5. Runs the settleLoop() to auto-settle expired leases every 10 seconds.

#Marketplace discovery

Providers and consumers find each other through the marketplace gossip topic (xe/marketplace):

1. Consumer publishes a ResourceRequest with desired vCPUs, memory, disk, and duration.
2. Provider receives the request, checks available resources, and replies with a ResourceOffer including the computed cost.
3. Consumer receives the offer and creates the lease block.

The entire negotiation happens over gossip before any on-chain blocks are created.

#Infrastructure requirements

Lima uses QEMU as its virtualisation backend. The host must have:

• QEMU installed (qemu-system-x86_64)

• Lima installed (limactl v1.0.6+)

• KVM support — Lima requires /dev/kvm (hardware virtualisation extensions: Intel VT-x or AMD-V). Hosts without KVM (most cloud VPS) cannot run Lima VMs.

• KVM permissions — the xe user must be able to access /dev/kvm. Since pm2's uid/gid setting drops supplementary groups, add a udev rule:

  # /etc/udev/rules.d/99-kvm.rulesKERNEL=="kvm", MODE="0666"

• Non-root execution — Lima refuses to run as root. The node process must run as a non-root user (e.g., the xe service user).

• Disk space — the Ubuntu cloud image is ~600 MB. Each running VM uses additional disk for its QCOW2 overlay (copy-on-write from the shared base image).