diff --git a/.gitignore b/.gitignore index 4dc3d79..b37f101 100644 --- a/.gitignore +++ b/.gitignore @@ -42,3 +42,5 @@ dist/ # generated files apis/platform.yaml build.log +AGENTS.md +unikernel-mentions-docs - Sheet1.csv diff --git a/pages/features/autoscale.mdx b/pages/features/autoscale.mdx index 61e7c88..6d7ef1f 100644 --- a/pages/features/autoscale.mdx +++ b/pages/features/autoscale.mdx @@ -9,6 +9,8 @@ You can transparently and effortlessly handle load increase including traffic pe No more headaches due to slow autoscale like keeping hot instances around to deal with peaks, coming up with complex predictive algorithms, or other painful workarounds. You can set autoscale on and let Unikraft Cloud handle your traffic increases and peaks. +![Horizontal scaling](/images/diagrams/horizontal-scaling.svg) + ## The basics diff --git a/pages/features/forking.mdx b/pages/features/forking.mdx index 5fa5122..efa0069 100644 --- a/pages/features/forking.mdx +++ b/pages/features/forking.mdx @@ -12,6 +12,8 @@ Instance forking lets a running instance create an independent copy of itself at Both the original instance (the **parent**) and the copy (the **child**) resume execution from the same memory state. This works like `fork(2)` in a POSIX process. +![Instance forking](/images/diagrams/forking.svg) + The app inside an instance triggers the fork through a filesystem interface. ## Filesystem interface diff --git a/pages/features/live-migration.mdx b/pages/features/live-migration.mdx new file mode 100644 index 0000000..5e3a780 --- /dev/null +++ b/pages/features/live-migration.mdx @@ -0,0 +1,15 @@ +--- +title: Live Migration +navigation_icon: move +--- + +Live migration moves a running instance from one host to another with minimal downtime. + +![Live migration](/images/diagrams/live-migration.svg) + +Unikraft Cloud uses pre-copy migration: the platform iteratively transfers memory pages while the instance continues to run. +During the final switchover, the platform pauses the instance for milliseconds (typically under 10ms) to transfer the remaining dirty pages and resume execution on the destination host. + +:::note +This feature is in preview and isn't available on the public platform. +::: diff --git a/pages/features/roms.mdx b/pages/features/roms.mdx index 7f1f7c7..13ba844 100644 --- a/pages/features/roms.mdx +++ b/pages/features/roms.mdx @@ -429,6 +429,78 @@ kraft cloud instance rm test-http-python-rom1 test-http-python-rom2 +{/* vale off */} +## Inline ROMs +{/* vale on */} + +Inline ROMs let you specify ROM contents directly in the API request, without pre-building and pushing a ROM image. +Instead of providing an `image` reference on a ROM entry, you provide a `files` array. +The `files` and `image` fields are mutually exclusive: each ROM entry must use one or the other. + +Each file object in the `files` array accepts the following fields: + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | Yes | The filename inside the ROM. | +| `data` | string | Yes | The file contents (plaintext or base64-encoded). | +| `encoding` | string | No | `"plaintext"` (default) or `"base64"`. | + +Inline ROMs have the following limits: + +- Maximum of 16 files per ROM. +- Maximum of 1024 bytes per file. + +Inline ROMs are ephemeral: Unikraft Cloud deletes them when the instance stops. +You can combine inline ROMs with the `at` field for [automounting](#automounting). + +The following example creates an instance with an inline ROM containing a Python function: + +```json title="POST /instances" +{ + "image": "myuser/http-python:latest", + "memory_mb": 256, + "autostart": true, + "roms": [ + { + "name": "python_function.py", + "at": "/tmp", + "files": [ + { + "name": "rom.py", + "data": "def function():\n return 'Hello from inline ROM!\\n'", + "encoding": "plaintext" + } + ] + } + ] +} +``` + +## ROMs and snapshots + +ROMs are immutable once attached to an instance. +Unikraft Cloud enforces the following rules for instances created from snapshots or templates: + +- You **can't delete or replace** ROMs that existed when the platform took the snapshot. +The instance uses them and a state change could cause corruption. +- You **can add new ROMs** to any instance, including those resumed from a snapshot. +- Once you add a new ROM, you also **can't update or remove** it. + +If your data changes frequently, a ROM isn't the right choice. +Use a [volume](/platform/volumes) instead. + +### Building template chains with ROMs + +You can layer ROMs across template chains. +For example: + +1. Create a snapshotted template with ROM A. +2. Create an instance from that template and add ROM B. +3. Create a new template from that instance. + +You now have two templates: one with ROM A, another with ROMs A and B. +You can start instances from either template. + ## Learn more * The [CLI reference](/docs/cli/unikraft) and the legacy [CLI reference](/docs/cli/kraft/overview). diff --git a/pages/features/sandbox-api.mdx b/pages/features/sandbox-api.mdx new file mode 100644 index 0000000..f2e0b98 --- /dev/null +++ b/pages/features/sandbox-api.mdx @@ -0,0 +1,12 @@ +--- +title: Sandbox API +navigation_icon: box +--- + +The sandbox API provides programmatic control over isolated execution environments. +Combine templates, inline ROMs, scheduled commands, and [forking](/features/forking) to build sandbox products on Unikraft Cloud. + +:::note +This feature is in development. +More details coming soon. +::: diff --git a/pages/features/scale-to-zero.mdx b/pages/features/scale-to-zero.mdx index 9be2195..2cbc004 100644 --- a/pages/features/scale-to-zero.mdx +++ b/pages/features/scale-to-zero.mdx @@ -25,6 +25,12 @@ This differs from [autoscale](/features/autoscale), in which you *don't* specify The platform does this for you based on traffic load. ::: +![Scale-to-zero request forwarding](/images/diagrams/stz-request.svg) + +![Scale-to-zero wake-up chain](/images/diagrams/stz-ready-chain.svg) + +![Scale-to-zero reply](/images/diagrams/stz-reply.svg) + ## Policies diff --git a/pages/features/snapshots.mdx b/pages/features/snapshots.mdx index 37982db..30b5a5b 100644 --- a/pages/features/snapshots.mdx +++ b/pages/features/snapshots.mdx @@ -57,11 +57,42 @@ Unikraft Cloud waits until the app's exposed port becomes available, and then st This assumes no traffic is coming to that port. At this point, Unikraft Cloud saves the state of your app and sets the app to standby (consuming no resources). +![Snapshot lifecycle](/images/diagrams/basic-snapshots.svg) + Next, when traffic arrives, Unikraft Cloud brings up the app including its saved state. This ensures statefulness across scale-to-zero and scale-to-one cycles. It also eliminates long initialization times from heavyweight apps. +{/* vale off */} +## Copy-on-write checkpointing +{/* vale on */} + +Unikraft Cloud uses copy-on-write checkpointing to capture VM state with minimal downtime. + +![Copy-on-write checkpointing](/images/diagrams/checkpointing.svg) + +The process works as follows: + +1. The platform forks the VM process, freezing a copy of memory through Linux copy-on-write semantics. +1. The VM resumes immediately after the fork, resulting in microseconds of downtime. +1. The platform reads the frozen memory asynchronously and writes incremental updates to the snapshot file. +1. Only dirty memory pages transfer to storage, reducing I/O overhead. + +The snapshot format supports mixed compressed and uncompressed blocks for efficient storage. +This mechanism enables fast [scale-to-zero](/features/scale-to-zero) transitions and serves as the foundation for continuous checkpointing and [forking](/features/forking). + +## Continuous checkpointing + +Continuous checkpointing extends copy-on-write checkpointing by periodically creating incremental snapshots of a running instance. +Each pass captures only memory that changed since the last checkpoint. +This enables point-in-time recovery and fast migration. + +:::note +This feature is currently in development. +The interface may change before general availability. +::: + ## Learn more * The [CLI reference](/docs/cli/unikraft) and the legacy [CLI reference](/docs/cli/kraft/overview). diff --git a/pages/integrations/kubernetes.mdx b/pages/integrations/kubernetes.mdx index fe5eb2f..a98b344 100644 --- a/pages/integrations/kubernetes.mdx +++ b/pages/integrations/kubernetes.mdx @@ -7,6 +7,8 @@ Unikraft Cloud integrates seamlessly with any Kubernetes cluster through a virtu This is a lightweight Kubernetes node implementation which connects your cluster to Unikraft Cloud's high-performance compute instead of running real pods locally. This enables developers to deploy and manage Unikraft microVMs as if they were native Kubernetes pods. +![Kubernetes integration](/images/diagrams/k8s.svg) + This integration extends Kubernetes' scheduling and orchestration capabilities to the Unikraft Cloud platform. This allows workloads to take advantage of microVM-level I/O performance, security, cold start and transparent scale-to-zero efficiency while retaining full compatibility with existing Kubernetes tooling. @@ -367,6 +369,78 @@ Kraftlet reads the following annotations from Pod and Service objects to configu |---|---|---|---| | `cloud.unikraft.v1.services/domain` | string | — | Custom domain for the Unikraft Cloud service. For multi-container pods, prefix with the container name (`cloud.unikraft.v1.services/domain.`) to set a per-container domain, or use the global annotation to derive `-` automatically. | +## CRD reference + +The operator defines three CRDs in the `unikraft.com/v1alpha1` API group. + +### Instance CRD + +| Field | Type | Required | Mutable | Description | +|-------|------|----------|---------|-------------| +| `name` | string | No | No | Instance name | +| `image` | string | Yes | No | Container image reference | +| `args` | []string | No | Yes | Arguments passed to the instance | +| `env` | map[string]string | No | Yes | Environment variables | +| `memory_mb` | int64 | No | Yes | Memory in MiB (default: 128) | +| `vcpus` | int32 | No | Yes | Virtual CPUs (default: 1) | +| `service_group` | object | No | No | Service group with domains and services | +| `volumes` | []object | No | No | Volume mounts with UUID/name, size_mb, at, readonly | +| `autostart` | bool | No | No | Start on creation | +| `replicas` | int64 | No | No | Number of replicas | +| `restart_policy` | enum | No | No | never, always, or on_failure | +| `scale_to_zero` | object | No | Yes | Policy, stateful, cooldown_time_ms, notify_time_ms | +| `features` | []enum | No | No | Feature flags (delete_on_stop) | +| `roms` | []object | No | No | ROM images with name and image | +| `tags` | []string | No | No | Arbitrary tags | +| `template` | object | No | No | Template reference with UUID or name | +| `sched_priority` | int32 | No | No | Scheduling priority | + +Most spec fields don't change after creation. +Fields marked mutable accept in-place updates. + +### Service CRD + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | No | Service group name | +| `services` | []object | No | Port mappings (port, destination_port, handlers) | +| `domains` | []object | No | Domain bindings (name, certificate) | +| `soft_limit` | uint64 | No | Soft connection limit | +| `hard_limit` | uint64 | No | Hard connection limit | + +Service CRDs don't support in-place updates. + +### Volume CRD + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | string | No | Volume name | +| `size_mb` | uint64 | Yes | Volume size in MiB | + +Volume CRDs don't support in-place updates. + +## Helm chart configuration + +| Value | Description | +|-------|-------------| +| `ukc.metro` | (required) Platform metro identifier | +| `ukc.token` | (required) API authentication token | +| `image.name` | Operator container image name | +| `image.tag` | Operator container image tag | +| `image.pullSecrets` | Image pull secrets | +| `leaderElection.enabled` | Enable leader election for HA | +| `resources` | Kubernetes resource requests/limits | + +## Operator flags + +| Flag | Environment variable | Default | Description | +|------|---------------------|---------|-------------| +| `--ukc-token` | `OPERATOR_UKC_TOKEN` | — | (required) API token | +| `--ukc-metro` | `OPERATOR_UKC_METRO` | — | (required) Metro identifier | +| `--metrics-addr` | — | `:8080` | Metrics endpoint address | +| `--probe-addr` | — | `:8081` | Health probe address | +| `--enable-leader-election` | — | false | Enable HA leader election | + ## Resources - See [Unikraft public roadmap](https://roadmap.unikraft.com) for planned features or to suggest use cases and ideas. diff --git a/pages/platform/instances.mdx b/pages/platform/instances.mdx index a5cac20..6100eb4 100644 --- a/pages/platform/instances.mdx +++ b/pages/platform/instances.mdx @@ -132,6 +132,8 @@ A manual start or stop of the instance aborts the restart sequence and resets th ## Instance templates +![Instance templating](/images/diagrams/templating.svg) + An instance template is a snapshotted instance that acts as a source for cloning new instances. Cloning a template creates a new instance that resumes from the exact original system state. It preserves memory contents, open files, and populated caches to bypass the standard boot sequence. @@ -285,7 +287,63 @@ Otherwise, you will receive metrics in Prometheus text format, that you can pars ## Instance logs Use the [`GET /instances/logs`](/api/platform/v1/instances#get-instances-logs) endpoint to retrieve logs for one or more instances. -The logs capture the instance's `stdout` and `stderr` output, and they're preserved across restarts and stops. +The logs capture the instance's `stdout` and `stderr` output, and Unikraft Cloud preserves them across restarts and stops. + +### Log storage + +Unikraft Cloud stores instance logs in a `vm.log` file inside the instance's working directory. + +:::caution +The platform doesn't enforce an upper size limit on log files. +Logs grow indefinitely until storage space runs out. +For long-running or verbose instances, consider implementing log rotation in your app or redirecting output to an external logging system. +::: + +### Console redirection + +If the platform operator redirects instance logs to a Unix domain socket by configuring the default console, the `vm.log` file remains empty. +In this case, the CLI `logs` command and the logs API endpoint don't return any output. +Query the external logging system that receives the socket output instead. + +## Queued updates + +When you send a `PATCH` request to a running instance, Unikraft Cloud queues the updates instead of rejecting them. +The queued updates apply automatically the next time the instance enters the `stopped` state. + +### Properties that support queuing + +The following properties queue when the instance isn't stopped: + +`image`, `args`, `memory_mb`, `vcpus`, `env`, `roms`, `dependencies`, `hostname`, `sched_priority`, `autokill`, `scale_to_zero` + +### Properties that always apply immediately + +The following properties take effect regardless of instance state: + +`tags`, `delete_lock`, `schedules` + +### Behavior + +If you send multiple `PATCH` requests that update the same property, Unikraft Cloud keeps only the last value. +Failed updates retry up to 2 times before Unikraft Cloud removes them from the queue. + +The following example patches a running instance to change its memory on the next stop: + +```json title="PATCH /instances" +{ + "name": "my-instance", + "memory_mb": 1024 +} +``` + +Unikraft Cloud applies the new `memory_mb` value the next time `my-instance` enters the `stopped` state. + +### Resource controls + +:::note +Unikraft Cloud supports cgroup-based resource controls for CPU and memory. +This feature is in development. +::: ## Learn more diff --git a/pages/platform/volumes.mdx b/pages/platform/volumes.mdx index d41f73b..89fe704 100644 --- a/pages/platform/volumes.mdx +++ b/pages/platform/volumes.mdx @@ -298,6 +298,29 @@ kraft cloud volume remove The explicit volume detach command is only available in the legacy CLI. +## Volume lifecycle with running instances + +Unikraft Cloud doesn't support hot-plugging volumes. +The following rules apply: + +- You **can't attach a new volume** to a running instance. +Instances resumed from a snapshot also count as running. +- You **can't detach a volume** from a running instance. +Snapshotted instances in standby also count as running for this purpose. +- When you stop an instance and start it again, you can attach any volume from the start. +This isn't hot-plugging because the instance boots with the volume already present. + +### Volumes and templates + +When you create a template from an instance with volumes, the platform also creates **volume templates** for each attached volume. +Each instance you create from that template gets its own independent volume clone. + +:::note +Unikraft Cloud is working on a feature to attach a new volume to an instance created from a template at creation time. +::: + +If your workload needs to swap volumes frequently on running instances, consider designing your app to read from a fixed volume path and update the contents instead. + ## Volume templates A volume template is a volume in the `TEMPLATE` state. @@ -356,6 +379,21 @@ If the platform operator has registered more named filesystems, you can select o } ``` +### Mount and unmount hooks + +Operators can configure mount and unmount scripts for custom filesystem types. +Place executable scripts at `/mount` and `/unmount`. + +- The mount script runs before a volume attaches to an instance. +- The unmount script runs after all instances detach from the volume. + +This enables remote storage (Network File System, iSCSI), encrypted volumes, and overlay filesystems. +If an instance start aborts while a mount operation is pending, the unmount hook still runs to clean up resources. + +:::caution +This is an advanced feature that's only available on the self-hosted platform. +::: + ### Managed volumes :::caution diff --git a/public/images/diagrams/basic-snapshots.svg b/public/images/diagrams/basic-snapshots.svg new file mode 100644 index 0000000..f20d17d --- /dev/null +++ b/public/images/diagrams/basic-snapshots.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/images/diagrams/checkpointing.svg b/public/images/diagrams/checkpointing.svg new file mode 100644 index 0000000..e2fa6de --- /dev/null +++ b/public/images/diagrams/checkpointing.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/images/diagrams/forking.svg b/public/images/diagrams/forking.svg new file mode 100644 index 0000000..6a51f71 --- /dev/null +++ b/public/images/diagrams/forking.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/images/diagrams/horizontal-scaling.svg b/public/images/diagrams/horizontal-scaling.svg new file mode 100644 index 0000000..b8343d6 --- /dev/null +++ b/public/images/diagrams/horizontal-scaling.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/images/diagrams/k8s.svg b/public/images/diagrams/k8s.svg new file mode 100644 index 0000000..7b5974d --- /dev/null +++ b/public/images/diagrams/k8s.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/images/diagrams/live-migration.svg b/public/images/diagrams/live-migration.svg new file mode 100644 index 0000000..5561c18 --- /dev/null +++ b/public/images/diagrams/live-migration.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/images/diagrams/stz-ready-chain.svg b/public/images/diagrams/stz-ready-chain.svg new file mode 100644 index 0000000..029a57d --- /dev/null +++ b/public/images/diagrams/stz-ready-chain.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/images/diagrams/stz-reply.svg b/public/images/diagrams/stz-reply.svg new file mode 100644 index 0000000..1cac801 --- /dev/null +++ b/public/images/diagrams/stz-reply.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/images/diagrams/stz-request.svg b/public/images/diagrams/stz-request.svg new file mode 100644 index 0000000..b1b719a --- /dev/null +++ b/public/images/diagrams/stz-request.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/images/diagrams/templating.svg b/public/images/diagrams/templating.svg new file mode 100644 index 0000000..4f6364b --- /dev/null +++ b/public/images/diagrams/templating.svg @@ -0,0 +1 @@ + \ No newline at end of file