feat(kvp): add store trait and Hyper-V KVP storage crate by peytonr18 · Pull Request #288 · Azure/azure-init

peytonr18 · 2026-03-09T19:03:47Z

Summary

Introduces libazureinit-kvp, a standalone workspace crate that defines the KvpStore storage trait and provides a production implementation backed by the Hyper-V binary pool file format.

This is the foundational layer (Layer 0) for the KVP architecture rework. It establishes the storage abstraction that higher layers (diagnostics, tracing, provisioning reports) will build on in follow-up PRs.

Adds new workspace crate libazureinit-kvp with KvpStore trait and HyperVKvpStore implementation.
Defines KvpLimits as a trait-level requirement so consumers and alternative implementations can query size constraints generically.
Rewrites doc/libazurekvp.md as a technical reference for the new crate.
No consumers wired yet — libazureinit still uses the existing kvp.rs module. Integration happens in follow-up PRs.

Changes

`KvpStore` trait

Storage abstraction with explicit semantics:

limits() — returns KvpLimits for the store (required by the trait so consumers can chunk/validate generically)
write(key, value) — validates against limits, then stores
read(key) — last-write-wins for duplicate keys
entries() — deduplicated HashMap view
delete(key) — removes all records for a key

`HyperVKvpStore`

Production implementation backed by the Hyper-V binary pool file:

Fixed-size 2,560-byte records (512-byte key + 2,048-byte value, zero-padded)
Append-only writes with flock-based concurrency (exclusive for writes, shared for reads)
truncate_if_stale() clears records from previous boots by comparing file mtime to boot time
Write validation rejects empty keys, oversized keys, and oversized values (no silent truncation)

`KvpLimits`

PR feedback asked for limits to be required at the trait level rather than only on the concrete type. This ensures:

Any KvpStore implementation must declare its constraints.
Generic consumer code (fn foo<S: KvpStore>(store: &S)) can call store.limits().max_value_size for chunking decisions without hardcoding constants or knowing the concrete type.

Exported as part of the trait contract with two presets:

KvpLimits::hyperv() — 512-byte key / 2,048-byte value (raw Hyper-V wire format)
KvpLimits::azure() — 512-byte key / 1,022-byte value (UTF-16: 511 chars + null terminator; matches Azure host read limit)

Higher layers are responsible for chunking oversized payloads before calling the store.

Next Steps

Follow-up PRs will add higher layers on top of this trait (Connected to #287):

Diagnostics layer (typed events, value splitting at the 1,022-byte Azure limit)
Tracing layer (tracing_subscriber::Layer adapter)
Provisioning report accessor
Integration into libazureinit/src/logging.rs to replace the existing kvp.rs module

cadejacobson · 2026-03-09T19:41:00Z

libazureinit-kvp/src/lib.rs

+pub const HYPERV_MAX_VALUE_BYTES: usize = 2048;
+/// Azure key limit in bytes (policy/default preset).
+pub const AZURE_MAX_KEY_BYTES: usize = 512;
+/// Azure value limit in bytes (UTF-16: 511 characters + null terminator).


Do you have a reference for the null termination of this? I opened up that link in the previous file, but did not see a reference to null termination

I was basing this off of what we have in our existing kvp.rs --

const HV_KVP_EXCHANGE_MAX_KEY_SIZE: usize = 512; const HV_KVP_EXCHANGE_MAX_VALUE_SIZE: usize = 2048; const HV_KVP_AZURE_MAX_VALUE_SIZE: usize = 1022;

and my old notes from that KVP process, but I'll look for the official documentation where it says that!

doc/libazurekvp.md

libazureinit-kvp/src/hyperv.rs

Copilot

Pull request overview

Adds a new standalone workspace crate (libazureinit-kvp) that defines a storage abstraction (KvpStore) and a production Hyper-V pool-file implementation (HyperVKvpStore), along with updated technical reference documentation.

Changes:

Introduces libazureinit-kvp crate with KvpStore and KvpLimits (Hyper-V/Azure presets).
Implements Hyper-V binary pool-file backend with flock-based concurrency and stale-file truncation support.
Rewrites doc/libazurekvp.md as a technical reference for the new crate/API.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`Cargo.toml`	Adds `libazureinit-kvp` to the workspace members.
`libazureinit-kvp/Cargo.toml`	Defines the new crate and its dependencies (`fs2`, `sysinfo`).
`libazureinit-kvp/src/lib.rs`	Defines `KvpStore`, `KvpLimits`, and exports `HyperVKvpStore` plus size constants.
`libazureinit-kvp/src/hyperv.rs`	Implements Hyper-V pool-file encoding/decoding and the `KvpStore` backend, plus unit tests.
`doc/libazurekvp.md`	Updates documentation to describe the new crate’s record format, semantics, truncation behavior, and limits.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

libazureinit-kvp/src/hyperv.rs

doc/libazurekvp.md

libazureinit-kvp/src/hyperv.rs

doc/libazurekvp.md

libazureinit-kvp/src/lib.rs

cjp256 · 2026-03-10T02:10:49Z

libazureinit-kvp/src/lib.rs

+/// Key/value byte limits for write validation.
+#[derive(Clone, Copy, Debug, PartialEq, Eq)]
+pub struct KvpLimits {
+    pub max_key_size: usize,


let's inline these into the trait

and have each implementation define them as their own const.

if you want to keep KvpLimits, make it a trait and implement Azure & HyperV variants

cjp256 · 2026-03-10T02:11:37Z

libazureinit-kvp/src/lib.rs

+    /// Consumers (e.g. diagnostics, tracing layers) should call this
+    /// instead of hardcoding size constants, so the limits stay correct
+    /// regardless of the underlying implementation.
+    fn limits(&self) -> KvpLimits;


remove in favor of two consts.

cjp256 · 2026-03-10T02:14:32Z

libazureinit-kvp/src/lib.rs

+    ///
+    /// Keys are deduplicated using last-write-wins semantics, matching
+    /// the behavior of [`read`](KvpStore::read).
+    fn entries(&self) -> io::Result<HashMap<String, String>>;


can we expose one more entries_raw(), or dump() which yields a list of key/value pairs that aren't necessarily unique?

this may be useful for testing or some dump command

cjp256 · 2026-03-10T02:25:15Z

libazureinit-kvp/src/lib.rs

+/// Storage abstraction for KVP backends.
+///
+/// Semantics:
+/// - `write`: stores one key/value or returns validation/I/O error.


these details should be consolidated into the function docs. This should just be a high level doc for the trait.

e.g. key-value store that supports the semantics of Hyper-V's KVP while also being generic to support non-file-backed-pool-KVP in-memory implementation, for tests etc.

cjp256 · 2026-03-10T02:26:03Z

libazureinit-kvp/src/lib.rs

+    ///
+    /// Returns an error if:
+    /// - The key is empty.
+    /// - The key exceeds the configured maximum key size.


what are allowed values for the key characters? are there restrictions?

cjp256 · 2026-03-10T02:28:00Z

libazureinit-kvp/src/lib.rs

+
+    /// Write a key-value pair into the store.
+    ///
+    /// Returns an error if:


these errors should be defined here so we can be specific about what is being returned under what conditions

cjp256 · 2026-03-10T02:30:17Z

libazureinit-kvp/src/hyperv.rs

+//! - No record-count header or explicit record cap in this layer.
+//!
+//! ## Behavior summary
+//! - **`write`**: append-only; one record appended per call.


leave these to function docs

cjp256 · 2026-03-10T02:32:59Z

libazureinit-kvp/src/hyperv.rs

+//! - No record-count header or explicit record cap in this layer.
+//!
+//! ## Behavior summary
+//! - **`write`**: append-only; one record appended per call.


Keep the module docs relevant to the Hyper-V KVP pool / protocol and don't include any rust implementations details. The implementation should flow from this description and be detailed in the impl docs.

cjp256 · 2026-03-10T02:35:49Z

libazureinit-kvp/src/hyperv.rs

+    pub fn new_autodetect(path: impl Into<PathBuf>) -> Self {
+        let limits = if is_azure_vm(None) {


this is clever, but let's not do this. The caller should decide with implementation to work with if it's separated in two different implementations. If it's one implementation, then constrain it to a required bool in the new() call, e.g. new(azure=true)

cjp256 · 2026-03-10T02:56:41Z

libazureinit-kvp/src/lib.rs

+    /// - The key exceeds the configured maximum key size.
+    /// - The value exceeds the configured maximum value size.
+    /// - An I/O error occurs during the write.
+    fn write(&self, key: &str, value: &str) -> io::Result<()>;


we'll actually want there to be an implementation here with common validations and then a call-out to the backend-processing.

for example:

trait KvpStore { const MAX_KEY_SIZE: usize; const MAX_VALUE_SIZE: usize; fn backend_read(&self, key: &str) -> Result<String, KvpError>; fn backend_write(&self, key: &str, value: &str) -> Result<(), KvpError>; fn read(&self, key: &str) -> Result<String, KvpError> { Self::validate_key(key)?; self.backend_read(key) } fn write(&self, key: &str, value: &str) -> Result<(), KvpError> { Self::validate_key(key)?; Self::validate_value(value)?; self.backend_write(key, value) } fn validate_key(key: &str) -> Result<(), KvpError> { let actual = key.len(); let max = Self::MAX_KEY_SIZE; if actual > max { return Err(KvpError::KeyTooLarge { max, actual }); } Ok(()) } fn validate_value(value: &str) -> Result<(), KvpError> { let actual = value.len(); let max = Self::MAX_VALUE_SIZE; if actual > max { return Err(KvpError::ValueTooLarge { max, actual }); } Ok(()) } }

this doesn't preclude additional validations the backend might want to do, but keeps a shared implementation of the basics that will be common everywhere

cjp256 · 2026-03-10T03:03:52Z

libazureinit-kvp/src/hyperv.rs

+const AZURE_CHASSIS_ASSET_TAG_PATH: &str =
+    "/sys/class/dmi/id/chassis_asset_tag";
+
+fn is_azure_vm(tag_path: Option<&str>) -> bool {


I don't think we want this logic (yet) but if there is a reason to keep it, move it into its own module, e.g. platform.rs or identity.rs.

cjp256 · 2026-03-10T03:11:06Z

libazureinit-kvp/src/hyperv.rs

+    /// check-then-truncate sequence. If the lock cannot be acquired
+    /// immediately (another client holds it), the call returns `Ok(())`
+    /// without blocking.
+    pub fn truncate_if_stale(&self) -> io::Result<()> {


there are two parts to this, truncate() and pool_is_stale(). let's split them apart.

I think the truncate() method can be part of the trait definition and implementable with trait methods or backend callouts for backend specific bits. It simply empties out the store (clears the hash map or truncates the file).

is_stale() can be separate. The in-memory case would return False and HyperV/Azure would check mtime as done here.

the new() constructor for hyperv/azure can have a parameter to truncate_on_stale which checks if stale then truncates.

…ests

…it, clear() API

cadejacobson · 2026-03-11T18:41:08Z

libazureinit-kvp/src/azure.rs

+        truncate_on_stale: bool,
+    ) -> Result<Self, KvpError> {
+        Ok(Self {
+            inner: HyperVKvpStore::new(path, truncate_on_stale)?,


Overall, I think this composition setup works super well! This is the intended replacement for inheritance for Rust. My only request is that I would recommend updating inner to be a more descriptive term, maybe kvp_store, since I do not see many official references to using the inner name. This could get confusing if we ever extend this further.
https://trpl.rantai.dev/docs/part-iii/chapter-20/#2043-implementing-composition-in-rust

For example, if AzureKvpStore were ever put into a separate composition, the pattern of calling the composed object inner could remain and would result in a ExtendedAzureKvpStore.inner.inner.path calling, instead of ExtendedAzureKvpStore.azure_kvp_store.kvp_store.path which strikes me as more verbose.

Big fan of this - will make this change!

cjp256 · 2026-03-13T16:06:53Z

libazureinit-kvp/src/lib.rs

+    ValueTooLarge { max: usize, actual: usize },
+    /// The key contains a null byte, which is incompatible with the
+    /// on-disk format (null-padded fixed-width fields).
+    KeyContainsNull,


TIL rust strings can contain nulls

are values allowed to have null bytes? or are they also null-terminated/padded?

feat(kvp): add store trait and Hyper-V KVP storage crate

f435f70

peytonr18 force-pushed the probertson/kvp-store-trait branch from b134a98 to f435f70 Compare March 9, 2026 19:19

cadejacobson requested changes Mar 9, 2026

View reviewed changes

add Azure autodetect limits selection for HyperVKvpStore

5226e32

cadejacobson self-requested a review March 9, 2026 20:42

Improving libazurekvp.md clarity

c1440c9

peytonr18 requested a review from Copilot March 9, 2026 22:34

Copilot started reviewing on behalf of peytonr18 March 9, 2026 22:34 View session

Copilot AI reviewed Mar 9, 2026

View reviewed changes

cjp256 reviewed Mar 10, 2026

View reviewed changes

doc/libazurekvp.md Outdated Show resolved Hide resolved

cjp256 reviewed Mar 10, 2026

View reviewed changes

libazureinit-kvp/src/lib.rs Outdated Show resolved Hide resolved

cjp256 reviewed Mar 10, 2026

View reviewed changes

peytonr18 added 2 commits March 10, 2026 09:28

feat(kvp): harden Hyper-V record decode and refactor stale truncate t…

d63dd42

…ests

Refactor libazureinit-kvp: KvpStore trait, KvpError, HyperV/Azure spl…

62a52fe

…it, clear() API

cadejacobson reviewed Mar 11, 2026

View reviewed changes

cjp256 reviewed Mar 13, 2026

View reviewed changes

		pub fn new_autodetect(path: impl Into<PathBuf>) -> Self {
		let limits = if is_azure_vm(None) {

Conversation

peytonr18 commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

KvpStore trait

HyperVKvpStore

KvpLimits

Next Steps

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cjp256 Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

peytonr18 commented Mar 9, 2026 •

edited

Loading

`KvpStore` trait

`HyperVKvpStore`

`KvpLimits`

cjp256 Mar 10, 2026 •

edited

Loading