use multi-threaded XZ decompression via liblzma

[bennyz]

Replace xz2 crate with liblzma 0.4 (parallel feature) to enable
lzma_stream_decoder_mt()

This helps remove the decompression bottleneck from XZ decompression
speeding flashing up by ~4x

[coderabbitai]

Warning

Rate limit exceeded

@bennyz has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 40 minutes and 29 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 40 minutes and 29 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f3bc5a9f-7b88-4fa4-a80b-672f6d43bd0f

📥 Commits

Reviewing files that changed from the base of the PR and between dc1ed62 and dc76f94.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (7)

• Cargo.toml
• src/fls/decompress.rs
• src/fls/from_url.rs
• src/fls/magic_bytes.rs
• src/fls/oci/from_oci.rs
• src/fls/stream_utils.rs
• tests/common/mod.rs

📝 Walkthrough

Walkthrough

The PR replaces the xz2 dependency with liblzma (enabling parallel feature) and refactors decompression from external subprocesses to in-process handling. New functions detect compression types from URLs and spawn dedicated threads for multithreaded XZ, Gzip, and pass-through decompression. Stream utilities were enhanced to report consumption progress.

Changes

Cohort / File(s)	Summary
Dependency Management Cargo.toml	Replaced xz2 = "0.1" with liblzma = { version = "0.4", features = ["parallel"] } to enable multithreaded compression support.
Core Decompression Logic src/fls/decompress.rs	Added get_compression_from_url() to detect compression type from file extensions and start_inprocess_decompressor() to spawn a decompression thread with runtime decoder selection (liblzma, flate2, or pass-through).
URL-based Flashing Refactor src/fls/from_url.rs	Replaced external decompressor process spawning with in-process decompression; removed subprocess I/O wiring and error handling, updated consumer to read from decompressed bytes channel instead of decompressor stdout, and simplified completion synchronization.
Decoder Import Updates src/fls/magic_bytes.rs, src/fls/oci/from_oci.rs, tests/common/mod.rs	Changed XZ decoder imports from xz2::read::XzDecoder and xz2::write::XzEncoder to corresponding liblzma equivalents across test and format-detection code.
Stream Progress Reporting src/fls/stream_utils.rs	Added optional progress-reporting field to ChannelReader struct and with_progress() builder method to track consumed bytes per chunk during decompression.

Sequence Diagram

sequenceDiagram
    participant dl as Download Loop
    participant buf as Buffer<br/>(buffer_rx)
    participant decomp as Decompressor<br/>Thread
    participant cons as Consumer<br/>Task
    participant dest as Destination<br/>(SSD)

    dl->>buf: Stream bytes chunks
    rect rgba(100, 200, 100, 0.5)
    Note over decomp: In-process decompression
    buf->>decomp: Read Bytes from buffer_rx
    decomp->>decomp: Decode chunk<br/>(liblzma/flate2)
    end
    decomp->>cons: Send decompressed Vec<u8>
    cons->>cons: Detect format &<br/>parse sparse data
    cons->>dest: Write to block device
    decomp->>decomp: Report progress<br/>via consumed_tx

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• fix OOM in fast networks #7: Introduces ByteBoundedReceiver and ChannelReader types that this PR builds upon and extends with progress reporting.

Suggested reviewers

• mangelajo

Poem

🐰 A hop toward swiftness, compression now threads,
From subprocess spawns to in-process reads,
Liblzma parallel spins with glee,
Decompressed chunks flow wild and free! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: replacing xz2 with liblzma to enable multi-threaded XZ decompression, which aligns with the core objective of the PR.
Description check	✅ Passed	The description is directly related to the changeset, explaining the motivation (enabling multi-threaded decompression via liblzma) and expected performance benefit (~4x speedup).
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/fls/decompress.rs`:
- Around line 127-130: The Compression::Zstd branch in decompress.rs currently
returns a static "install zstdcat" error but get_compression_from_url routes
.zst/.zstd URLs here, so the message is misleading and no external fallback is
used; either implement an external fallback by invoking zstdcat (use
std::process::Command in the function handling decompression, detect absence of
the binary and surface a clear error, stream input/output and propagate exit
status) or update the error text in the Compression::Zstd return to explicitly
state "URL Zstd flashing is unsupported" (and keep a comment referencing
get_compression_from_url), choosing one approach and ensuring the symbol
Compression::Zstd in decompress.rs and the caller used by
get_compression_from_url are updated accordingly.
- Around line 83-90: get_compression_from_url currently takes the full URL
including query strings/fragments, so filenames like
"image.xz?X-Amz-Signature=..." yield Compression::None; update
get_compression_from_url to first strip any query string or fragment by
splitting the input on '?' and '#' and using the part before them, then extract
the extension (rsplit('.') etc.) and match to Compression (Gzip, Xz, Zstd, None)
as before; this ensures signed URLs and URLs with fragments are detected
correctly.
- Around line 117-123: The MT XZ decoder currently disables memory limits by
calling memlimit_threading(u64::MAX) and memlimit_stop(u64::MAX) on
liblzma::stream::MtStreamBuilder, which is unsafe for untrusted firmware; change
this to use a conservative bounded limit instead (e.g., a constant
DEFAULT_XZ_MEMLIMIT) or read a configurable limit from BlockFlashOptions (add an
xz_memlimit or xz_max_memory field), validate it, and pass that value into
memlimit_threading(...) and memlimit_stop(...); leave the rest of the
MtStreamBuilder/decoder creation and the XzDecoder::new_stream(channel_reader,
stream) call intact, and ensure the option has a sensible default so callers
that don’t opt-in get safe behavior.

In `@src/fls/oci/from_oci.rs`:
- Line 16: The OCI tar-entry XZ extraction paths still instantiate
single-threaded decoders with XzDecoder::new(); replace those constructions with
the multi-threaded builder pattern used in src/fls/decompress.rs by importing
liblzma::MtStreamBuilder and creating decoders via MtStreamBuilder::new_stream()
(mirroring the same options/unwrap/error handling used in decompress.rs),
updating the two places that call XzDecoder::new() to use the
MtStreamBuilder::new_stream() stream wrapper instead so OCI extraction uses the
MT decoder.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d5b19a2c-d622-443f-9e43-05306f3326b8

📥 Commits

Reviewing files that changed from the base of the PR and between 6565ee0 and dc1ed62.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (7)

• Cargo.toml
• src/fls/decompress.rs
• src/fls/from_url.rs
• src/fls/magic_bytes.rs
• src/fls/oci/from_oci.rs
• src/fls/stream_utils.rs
• tests/common/mod.rs