Skip to content

output: azure_blob: compression with zstd update#2309

Merged
eschabell merged 1 commit intofluent:masterfrom
nberlee:zstd
Mar 23, 2026
Merged

output: azure_blob: compression with zstd update#2309
eschabell merged 1 commit intofluent:masterfrom
nberlee:zstd

Conversation

@nberlee
Copy link
Copy Markdown

@nberlee nberlee commented Dec 19, 2025

Document the new zstd option for Azure Blob network compression and explain how compress_blob inherits its codec, clarifying precedence between transport and blob-level compression.

⚠️ Heads-up: the code enabling Zstandard support is still under review in fluent/fluent-bit#11202, so please keep this docs PR in waiting-on-code-merge until that lands.

Summary by CodeRabbit

  • Documentation
    • Expanded Azure Blob output docs to list both gzip and zstd compression options.
    • Clarified compression behavior: defaulting to GZIP when no codec is set and inheriting the chosen codec when compression is configured.
    • Added compatibility note: using compressed blobs with append-blob type yields a configuration error and prevents startup.

@nberlee nberlee requested review from a team as code owners December 19, 2025 13:34
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Dec 19, 2025

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5fb6103f-e687-4701-a42c-40a0c22b5155

📥 Commits

Reviewing files that changed from the base of the PR and between 5d1f7e5 and 51db65f.

📒 Files selected for processing (1)
  • pipeline/outputs/azure_blob.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • pipeline/outputs/azure_blob.md

📝 Walkthrough

Walkthrough

Updated Azure Blob output docs: compress now lists gzip and zstd; compress_blob description rewritten to describe generic compression, defaulting to GZIP when compress is unset, inheriting codec from compress, and failing on blob_type = appendblob. (49 words)

Changes

Cohort / File(s) Summary
Azure Blob Output Documentation
pipeline/outputs/azure_blob.md
Expanded compress to list gzip and zstd; rewrote compress_blob to (1) reference compression generically, (2) state GZIP default when compress is unset, (3) state inheritance of codec when compress is set, and (4) note incompatibility with blob_type = appendblob causing startup failure.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

Suggested labels

5.0

Suggested reviewers

  • patrick-stephens

Poem

🐰 I nibble on docs beneath the moonlight,
Zstd and gzip twinkle, cozy and bright,
Blobs learn to tuck in their compressed beds,
Clear notes guide bytes to soft, sleepy threads,
Hooray — documentation hops into sight! 🥕✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: updating Azure Blob compression documentation to include zstd support alongside gzip.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

CodeRabbit can suggest fixes for GitHub Check annotations.

Configure the reviews.tools.github-checks setting to adjust the time to wait for GitHub Checks to complete.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7fb7ce2 and 749e1f9.

📒 Files selected for processing (1)
  • pipeline/outputs/azure_blob.md (1 hunks)
🔇 Additional comments (1)
pipeline/outputs/azure_blob.md (1)

29-30: Well-documented compression options and inheritance behavior.

The documentation clearly explains:

  • Both compress (transport-level) and compress_blob (blob-level) compression support, with zstd now available
  • How compress_blob inherits the codec from compress when set, establishing clear precedence
  • The incompatibility constraint with blob_type = appendblob

These clarifications significantly improve the documentation. Once the underlying code merges in fluent-bit/fluent-bit#11202, verify that the documented behavior matches the implementation.

@esmerel
Copy link
Copy Markdown
Contributor

esmerel commented Dec 22, 2025

this looks fine to me, writing-wise. I don't know why the vale test is being picky, it's not supposed to fail out on a spelling issue. Pat's question remains though so I won't stamp it at this time.

@eschabell eschabell self-assigned this Jan 9, 2026
@eschabell
Copy link
Copy Markdown
Collaborator

@patrick-stephens can you suggest a change that would make sense for @nberlee to apply?

@patrick-stephens
Copy link
Copy Markdown
Contributor

@patrick-stephens can you suggest a change that would make sense for @nberlee to apply?

For me I just want to know what happens with the misconfiguration, does it fail explicitly or silently ignore or do something else? I know it's not something changed but as part of the update we should try to improve and capture that.

@nberlee
Copy link
Copy Markdown
Author

nberlee commented Mar 19, 2026

I will put this on draft, there is not a reason for review at this time as the feature is still not merged.

@nberlee nberlee marked this pull request as draft March 19, 2026 09:21
Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
@nberlee nberlee marked this pull request as ready for review March 22, 2026 17:02
@nberlee nberlee requested a review from eschabell as a code owner March 22, 2026 17:02
@nberlee
Copy link
Copy Markdown
Author

nberlee commented Mar 22, 2026

Rebased and addressed the comment. Since the zstd PR in fluent/fluent-bit has just been merged, I’ll update the PR status and remove the draft flag.

@eschabell
Copy link
Copy Markdown
Collaborator

Thanks @nberlee for the PR, merging!

@eschabell eschabell merged commit d1a3a63 into fluent:master Mar 23, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants