Skip to content

lib: Update chunkio to 98988d25df15595d47af1ade49989f177dc42f9c#10754

Merged
edsiper merged 1 commit intomasterfrom
cosmo0920-update-chunkio-6ee11e0d775e784abf94ff95c4b2ca36a7768b06
Aug 20, 2025
Merged

lib: Update chunkio to 98988d25df15595d47af1ade49989f177dc42f9c#10754
edsiper merged 1 commit intomasterfrom
cosmo0920-update-chunkio-6ee11e0d775e784abf94ff95c4b2ca36a7768b06

Conversation

@cosmo0920
Copy link
Copy Markdown
Contributor

@cosmo0920 cosmo0920 commented Aug 19, 2025


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

T eliminate Shlwapi.lib dependency, we need to sync the current revision of chunkio from its upstream.

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • Bug Fixes

    • More reliable creation of nested directories on Windows, correctly handling existing directories.
    • Improved directory detection on Windows to reduce false negatives.
  • Chores

    • Removed a legacy Windows linking dependency to simplify builds and reduce installation issues.
    • No changes to public interfaces or behavior on non‑Windows platforms.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Aug 19, 2025

Walkthrough

Replaces Windows directory creation with a custom recursive implementation, changes directory-checking to use GetFileAttributesA, and removes the Shlwapi.lib link dependency. No public API changes or non-Windows behavior modifications.

Changes

Cohort / File(s) Summary
Windows mkpath recursive creation
lib/chunkio/src/cio_os.c
Adds cio_os_win32_make_recursive_path() that canonicalizes a path with _fullpath and iteratively calls CreateDirectoryA for each segment; cio_os_mkpath now uses this helper. Non-Windows branches unchanged.
Directory check & shlwapi removal (Windows)
lib/chunkio/src/win32/dirent.c, lib/chunkio/src/CMakeLists.txt
Adds path_is_directory() which uses GetFileAttributesA and replaces PathIsDirectoryA usage; removes dependency on shlwapi.h/Shlwapi.lib from the build while keeping Shell32.lib.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Caller
  participant cio_os as cio_os_mkpath (Windows)
  participant FS as Windows FS APIs

  Caller->>cio_os: mkpath(path)
  cio_os->>cio_os: _fullpath(path)
  alt _fullpath fails
    cio_os-->>Caller: return 1
  else success
    loop per path segment
      cio_os->>FS: CreateDirectoryA(partial)
      alt ERROR_ALREADY_EXISTS
        Note right of FS: continue
      else on error
        cio_os-->>Caller: return 1
      end
    end
    cio_os-->>Caller: return 0
  end
Loading
sequenceDiagram
  autonumber
  actor Client
  participant dirent as opendir (Windows)
  participant FS as GetFileAttributesA

  Client->>dirent: opendir(path)
  dirent->>FS: GetFileAttributesA(path)
  alt attributes invalid or not directory
    dirent-->>Client: return NULL
  else is directory
    dirent-->>Client: proceed with enumeration
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

I hopped through paths on Windows’ sod,
Crafted each folder with a nod.
Shlwapi shed from my small pack,
I check attributes and pave the track.
CreateDirectory hums — the chunks stay glad. 🐇✨

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 9326119 and 2e353ee.

📒 Files selected for processing (3)
  • lib/chunkio/src/CMakeLists.txt (1 hunks)
  • lib/chunkio/src/cio_os.c (2 hunks)
  • lib/chunkio/src/win32/dirent.c (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
  • lib/chunkio/src/CMakeLists.txt
  • lib/chunkio/src/win32/dirent.c
  • lib/chunkio/src/cio_os.c
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (28)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
  • GitHub Check: pr-compile-centos-7
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
  • GitHub Check: PR - fuzzing test
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch cosmo0920-update-chunkio-6ee11e0d775e784abf94ff95c4b2ca36a7768b06

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (4)
lib/chunkio/src/CMakeLists.txt (1)

26-26: Drop unnecessary Shell32.lib in chunkio

No Shell32-dependent APIs remain in lib/chunkio, so it’s safe to remove the Windows link. Update lib/chunkio/src/CMakeLists.txt:

@@ lib/chunkio/src/CMakeLists.txt
-  set(libs
-    ${libs}
-    Shell32.lib)
+  set(libs
+    ${libs}
+    )

Optionally, you can also clean up now-unused headers in lib/chunkio/src/cio_os.c (e.g. remove the #include <Shlobj.h> if no Shell APIs are called).

lib/chunkio/src/win32/dirent.c (1)

77-95: Make path_is_directory file-local and add a minimal guard

This helper isn’t used outside this file; give it internal linkage and defensively handle NULL/empty input.

-BOOL path_is_directory(LPCSTR pszPath) {
+static BOOL path_is_directory(LPCSTR pszPath) {
+    if (pszPath == NULL || *pszPath == '\0') {
+        return FALSE;
+    }
     DWORD dwAttrib = GetFileAttributesA(pszPath);

     if (dwAttrib == INVALID_FILE_ATTRIBUTES) {
         return FALSE;
     }

     if (dwAttrib & FILE_ATTRIBUTE_DIRECTORY) {
         return TRUE;
     }

     return FALSE;
 }
lib/chunkio/src/cio_os.c (2)

127-130: Prefer explicit 0/1 check over ERROR_SUCCESS

cio_os_win32_make_recursive_path returns 0/1, not Win32 error codes. Comparing against ERROR_SUCCESS is misleading. Use != 0.

-    if (cio_os_win32_make_recursive_path(path) != ERROR_SUCCESS) {
+    if (cio_os_win32_make_recursive_path(path) != 0) {
         return 1;
     }

52-90: Follow-up: You can also drop Shlobj.h include now

Since SHCreateDirectoryExA is no longer used, <Shlobj.h> can be removed to avoid implying a Shell32 dependency.

Proposed change outside the edited hunk (do this if the repo-wide search confirms no Shell32 usage):

C code (outside this hunk):

#ifdef _WIN32
-#include <Shlobj.h>
#endif
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 610a125 and 9326119.

📒 Files selected for processing (3)
  • lib/chunkio/src/CMakeLists.txt (1 hunks)
  • lib/chunkio/src/cio_os.c (2 hunks)
  • lib/chunkio/src/win32/dirent.c (1 hunks)
🔇 Additional comments (1)
lib/chunkio/src/win32/dirent.c (1)

100-102: LGTM: Replacing PathIsDirectoryA with GetFileAttributesA-based check

Early return when not a directory keeps behavior clear and removes Shlwapi dependency.

@cosmo0920
Copy link
Copy Markdown
Contributor Author

cosmo0920 commented Aug 19, 2025

This PR itself would be good because merged patch is sent as a PR but coderabbitai suggested more rigid approach of this issue. So, I just converted this PR as a draft and will revisit after the fluent/chunkio#110 patch is merged into chunkio upstream.

Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
@cosmo0920 cosmo0920 force-pushed the cosmo0920-update-chunkio-6ee11e0d775e784abf94ff95c4b2ca36a7768b06 branch from 9326119 to 2e353ee Compare August 20, 2025 07:01
@cosmo0920 cosmo0920 changed the title lib: Update chunkio to 6ee11e0d775e784abf94ff95c4b2ca36a7768b06 lib: Update chunkio to 98988d25df15595d47af1ade49989f177dc42f9c Aug 20, 2025
@cosmo0920 cosmo0920 marked this pull request as ready for review August 20, 2025 07:02
@edsiper edsiper merged commit ecdde29 into master Aug 20, 2025
55 checks passed
@edsiper edsiper deleted the cosmo0920-update-chunkio-6ee11e0d775e784abf94ff95c4b2ca36a7768b06 branch August 20, 2025 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants