fix(metadata): normalize backslashes in virtual paths to prevent FUSE open() deadlock#685
Conversation
) A backslash (0x5C) is a legal Linux filename byte but deadlocks the FUSE page-cache layer on open(): the kernel hangs in invalidate_inode_pages2 (folio_wait_bit_common) and the caller goes uninterruptible D-state until the host is rebooted. WriteFileMetadata ran filepath.Base on the raw virtualPath, so a path like Release/A.mkv\A.mkv was written as a .meta file named A.mkv\A.mkv.meta and served verbatim by ListDirectory over WebDAV, while DB lookups (nzbfilesystem.normalizePath) already converted backslashes. PR javi11#74 normalized the RAR/7zip/filesystem extraction layers but not the metadata-service chokepoint every import writes through. Normalize backslash to forward slash at the metadata path entry points so no backslash path is ever persisted or served. Adds a regression test. Existing .meta files predating this change need a one-off rename or re-import.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3fe3d5628c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| // WriteFileMetadata writes file metadata to disk | ||
| func (ms *MetadataService) WriteFileMetadata(virtualPath string, metadata *metapb.FileMetadata) error { | ||
| virtualPath = normalizeVirtualPath(virtualPath) |
There was a problem hiding this comment.
Normalize the shared deletion entry point
When a single-file import uses a name like Release/A.mkv\A.mkv, this line persists it as Release/A.mkv/A.mkv, but callers keep the original string in writtenPaths/health records (ScheduleHealthCheck stores p as the FilePath). I checked the cleanup/API paths and they call DeleteFileMetadataWithSourceNzb directly rather than this normalized wrapper, so deleting the health item or zombie record with the stored backslash path builds the old literal-backslash .meta path, silently removes nothing, and leaves the normalized metadata visible. Normalize inside DeleteFileMetadataWithSourceNzb itself (including the cache key) so all direct callers delete the same path that writes now create.
Useful? React with 👍 / 👎.
… chokepoint Replace the per-method normalizeVirtualPath calls with two private helpers (metaFilePath/metaDirPath) that every on-disk path flows through, so no method — current or future — can persist or serve a backslash (javi11#660). This also covers the methods the per-method approach missed: - RenameFileMetadata (renaming into a backslash path re-persisted a backslash .meta name, re-triggering javi11#660) - DeleteFileMetadataWithSourceNzb when called directly - MoveToCorrupted Side effect: ReadFileMetadata and GetMetadataFilePath now apply truncateFilename like WriteFileMetadata, fixing a latent read/write mismatch for >250-char names. Adds regression tests for the rename and direct-delete paths.
|
Thanks for tracking this down and writing it up so clearly @firestaerter3 — the root-cause analysis (metadata service being the chokepoint every import writes through, and the write/lookup disagreement vs I pushed a follow-up commit (
Added regression tests for the rename and direct-delete paths. One thing neither commit handles: existing |
Closes #660.
A file whose name contains a backslash deadlocks the FUSE page-cache layer on
open()— the kernel hangs ininvalidate_inode_pages2/folio_wait_bit_commonand the reader is stuck in uninterruptible D-state until the host is rebooted.Root cause
The metadata service persisted and served paths verbatim.
WriteFileMetadatarunsfilepath.Baseon the rawvirtualPath, so a path likeRelease/A.mkv\A.mkvlands on disk as a.metafile literally namedA.mkv\A.mkv.meta, andListDirectory/ListDirectoryAllserve that name over WebDAV. Database lookups (nzbfilesystem.normalizePath) already convert\to/, so writes and lookups disagreed. #74 normalized the RAR/7zip/filesystem extraction layers, but not the metadata-service chokepoint every import ultimately writes through.Fix
Normalize
\→/(matchingnzbfilesystem.normalizePath, so writes and lookups resolve to the same key) at the metadata-service path entry points, so no backslash path is ever persisted or served. Adds a regression test that writes a backslash path and asserts no persisted.metaname contains a backslash and that a read via the original backslash path still resolves.Note
This prevents new backslash paths. Existing
.metafiles from before this change still carry the backslash and need a one-off rename (or re-import) to recover.