Skip to content

Improve WebDAV compatibility and performance#381

Merged
AptS-1547 merged 6 commits into
masterfrom
feat/webdav-enhancement
Jul 5, 2026
Merged

Improve WebDAV compatibility and performance#381
AptS-1547 merged 6 commits into
masterfrom
feat/webdav-enhancement

Conversation

@AptS-1547

@AptS-1547 AptS-1547 commented Jul 5, 2026

Copy link
Copy Markdown
Member

Summary

  • add WebDAV HTTP validator/precondition handling and preserve RFC-compatible GET/HEAD behavior
  • optimize WebDAV PROPFIND Depth: 1 with batch resource, dead property, and lock discovery loading
  • split WebDAV path resolution into reader/writer-aware read and mutation paths
  • coalesce repeated WebDAV download audit records by account/file/request type/client window while keeping audit writes on the existing audit queue

Tests

  • cargo test --test test_webdav
  • cargo test --lib webdav::download_audit
  • cargo test --lib webdav::auth
  • cargo test --test test_refactor_contracts config
  • cargo test --lib db::repository::config_repo::tests::ensure_defaults_keeps_media_processing_registry_default_when_bootstrap_env_disabled
  • cargo check
  • bun run biome check src/i18n/locales/en/admin/settings-common.json src/i18n/locales/zh/admin/settings-common.json
  • ASTER_E2E_SKIP_BUILD=1 bunx playwright test e2e/webdav.spec.ts
  • ignored WebDAV client e2e for rclone/cadaver/curl

Closes #362
Closes #363
Closes #364
Closes #365

Summary by CodeRabbit

  • 新功能
    • 新增 WebDAV 下载审计合并时间窗口设置(默认 30 秒,可将值设为 0 禁用合并)。
    • Range GET/GET/HEAD 的 Last-Modified 与条件请求逻辑完善;PROPFIND 支持批量属性与锁发现,提升深层目录响应准确性。
  • Bug 修复
    • 文件夹路径缓存改为按影响范围精确失效,减少不必要刷新。
    • 修正 ETag/If-* 条件头匹配规则与 304/412 行为,使 Range 与审计合并结果更符合预期。

AptS-1738 added 5 commits July 5, 2026 16:25
…upport

- Add `src/utils/http_validators.rs` with shared HTTP validation helpers:
  - `if_match_header_matches`: strong ETag comparison per RFC 9110
  - `if_none_match_header_matches`: weak ETag comparison per RFC 9110
  - `format_http_date` / `parse_http_date`: HTTP-date serialization and parsing
  - `http_date_epoch_seconds`: SystemTime to epoch seconds for date comparison
- Remove duplicated ETag helpers from `webdav/protocol.rs` and `file_service/common.rs`, replacing them with calls to `http_validators`
- Add `evaluate_http_download_preconditions` to `webdav/protocol.rs` supporting `If-Match`, `If-None-Match`, `If-Modified-Since`, and `If-Unmodified-Since` with correct header precedence rules
- Update `webdav/transfer/mod.rs` to use `evaluate_http_download_preconditions` and include `Last-Modified` in GET/HEAD responses
- Add `get_range` stub to `TrailingErrorStreamDriver` in handler tests and add test `handle_get_range_uses_driver_range_without_opening_full_stream`
- Add integration tests for `Last-Modified` header presence, `If-Modified-Since`, and `If-Unmodified-Since` preconditions

Closed #362
Optimize PROPFIND performance for large directories by batching property and lock queries:

- Add `find_by_entities` to property repository for batch property lookup across multiple entities
- Implement `get_props_many` in DavFileSystem trait for batch dead property loading
- Implement `discover_many` in DavLockSystem trait for batch lock discovery
- Add PropfindPreload structure to preload all properties and locks before building responses
- Skip dead property loading when PROPFIND requests only standard live properties
- Skip lock discovery when PROPFIND does not request lockdiscovery
- Add integration test verifying large directory PROPFIND performance with live-only properties
- Add unit tests validating batch loading behavior and selective property/lock loading

closed #363
Split resolve_path_cached_in_scope into separate read and write variants to prevent stale cache hits after write operations. Write operations (PUT/MKCOL/COPY/MOVE/DELETE/LOCK) now use writer_db() for authoritative resolution, while read operations (GET/PROPFIND metadata) use reader_db() for better load distribution.

- Add resolve_path_cached_for_read_in_scope() that queries reader_db
- Keep resolve_path_cached_in_scope() pinned to writer_db for write preconditions
- Extract resolve_path_cached_in_scope_with_db() as internal implementation
- Add resolve_entity_for_read() helper for property queries on read paths
- Update validate_cached_folder_path() to accept explicit db connection
- Update validate_cached_parent() to accept explicit db connection
- Update load_cached_resolved_node() to accept explicit db connection
- Switch open_file(), read_dir(), metadata(), get_props(), get_props_bulk(), has_props(), get_quota() to use reader_db
- Add comprehensive test_webdav_immediate_read_after_write_operations() covering PUT→GET, COPY→GET, MOVE→GET, LOCK→PROPFIND, DELETE→GET sequences

Closed #364
…approach

Refactor storage change cache invalidation to use targeted key deletion instead of broad prefix-based invalidation where possible. This improves cache efficiency by only invalidating affected entries.

Changes:
- Add audience-aware WebDAV path cache prefix generation (personal/team/any)
- Introduce `CacheInvalidationTargets` struct to support both prefix and key-based invalidation
- Replace blanket folder path cache prefix invalidation with targeted key deletion for specific folder changes
- Add batch invalidation support for folder path chains
- Expose `folder_path_cache_key` for targeted cache invalidation
- Update cache invalidation logic to use event context instead of just event kind
- Add helper functions for personal/team-scoped cache prefix generation
- Remove unnecessary full cache invalidation calls from folder operations
- Update all tests to use event-based cache invalidation with proper scope context
Add configurable coalescing window for WebDAV download audit records to prevent log flooding from repeated reads by the same client.

**Features:**
- Add `webdav_download_audit_coalesce_window_secs` configuration (default 30s)
- Coalesce repeated downloads by account, file, request type (full/ranged), and client fingerprint
- Track WebDAV account ID through authentication and filesystem layers
- Separate coalescing for full vs ranged reads
- Support disabling coalescing by setting window to 0

**Implementation:**
- New `download_audit` module with cache-based deduplication logic
- Client fingerprint derived from IP address and User-Agent SHA256 hash
- Integrate audit identity (account_id, scope, root_folder_id) into download recording
- Add i18n labels for English and Chinese admin settings

**Testing:**
- Verify default 30-second coalescing reduces multiple range requests to single audit row
- Confirm full and ranged reads tracked independently
- Validate disabling coalescing (window=0) records every read
- Update existing auth tests to handle new account_id field

Closed #365
@coderabbitai

coderabbitai Bot commented Jul 5, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9ed78aff-c56f-4d26-847d-2a6ca11eb3d4

📥 Commits

Reviewing files that changed from the base of the PR and between f8af50c and c1616f4.

📒 Files selected for processing (7)
  • src/services/file_service/common.rs
  • src/webdav/dav.rs
  • src/webdav/db_lock_system.rs
  • src/webdav/fs/mod.rs
  • src/webdav/metadata.rs
  • src/webdav/props/mod.rs
  • tests/test_webdav.rs
🚧 Files skipped from review as they are similar to previous changes (4)
  • src/webdav/db_lock_system.rs
  • tests/test_webdav.rs
  • src/webdav/fs/mod.rs
  • src/webdav/props/mod.rs

📝 Walkthrough

Walkthrough

本次改动新增 WebDAV 下载审计合并窗口、按文件夹 ID 定向缓存失效、共享 HTTP 校验器与条件下载、PROPFIND 批量预加载,并将部分 WebDAV 只读路径切到 reader DB,同时补充相关测试与配置文案。

Changes

WebDAV 审计、缓存、条件请求与读取优化

Layer / File(s) Summary
下载审计合并窗口
src/config/definitions.rs, frontend-panel/src/i18n/locales/en/admin/settings-common.json, frontend-panel/src/i18n/locales/zh/admin/settings-common.json, src/webdav/download_audit.rs, src/webdav/fs/mod.rs, src/webdav/mod.rs, src/webdav/auth.rs, tests/test_webdav.rs
新增下载审计合并窗口配置与文案,WebDAV 下载审计改为按账号/文件/请求类型/指纹合并,认证结果携带 account_id,相关测试覆盖合并与关闭合并窗口行为。
文件夹路径缓存定向失效
src/services/folder_service/*, src/services/task_service/archive/extract/mod.rs, src/services/trash_service/common.rs, src/services/webdav_service.rs
新增按 folder_ids 批量失效路径链缓存的能力,并把创建、删除、更新、复制、清理与 purge 的失效调用改为定向删除。
存储变更事件缓存失效
src/services/storage_change_service.rs
按完整 StorageChangeEvent 计算 WebDAV 前缀与目录 key,分别执行前缀失效和批量删除,并更新事件覆盖测试。
HTTP 校验器与条件下载
src/utils/http_validators.rs, src/utils/mod.rs, src/services/file_service/common.rs, src/webdav/protocol.rs, src/webdav/transfer/mod.rs, tests/test_webdav.rs
抽取共享 ETag/日期校验工具,WebDAV 协议与 GET/HEAD 使用 Last-Modified 和 If-Modified-Since/If-Unmodified-Since,范围读取与条件请求测试同步更新。
PROPFIND 批量预加载
src/db/repository/property_repo.rs, src/webdav/dav.rs, src/webdav/db_lock_system.rs, src/webdav/metadata.rs, src/webdav/props/mod.rs
新增批量属性与锁发现接口,元数据携带 property_entity,PROPFIND 改为先预加载死属性与锁,再构建 multistatus。
WebDAV 只读路径切换到 reader DB
src/webdav/path_resolver.rs, src/webdav/fs/mod.rs, src/webdav/handler_tests/mod.rs, tests/test_webdav.rs
WebDAV 读路径改用 reader DB 与读专用路径解析,范围读取不再打开整流,补充即时读写一致性与大目录测试。

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant AsterDavFs
  participant DownloadAudit
  participant Cache
  participant AuditService

  Client->>AsterDavFs: GET/Range 请求
  AsterDavFs->>DownloadAudit: record_download(identity, kind)
  DownloadAudit->>Cache: set_bytes_if_absent(key)
  Cache-->>DownloadAudit: 是否首次占位
  alt 窗口内首次
    DownloadAudit->>AuditService: log_with_details
  else 窗口内重复
    DownloadAudit-->>AsterDavFs: 跳过记录
  end
Loading
sequenceDiagram
  participant Client
  participant PropfindHandler
  participant PropfindPreload
  participant AsterDavFs
  participant DbLockSystem

  Client->>PropfindHandler: PROPFIND (Depth:1)
  PropfindHandler->>PropfindHandler: collect_propfind_resources
  PropfindHandler->>PropfindPreload: load(resources)
  PropfindPreload->>AsterDavFs: get_props_many(paths)
  PropfindPreload->>DbLockSystem: discover_many(paths)
  PropfindHandler->>PropfindPreload: dead_props_for / locks_for
  PropfindHandler-->>Client: multistatus 响应
Loading

Possibly related PRs

Suggested labels: TypeScript

Poem

缓存别乱跑,定向清掉就好
审计别狂刷,窗口一开就稳了
GET/HEAD 记得带上 Last-Modified
PROPFIND 先预热,再慢慢摆盘
读路径去 reader,别再撞 writer
你小子这回,总算没把锅甩歪

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed 标题概括了本次 WebDAV 兼容性与性能改进的主线,和变更内容一致。
Description check ✅ Passed 摘要、测试计划和审阅备注基本齐全,虽然未严格按模板排版但信息足够完整。
Linked Issues check ✅ Passed #362#363#364#365 的核心目标都被对应实现和测试覆盖了。
Out of Scope Changes check ✅ Passed 未见明显与四个链接目标无关的改动,新增代码都在 WebDAV 兼容性、性能或支撑修复范围内。
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/webdav-enhancement

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@AptS-1547 AptS-1547 added Documentation Improvements or additions to documentation Enhancement New feature or request Rust Pull requests that update Rust code Priority: Medium Medium priority issue labels Jul 5, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
src/webdav/fs/mod.rs (1)

700-742: 🚀 Performance & Scalability | 🔵 Trivial | 🏗️ Heavy lift

PROPFIND 的批量只省了一半
get_props_many 里还是逐个 resolve_entity_for_read(...),而缓存命中的路径也会先 find_by_id,再通过 validate_cached_folder_path 做一轮 DB 校验。read_dir 目前只往下游传 path/meta,没把 folder/file id 带出来,所以大目录 Depth:1 还是会留下 N 次路径解析。要真吃到这轮优化,得把实体 id 一路传到这里,或者把属性预取并进 read_dir

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/webdav/fs/mod.rs` around lines 700 - 742, get_props_many still resolves
each DavPath one by one via resolve_entity_for_read, so PROPFIND Depth:1 remains
N path lookups despite the batch property fetch. Update the read path so
read_dir (or its callers) carries the entity identifiers needed here, and then
use those ids in get_props_many instead of per-path resolution; if that refactor
is too broad, fold the property prefetch into read_dir and keep the existing
property_repo::find_by_entities flow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/services/file_service/common.rs`:
- Around line 32-35: The ETag matching logic in if_none_match_matches_value has
changed semantics from the old case-insensitive comparison to
http_validators::if_none_match_header_matches, which now compares If-None-Match
against etag_value differently. Update this helper to preserve the previous
trim_matches('"').eq_ignore_ascii_case(etag_value) behavior, or add explicit
compatibility handling plus tests in if_none_match_matches_value so existing
blob.hash values continue to match as before.

In `@src/webdav/db_lock_system.rs`:
- Around line 521-565: The discover_many flow currently gathers all ancestors
into one lock_repo::find_ancestors query, which can exceed sqlx-sqlite parameter
limits on large Depth:1 directories. Update discover_many to batch or chunk
all_ancestors before calling lock_repo::find_ancestors, then merge the per-chunk
results back into locks_by_path while preserving the existing filtering and
ordering logic. Also add an integration test in tests/test_webdav.rs that
exercises a directory with more than 500 children so the chunking path is
covered.

---

Nitpick comments:
In `@src/webdav/fs/mod.rs`:
- Around line 700-742: get_props_many still resolves each DavPath one by one via
resolve_entity_for_read, so PROPFIND Depth:1 remains N path lookups despite the
batch property fetch. Update the read path so read_dir (or its callers) carries
the entity identifiers needed here, and then use those ids in get_props_many
instead of per-path resolution; if that refactor is too broad, fold the property
prefetch into read_dir and keep the existing property_repo::find_by_entities
flow.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ddd4e0af-1d8f-4a04-8ee2-26950f1bc7ff

📥 Commits

Reviewing files that changed from the base of the PR and between 49fa7bd and f8af50c.

📒 Files selected for processing (28)
  • frontend-panel/src/i18n/locales/en/admin/settings-common.json
  • frontend-panel/src/i18n/locales/zh/admin/settings-common.json
  • src/config/definitions.rs
  • src/db/repository/property_repo.rs
  • src/services/file_service/common.rs
  • src/services/folder_service/cache.rs
  • src/services/folder_service/copy.rs
  • src/services/folder_service/hierarchy.rs
  • src/services/folder_service/mod.rs
  • src/services/folder_service/mutation.rs
  • src/services/storage_change_service.rs
  • src/services/task_service/archive/extract/mod.rs
  • src/services/trash_service/common.rs
  • src/services/webdav_service.rs
  • src/utils/http_validators.rs
  • src/utils/mod.rs
  • src/webdav/auth.rs
  • src/webdav/dav.rs
  • src/webdav/db_lock_system.rs
  • src/webdav/download_audit.rs
  • src/webdav/fs/mod.rs
  • src/webdav/handler_tests/mod.rs
  • src/webdav/mod.rs
  • src/webdav/path_resolver.rs
  • src/webdav/props/mod.rs
  • src/webdav/protocol.rs
  • src/webdav/transfer/mod.rs
  • tests/test_webdav.rs
💤 Files with no reviewable changes (1)
  • src/services/folder_service/copy.rs

Comment thread src/services/file_service/common.rs
Comment thread src/webdav/db_lock_system.rs
@codecov

codecov Bot commented Jul 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@AptS-1547 AptS-1547 merged commit d6f4910 into master Jul 5, 2026
6 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Documentation Improvements or additions to documentation Enhancement New feature or request Priority: Medium Medium priority issue Rust Pull requests that update Rust code

Projects

None yet

2 participants