Skip to content

Fix/recoverable checkpoint and 5.0.4#21

Merged
AthrunArthur merged 6 commits into
YeeZTech:developfrom
freeof123:fix/recoverable-checkpoint-and-5.0.4
Jul 9, 2026
Merged

Fix/recoverable checkpoint and 5.0.4#21
AthrunArthur merged 6 commits into
YeeZTech:developfrom
freeof123:fix/recoverable-checkpoint-and-5.0.4

Conversation

@freeof123

Copy link
Copy Markdown
Member

Summary

本 PR 在 develop(已合入 #20 测试改动)基础上,补齐 Node Recoverable 库侧修复,并 同步 main 5.0.4 浏览器更新。


一、Node — Recoverable 断点续传修复

背景

#20 已将 pause/resume 测试改为 README 标准用法(不手动 saveContext),但库侧存在:

  1. saveContext() 原地 open('w') 覆盖 + fire-and-forget,多次并发写同一文件 → context.dat meta JSON 损坏(multipause 在 loadContextBad control character
  2. Unsealerremaining 写成 Uint8ArraysaveContext 只认 Buffer → 密文尾巴被 JSON.stringify 塞进 meta(体积膨胀、写盘更慢)
  3. readItemCount 未持久化 / resume 未从 context 恢复 → 续传密文错位(Public Key could not be parsed
  4. Write 侧对 data 的错误 subarray 与 Unsealer remaining 语义不一致

设计要求:不依赖宿主或 pause 回调调 saveContext,pause 与断电走同一套自动落盘路径。

PipelineContextInFile — 落盘可靠性

改动 说明
原子写(临时文件) 内存拼好 payload → 写 context.dat.tmpfsyncrename 替换正式文件;写盘中断时旧 context.dat 仍完整可读
不再原地覆盖 避免并发/重叠 save 互相 truncate 把 meta 写花
save 合并排队 _saveDirty + _saveTail:流式解密中多次触发 save,合并为少量完整写入
loadContext 等待 pending save await _saveTail 后再读盘,避免 stage 结束立刻 load 读到半成品
Uint8Array → binary 区 Buffer / Uint8Array 均进 binary 段,不再误进 JSON

文件格式未变:仍是 [4B meta 长度][meta JSON][binary 区]readStart / writeStart / readItemCount 等指针仍在 meta JSON 里;仅写盘策略从「原地覆盖」改为「临时文件 + 原子替换」。

RecoverableWriteStream — 自动检查点

  • item 整块明文提交后:更新 readStart / writeStart / readItemCountsaveContext()
  • ws.end() / _final:truncate 明文到 writeStart 后再 save(与 pause、异常退出路径一致)
  • 检查点语义:只持久化已提交进度;半 item 的 data 清空,resume 从上一 item 边界重解(与断电回退一致)
  • 移除 Write 侧对 data 的错误 subarray

Unsealer — resume 自恢复

  • 构造时从 context.context 读取 readItemCount / readStart / writeStart(宿主不必再传 processedItemCount
  • data 统一写成 Buffer

#20 测试的关系

  • add test, multipause failed #20 提供标准 pause/resume + 浏览器 callback 测试
  • 本 PR 提供库侧修复,使以下用例无需 pause 内手动 save 即可通过:
    • test pipeline context with pause and resume from large file
    • test pipeline context with multiple random pause and resume

二、同步 main — 5.0.4(PR #18

改动 说明
onDownloadReady 浏览器下载 HEAD+tail 完成后、首个数据块进管道时回调(关 loading 蒙层)
createDownloadReadyTransformer ready 信号独立 TransformStream,与 progress transformer 同模式
signMessage 前缀修复 ypccrypto.common.js 修正 Ethereum 前缀,恢复 v4.x 兼容签名
版本号 5.0.35.0.4
测试 新增 test/progress.spec.mjsjest.config.browser.mjs 注册

变更文件(相对 develop)

  • src/node/PipelineConext.js — 原子落盘、save 排队、Uint8Array binary
  • src/node/Recoverable.js — 自动检查点、readItemCount 持久化
  • src/node/Unsealer.js — resume 从 context 恢复
  • src/browser/*src/common/progress.jssrc/common/ypccrypto.common.js — 5.0.4
  • package.jsonjest.config.browser.mjs、相关测试

Test plan

  • npm run test:node -- test/Recoverable.spec.js -t "^test pipeline context with pause and resume from large file$"
  • npm run test:node -- test/Recoverable.spec.js -t "^test pipeline context with multiple random pause and resume$"
  • npm run test:node -- test/Recoverable.spec.js --forceExit
  • npm run test:node -- test/PipelineContext.spec.js
  • npm run test:browser(含 progress / downloadCallbacks / downloadUnsealedCallbacks
  • npm run test:node -- test/BrowserCrypto.spec.mjs(signMessage 回归)

freeof123 and others added 6 commits June 23, 2026 16:51
Expose onDownloadReady so UI can dismiss loading overlays after HEAD+tail
validation; fix eth_hash_prefix to restore v4.x-compatible signatures.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Move download-ready signaling out of HttpSealedFileStream into a
TransformStream (same pattern as createProgressTransformer) and pipe it
in stream/blob download paths without modifying the HTTP stream class.

Co-authored-by: Cursor <cursoragent@cursor.com>
…nsformer-5.0.4

Feat/download ready transformer 5.0.4
Resolve jest.config.browser.mjs conflict: keep callback tests and progress.spec.mjs.

Co-authored-by: Cursor <cursoragent@cursor.com>
@AthrunArthur AthrunArthur merged commit 2f55d88 into YeeZTech:develop Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants