Skip to content

fix(subtitle): decrypt LL-HLS VTT AES parts per-part#7881

Open
hongjun-bae wants to merge 2 commits into
video-dev:masterfrom
hongjun-bae:fix/ll-hls-vtt-aes-part-decrypt
Open

fix(subtitle): decrypt LL-HLS VTT AES parts per-part#7881
hongjun-bae wants to merge 2 commits into
video-dev:masterfrom
hongjun-bae:fix/ll-hls-vtt-aes-part-decrypt

Conversation

@hongjun-bae
Copy link
Copy Markdown
Collaborator

@hongjun-bae hongjun-bae commented May 28, 2026

Problem

`SubtitleStreamController` only decrypts AES-encrypted VTT data at full-segment boundaries via `_handleFragmentLoadComplete`. When low-latency parts are in use, each part arrives through the progress callback (`_handleFragmentLoadProgress`), which the base class leaves as an empty no-op for subtitles. Concretely:

  • `SubtitleStreamController` does not override `_handleFragmentLoadProgress`, so encrypted VTT parts never get decrypted on the fly.
  • `_handleFragmentLoadComplete` takes `FragLoadedData` (full segment), not `PartsLoadedData`.
  • `FragDecryptedData` has no `part` field, so a decrypted part cannot be routed downstream with its part anchor.

Net effect: encrypted VTT captions over LL-HLS either stay stuck in `FRAG_LOADING` for the tail parts of a partially-loaded segment, or — at best — only surface after a whole segment is assembled, losing the low-latency benefit.

#7626 covered the part-loading + segment-finding unification but the encrypted-VTT case was out of scope.

Changes

  1. `SubtitleStreamController` overrides `_handleFragmentLoadProgress` to decrypt each encrypted VTT part as an independent AES-CBC stream using the segment-level IV, then emits `FRAG_DECRYPTED` with the part reference.
  2. The full-segment path in `_handleFragmentLoadComplete` is reorganised to share a single `decryptPayload` helper with the part path (one decrypter lifecycle, one error path, single `shouldDecrypt` guard).
  3. `FragDecryptedData` gains a `part: Part | null` field; `base-stream-controller` (init segment) and the full-segment subtitle path are updated to emit `part: null` so existing consumers compile.
  4. A new `Decrypter` instance is constructed per call to avoid racing on the software-decrypter's shared remainder state when concurrent parts decrypt.

Test plan

  • Added unit tests in `tests/unit/controller/subtitle-stream-controller.ts` for `_handleFragmentLoadProgress`: guard cases (no part / empty payload / not encrypted), success path (asserts `FRAG_DECRYPTED` is emitted with the part reference and the decrypted payload), and failure path (asserts `FRAG_DECRYPT_ERROR` is emitted with the part reference).
  • Verified end-to-end against a live LL-HLS + AES-128 VTT stream: subtitles now appear on the part path without segment-duration latency, and the controller no longer stalls when subsequent ticks load the tail parts of a segment whose head parts were buffered earlier.
  • `npm run type-check` and `npm run lint` clean.

SubtitleStreamController only decrypts AES-encrypted VTT data at full-segment
boundaries via _handleFragmentLoadComplete. When low-latency parts are in use,
each part arrives through the progress callback (_handleFragmentLoadProgress),
which the base class leaves as an empty no-op for subtitles. Encrypted parts
therefore never fire FRAG_DECRYPTED on the part path, leaving subtitle parsing
stuck and adding a segment-duration latency to encrypted VTT captions.

This change overrides _handleFragmentLoadProgress in SubtitleStreamController
to decrypt each encrypted VTT part as an independent AES-CBC stream using the
segment-level IV, and emits FRAG_DECRYPTED with the part reference so the
timeline-controller can anchor cues at part.start when appropriate. The full-
segment path is reorganised to share the same decryptPayload helper.

A 'part: Part | null' field is added to FragDecryptedData and to the two
existing FRAG_DECRYPTED emit sites (base-stream-controller init-segment path
and subtitle-stream-controller full-segment path) so consumers can distinguish
part-level decryption from segment-level decryption.
@hongjun-bae hongjun-bae force-pushed the fix/ll-hls-vtt-aes-part-decrypt branch from 54ee47e to 6679509 Compare May 28, 2026 05:37
Comment on lines +388 to +389
// A new Decrypter is constructed per call so concurrent part decryptions do
// not race on the software-decrypter's shared remainder state.
Copy link
Copy Markdown
Collaborator

@robwalch robwalch May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is it possible to have concurrent part decryption?

Parts are loaded serially. This has never come up as an issue for encrypted audio or video parts handled in transmuxer.push.

A new Decrypter instance is constructed per call to avoid racing on the software-decrypter's shared remainder state when concurrent parts decrypt.

The remainder state may be shared, but software decryption is synchronous. If there is remainder data then you have other problems - you are not using it and you are not resetting/reinitializing the decrypter like the transmuxer is in other stream controllers on continuity change. I would expect that neither is the case, since you parts would need to be encrypted whole to be delivered before the segment is complete.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed — parts load serially (doFragPartsLoad recurses inside the prior part's .then(), and loadPart fires onProgress once per part), so there is no concurrent decryption and no shared-state race. Reverted to this.decrypter and removed the per-call instance and its rationale comment in 6a55fab.

On "encrypted whole": verified against the target stream — each part is a separate INDEPENDENT=YES resource with a static IV (no BYTERANGE), and the video renditions use byte-identical packaging and already decrypt per-part via transmuxer.push. Decrypting each part with the key's IV is correct here and mirrors the existing media-part path.

private shouldDecrypt(frag: Fragment): boolean {
const d = frag.decryptdata;
return !!(
frag.encrypted &&
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking frag.encrypted is redundant.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — decryptdata.key && decryptdata.iv && isFullSegmentEncryption(method) already implies the fragment is encrypted, so the frag.encrypted check is redundant. Removed in 6a55fab.

}
}
})
.finally(() => {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Promise.finally is not available in the minimum browser requirements.

Use this.decrypter. The base controller cleans up on destroy.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both addressed in 6a55fab. Promise.prototype.finally is ES2018 — below the documented minimum (Chrome 39+/Safari 9+) and the es2015 lib target — so I dropped it and reverted to this.decrypter, letting the base controller's destroy() handle teardown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants