Skip to content

fix(importer): skip files with missing yEnc segments instead of importing them corrupt (#681)#688

Merged
javi11 merged 1 commit into
mainfrom
session/exciting-hoover-20cbfd
Jun 17, 2026
Merged

fix(importer): skip files with missing yEnc segments instead of importing them corrupt (#681)#688
javi11 merged 1 commit into
mainfrom
session/exciting-hoover-20cbfd

Conversation

@javi11

@javi11 javi11 commented Jun 15, 2026

Copy link
Copy Markdown
Owner

What

Fixes #681. When a multi-segment file could not be size-normalized, parseFile swallowed the error and imported the file anyway using the NZB's raw yEnc-encoded byte counts. Those encoded sizes produce wrong segment offsets → corrupt media file.

Now: when normalization fails because a required segment is missing from all providers (errors.Is(err, nntppool.ErrArticleNotFound)), the file is skipped. The existing aggregation loop drops it and continues, so the rest of the release imports normally.

Why this scope

The reported trigger: the part-size probe fetches a single segment; if that article returns 430 on the provider used, normalization fails for the whole file and the corrupt fallback runs. The corruption only happens when we'd fall back to encoded NZB sizes for a file we can't verify.

The skip is intentionally scoped to not-found errors:

  • Missing article (430) → skip (fixes the corruption).
  • Present but non-yEnc / no usable part size → keep using the NZB-declared sizes (long-standing, non-corrupting behavior; importer integration battery relies on this).

To make detection uniform, the "known not found" branches in normalizeSegmentSizesWithYenc now wrap ErrArticleNotFound (NonRetryableError.Unwrap lets errors.Is traverse the fetch path too).

Changes

  • internal/importer/parser/parser.go — scoped skip-on-not-found in parseFile; not-found branches in normalizeSegmentSizesWithYenc wrap ErrArticleNotFound.
  • internal/importer/parser/parser_skip_test.go (new):
    • TestParseNzbSkipsFileThatFailsNormalization — a 430 on a needed segment skips that file; the healthy file is kept with normalized (decoded) sizes, not raw encoded values.
    • TestParseNzbKeepsFileWhenYencHeaderUnavailable — present-but-no-yEnc article is not skipped.
  • internal/importer/parser/parser_parsenzb_test.go — serve realistic yEnc for healthy single-segment fixtures.

Reviewer notes

  • The skip is per-parse and final. A 430 from a provider that actually has the article but isn't tried for the body fetch would now drop the file rather than corrupt it — the safe outcome. Cross-provider / multi-segment-probe recovery of such files was deliberately deferred (separate, larger change).

Verification

  • go test ./internal/importer/parser/... -race ✅, go vet
  • go test ./internal/importer/ -race (integration battery) ✅
  • make → exit 0, 0 failures ✅

…ting them corrupt (#681)

When a multi-segment file could not be size-normalized, parseFile swallowed
the error and imported the file anyway using the NZB's raw yEnc-encoded byte
counts. Those encoded sizes produce wrong segment offsets and yield a corrupt
media file. The trigger reported in #681: the part-size probe fetches a single
segment; if that article returns 430 on the provider used, normalization fails
for the whole file and the corrupt fallback path runs.

Skip the file instead when normalization fails because a required segment is
missing from all providers (errors.Is ErrArticleNotFound). The existing
aggregation loop drops it and continues, so the rest of the release imports
normally. A present-but-non-yEnc article (no usable part size) still falls back
to the NZB-declared sizes, preserving long-standing behavior. The "known not
found" branches in normalizeSegmentSizesWithYenc now wrap ErrArticleNotFound so
detection is uniform across the fetch and pre-stat paths.

Tests: add TestParseNzbSkipsFileThatFailsNormalization (430 -> file skipped,
healthy file kept with normalized sizes) and
TestParseNzbKeepsFileWhenYencHeaderUnavailable (present-but-no-yEnc -> kept).
Update parser_parsenzb_test.go to serve realistic yEnc for healthy fixtures.
@javi11 javi11 merged commit b9293d5 into main Jun 17, 2026
2 checks passed
@javi11 javi11 deleted the session/exciting-hoover-20cbfd branch June 17, 2026 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provider Issues

1 participant