Skip to content

Library optimizations#401

Merged
therobbiedavis merged 9 commits intocanaryfrom
feature/library-optimizations
Mar 9, 2026
Merged

Library optimizations#401
therobbiedavis merged 9 commits intocanaryfrom
feature/library-optimizations

Conversation

@therobbiedavis
Copy link
Collaborator

Small optimization release. I noticed that for users with larger libraries, the library page would take a long time to load. This was due to some redundant and unnecessary checks. Also the the api endpoint was returning a lot of data that was not needed for the page. Additionally I have added url resolution normalizations to download client host/ip:port connection strings so the connection tests are more robust.

Added

  • Shared backend/frontend audiobook status helpers so list views can use a consistent status model without relying on full file metadata from /library
  • A dedicated slim library list DTO for GET /library
  • A shared download-client URI builder for normalizing host, scheme, port, and path handling
  • Backend regression tests covering:
    • slim /library payload behavior
    • wanted-flag correctness
    • NZBGet host normalization and queue-path handling
    • qBittorrent, SABnzbd, and Transmission host normalization

Changed

  • Slimmed GET /library from a hybrid list/detail payload into a lighter list response
  • Kept GET /library/{id} as the rich full-detail audiobook endpoint
  • Moved library list status evaluation to shared backend/frontend helpers instead of deriving it ad hoc in views
  • Switched the app from timer-based Wanted badge refreshes to store-driven updates backed by existing SignalR events
  • Updated the frontend library store to collapse concurrent /library requests into a single in-flight fetch
  • Reused cached library state for app-shell lookups and related UI flows instead of issuing redundant library requests
  • Standardized host/URL interpretation across NZBGet, qBittorrent, SABnzbd, and Transmission for both test/save and runtime monitor paths

Fixed

  • Slow /library responses caused by per-audiobook filesystem existence checks during list loading
  • Duplicate /library requests during startup and view initialization
  • Full-library polling churn for the Wanted badge
  • Extra DB work in audiobook detail loading by collapsing a two-query existence/fetch path into a single no-tracking query
  • NZBGet malformed host parsing that could produce errors like Name or service not known (http:80)
  • The same malformed host/URL bug class across other download clients when users pasted scheme/path data into host fields

Removed

  • Periodic full-library polling for the Wanted badge
  • Per-item filesystem probes from the main /library list path
  • Redundant client-side dependence on full files[] metadata in library list views for status calculation

Stop probing the filesystem when computing an audiobook's wanted flag and treat DB file records as the source of truth. Make ComputeWantedFlag static and return wanted based on presence of file records (no File.Exists checks or per-file logging). Simplify the GetAll endpoint to AsNoTracking, include Files, and order results by Title (removed QualityProfile include + retry fallback). Update frontend comment to trust server semantics for wanted status and add a unit test to prevent regression where a DB file record would be treated as desired only if the file exists on disk.
Introduce request deduplication for library fetches by adding an inFlightFetch promise to the library Pinia store so concurrent fetchLibrary() calls share a single API request. Update App.vue to use the library store (fetchLibrary and audiobooks) instead of calling the API directly to avoid duplicate calls and keep wanted badge/search logic aligned with store state. Update tests to initialize Pinia when mounting App.vue and add a new library-fetch.spec.ts that verifies concurrent fetch deduplication. Bump project versions to 0.2.56.
Frontend: add computeAudiobookStatus util, AudiobookStatus type and format helper; integrate into AudiobooksView and CollectionView; derive wantedCount from hydrated library store (remove polling) and sync via SignalR onConnected; set wanted/status when files change in library store; normalize download client host input in DownloadClientFormModal and update tests; add unit tests for audiobook status and activity badge.

Backend: add LibraryAudiobookListItemDto and change LibraryController.GetAll to return a slim payload with Wanted and Status computed (uses new AudiobookStatusEvaluator); add DownloadClientUriBuilder to centralize URI/authority construction and update Nzbget/Qbittorrent/Sabnzbd/Transmission adapters to use it; various logging and request improvements. Tests updated/added for adapters and controller behaviors.
Replace a broad `as any` assertion with `ReturnType<typeof useDownloadsStore>['downloads']` in fe/src/__tests__/WantedView.spec.ts to improve type safety for the mocked downloads array. This makes the test's typings accurate and helps catch type-related issues earlier.
Adds GetQueueAsync_NormalizesHostWithSchemeAndPath unit test to NzbgetAdapterTests.cs. The test uses a DelegatingHandlerMock and TestHttpClientFactory to capture the outgoing request URI when the DownloadClientConfiguration.Host contains a scheme and path (e.g. "http://192.168.50.111/nzbget"). It asserts the adapter normalizes the URI to the provided scheme/host, uses the configured port, and targets the "/xmlrpc" path, and that the returned queue is empty.
Copilot AI review requested due to automatic review settings March 8, 2026 23:11
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a6506172f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes the library page performance and download client connection handling. For large libraries, the /library endpoint was slow due to per-audiobook filesystem existence checks and excessive data returned. The PR introduces a slim list DTO, moves status computation server-side, and introduces a shared DownloadClientUriBuilder to normalize host/scheme/port handling across all download clients.

Changes:

  • Introduced a slim GET /library list response (LibraryAudiobookListItemDto) with server-computed status via AudiobookStatusEvaluator, replacing the old hybrid list/detail payload that included full file metadata and performed filesystem probes.
  • Added DownloadClientUriBuilder to normalize download client URI construction (handling embedded schemes, paths, and explicit port overrides) across NZBGet, qBittorrent, SABnzbd, and Transmission in both adapters and monitor service.
  • Moved audiobook status computation to shared helpers (AudiobookStatusEvaluator.cs backend, audiobookStatus.ts frontend) and switched the wantedCount badge from periodic polling to a store-driven computed property.

Reviewed changes

Copilot reviewed 31 out of 33 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
listenarr.api/Services/Adapters/DownloadClientUriBuilder.cs New shared URI builder normalizing scheme/host/port/path for all download client adapters
listenarr.api/Services/AudiobookStatusEvaluator.cs New shared backend service computing audiobook list status from DB data
listenarr.api/Models/LibraryAudiobookListItemDto.cs New slim DTO for GET /library list response
listenarr.api/Controllers/LibraryController.cs Refactored GetAll() to use slim DTO + server status; removed filesystem checks from ComputeWantedFlag
listenarr.api/Services/DownloadMonitorService.cs Replaced inline URL construction with DownloadClientUriBuilder calls
listenarr.api/Services/DownloadService.cs Replaced inline qBittorrent URL construction with DownloadClientUriBuilder
listenarr.api/Services/Adapters/NzbgetAdapter.cs Replaced inline URL construction with DownloadClientUriBuilder; removed credential-in-URL manual encoding
listenarr.api/Services/Adapters/QbittorrentAdapter.cs Replaced all inline URL constructions with DownloadClientUriBuilder.BuildAuthority
listenarr.api/Services/Adapters/SabnzbdAdapter.cs Replaced all inline URL constructions with DownloadClientUriBuilder.BuildUri
listenarr.api/Services/Adapters/TransmissionAdapter.cs Replaced inline URL construction in BuildBaseUrl with DownloadClientUriBuilder.BuildUri
fe/src/utils/audiobookStatus.ts New shared frontend utility for status computation and formatting
fe/src/types/index.ts Added AudiobookStatus type and status field to Audiobook interface
fe/src/stores/library.ts Added in-flight fetch deduplication; updated applyFilesRemoved to maintain status/wanted
fe/src/App.vue Switched wantedCount from polling to computed from library store; load library on mount
fe/src/views/library/AudiobooksView.vue Replaced local status computation with shared computeAudiobookStatus utility
fe/src/views/library/CollectionView.vue Replaced local status computation with shared computeAudiobookStatus utility
fe/src/components/domain/download/DownloadClientFormModal.vue Added normalizeHost to strip scheme/path from host field on save/test
tests/Listenarr.Api.Tests/NzbgetAdapterTests.cs New tests for NZBGet host normalization and queue path
tests/Listenarr.Api.Tests/SabnzbdAdapterTests.cs New test for SABnzbd host normalization
tests/Listenarr.Api.Tests/QbittorrentAdapterTests.cs New test for qBittorrent host normalization
tests/Listenarr.Api.Tests/TransmissionAdapterTests.cs New test for Transmission host normalization and custom RPC path
tests/Listenarr.Api.Tests/LibraryController_WantedFlagRegressionTests.cs New regression test for DB-backed wanted-flag logic
tests/Listenarr.Api.Tests/LibraryController_LibraryListSlimPayloadTests.cs New regression test for slim /library payload and status
fe/src/__tests__/audiobookStatus.spec.ts New frontend tests for computeAudiobookStatus
fe/src/__tests__/library-fetch.spec.ts New frontend test for concurrent fetch deduplication
fe/src/__tests__/AppActivityBadge.spec.ts Updated test stubs for new SignalR event and Pinia setup
fe/src/__tests__/DownloadClientFormModal.spec.ts Updated test to exercise normalizeHost
fe/src/__tests__/WantedView.spec.ts Fixed type cast to use proper store type
CHANGELOG.md Added release notes for version 0.2.57 (contains version mismatch)
package.json / package-lock.json / fe/package.json / fe/package-lock.json Version bump to 0.2.56
Files not reviewed (1)
  • fe/package-lock.json: Language not supported

You can also share your feedback on Copilot code review. Take the survey.

Add support for fileCount across frontend and backend, improve torrent/magnet handling, and expand tests. Frontend: add Audiobook.fileCount, update library store to compute fileCount and wanted/status logic, use fileCount in filter evaluators and custom filters, simplify wantedCount compute, and retry library sync on SignalR reconnect even if initial hydrate failed (new tests added/updated). Backend: include FilePath, FileSize, and FileCount in Library DTO and controller slim list payloads. Download client: introduce DownloadClientUriBuilder helpers (ResolveTorrentAddTarget, NormalizeMagnetLink, TryParseHttpOrHttpsAbsoluteUri) and update qBittorrent/Transmission adapters to uniformly handle magnet vs .torrent URLs and pre-download logic. Tests: add unit tests for audiobook status evaluator and DownloadClientUriBuilder and update existing adapter tests to reuse response objects.
Clear isMagnetTarget assignment when an indexer redirects to a magnet link in QbittorrentAdapter and TransmissionAdapter. The code now normalizes and logs the magnet URL without toggling the isMagnetTarget flag to avoid carrying redundant or potentially incorrect state into downstream logic.
Normalize and prefer a validated magnet link or an HTTP(S) torrent URL when adding torrents. Parse and normalize the magnet link, attempt to parse an absolute HTTP/HTTPS torrent URL, and prefer the HTTP URL for pre-downloading torrent bytes (so authenticated/private-tracker content can be added via file data). Use the downloaded result's magnet redirect when necessary and set torrentUrl appropriately. Move inline magnet-hash extraction into a new TryExtractMagnetHash helper and throw if neither a magnet nor a torrent URL is available. Adjusted download call to use the validated httpTorrentUrl and simplified related logic.
Introduce NormalizeTorrentUrl helper in qBittorrent and Transmission adapters to validate/normalize HTTP(S) torrent URLs and centralize parsing. Refactor pre-download flow: prefer pre-downloading a valid HTTP(S) .torrent when present (even if a magnet exists), consolidate torrentUrl selection, and add TryPredownloadTorrentFileAsync with safe exception handling and logging. Update tests to cover pre-download behavior and invalid-scheme validation, plus add test helpers for HTTP listener and response writing. This prevents accidental network calls for invalid schemes and reduces duplicated logic.
Copy link

@tsolo4ever tsolo4ever left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude AI Code Review — Library Optimizations #401

Overall this is a solid improvement — the slim DTO approach meaningfully reduces payload size and moving file-existence logic out of the list endpoint is the right call. A few things worth discussing before merge:

Potential regression: ComputeWantedFlag drops legacy FilePath check
LibraryController.cs:103-114

The simplified version only checks audiobook.Files (the AudiobookFile collection). Any book tracked via the legacy FilePath field with no corresponding AudiobookFile row will now show as Wanted even though the file exists on disk. Is this intentional? If there's no migration step to backfill AudiobookFile records from existing FilePath values, users upgrading from older versions could see previously-found books flip to missing.

wanted logic is duplicated
LibraryController.cs:682

GetAll() computes var wanted = a.Monitored && !hasFiles; inline, bypassing the ComputeWantedFlag static method defined earlier in the same file. They're equivalent now, but if one changes the other won't follow. Worth consolidating to a single call.

Redundant NoFile branch in AudiobookStatusEvaluator
AudiobookStatusEvaluator.cs:38-46

Both wanted == true and !hasAnyFile return NoFile. Since wanted is derived from files.Count == 0, the second branch can never be reached when the first fires. Either one is dead code, or the intent was to distinguish "monitored with no file" from "unmonitored with no file" — in which case !hasAnyFile should return a different status.

FilePath still in the slim DTO
LibraryAudiobookListItemDto.cs:22

The slim DTO kept the legacy FilePath string field. If the intent of this PR is to make AudiobookFile records the source of truth for file state, including FilePath in the list response perpetuates client-side reliance on the legacy field. Worth being explicit about whether it stays for backward compat or gets dropped. For reference, the current frontend reads filePath in at least three places (audiobookStatus.ts, library.ts, WantedView.vue) for file-existence checks — those callers should eventually migrate to use fileCount/status instead. We'll handle the adjustments needed on our end when we rebase our PR on top of this.

nit: activeDownloadStatuses array allocated on every request
LibraryController.cs:660-667

This array is recreated on every GET /library call. A private static readonly field would be cleaner.

nit: DeriveQualityLabel lossless detection is incomplete
AudiobookStatusEvaluator.cs

The lossless check covers flac, alac, wav but misses aiff, ape, dsd, wv (WavPack). Rare in audiobooks but worth noting for completeness.

Thanks for the work on this — the query split approach (audiobooks → files → quality profiles → downloads) is the right call for keeping the list endpoint fast. Consider a brief inline comment explaining why it's structured as 4 queries rather than joins, just to help future contributors understand the intent.

@therobbiedavis therobbiedavis merged commit 736446a into canary Mar 9, 2026
13 of 14 checks passed
@therobbiedavis therobbiedavis deleted the feature/library-optimizations branch March 9, 2026 01:07
tsolo4ever added a commit to tsolo4ever/Listenarr that referenced this pull request Mar 9, 2026
…nings

- audiobookStatus.ts: remove dead filePath/fileSize check inside hasFiles
  branch — was always quality-match since || hasFiles was always true
- WantedView.vue: fallback wanted logic now prefers fileCount (slim DTO)
  over files[] length, drops hasPrimaryFile (legacy filePath) check
- LibraryController.cs: add missing deleteFiles XML param tag (CS1573)
- ManualImportController.cs: null-forgiving on metadata (already guarded
  above, surfaced by stricter nullable settings from PR Listenarrs#401 merge)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
tsolo4ever added a commit to tsolo4ever/Listenarr that referenced this pull request Mar 9, 2026
…on 0.2.57)

Keep fork version 0.2.111 over upstream 0.2.57.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants