Skip to content

Implement AO3 rawExtendedMeta#8

Open
chatterchats wants to merge 19 commits intofichub-cli-contrib:mainfrom
chatterchats:main
Open

Implement AO3 rawExtendedMeta#8
chatterchats wants to merge 19 commits intofichub-cli-contrib:mainfrom
chatterchats:main

Conversation

@chatterchats
Copy link
Copy Markdown

@chatterchats chatterchats commented Nov 27, 2025

Summary

This pull request implements fichub's AO3 rawExtendedMeta handling without major changes to the database format by normalizing to fit fichub's FFN handling.
It ensures that fields already introduce are fully normalized, stored, and applied consistently across insert and update paths.


What Changed

Database Updates

  • Added columns:
    • relationships
    • tags
  • Migration helper updated to auto-create columns on existing DBs

Normalization

  • rawExtendedMeta.relationship[] → relationships
  • rawExtendedMeta.freeform[] → tags
  • rawExtendedMeta.rating[] → rated
  • rawExtendedMeta.fandom[] → raw_fandom
  • rawExtendedMeta.category[] → genres
  • rawExtendedMeta.stats.words → words
  • rawExtendedMeta.stats.kudos→ favorites
  • rawExtendedMeta.stats.comments → reviews
  • rawExtendedMeta.stats.status → updated
  • Extract fic_id from AO3 URLs (/works/<id>)

Persistence Logic

Both insert and update paths now:

  • Prefer normalized values (item.get(...))
  • Fall back to process_extendedMeta(...) only if normalized values are missing

Testing Notes

Manual verification performed:

  • DB migration executes cleanly on existing databases
  • Imported AO3 example payload with relationships and tags
  • AO3 refresh replaces metadata correctly
  • Normalization overrides fallback parsing
  • AO3 tags and relationships persist correctly
  • Verified DB contains normalized values
  • Refreshed metadata to confirm overwrite behavior
  • Confirmed FFN records remain unchanged
  • No new runtime errors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant