Fix stuck scan bug, reorganize repo, and clean up docs (v2.5.67)#38
Merged
ttlequals0 merged 3 commits intomainfrom Mar 6, 2026
Merged
Fix stuck scan bug, reorganize repo, and clean up docs (v2.5.67)#38ttlequals0 merged 3 commits intomainfrom
ttlequals0 merged 3 commits intomainfrom
Conversation
Fix crash recovery session handling -- scans could get permanently stuck after a Celery task crash because the DB session was in a rolled-back state and recovery writes silently failed. Also improve stuck scan detection when Celery task state is unknown. Move all application modules into pixelprobe/ package so root only contains entry points (app.py, celery_worker.py) and build/config files. Update 73+ import paths across 43+ files. Delete legacy files, broken scripts, and obsolete docs. Fix documentation inaccuracies (wrong ports, stale version refs, SQLite references). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Read real-time scan progress from Redis in scan-status API endpoint (Redis is updated per-file by Celery workers; PostgreSQL lagged behind) - Write files_processed/estimated_total in all scan completion SQL UPDATEs - Add final Redis-to-DB sync in ui_progress_update_task before exit - Extract _mark_scan_completed() helper replacing 7 duplicated SQL blocks - Add get_scan_progress_redis() and _PROGRESS_KEY_PREFIX to progress_utils - Move get_scan_progress_redis import to module level in scan_routes - Update architecture docs to reflect Redis-first progress flow - Fix README: WebSocket->polling, stale media_checker import path Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move inline import of get_scan_progress_redis to module level in tasks.py - Add ui_session.refresh(scan_state) in _final_sync_redis_to_db to prevent stale ORM values after raw SQL updates from _mark_scan_completed - Remove trailing whitespace in scan_routes.py - Add 12 unit tests for get_scan_progress_redis, _final_sync_redis_to_db sync logic, and _mark_scan_completed SQL behavior Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scan_service.pynow callsdb.session.rollback()before recovery writes, preventing scans from getting permanently stuck after Celery task crashes (e.g.,psycopg2.DatabaseError). Also improved stuck scan detection inscan_routes.py(time-based fallback when task state is unknown) andscheduler.py(verify Celery task state, not just time threshold).get_scan_progress_redis()for reading Redis progress, and_mark_scan_completed()to consolidate 7 duplicated scan completion SQL blocks into one method.ui_session.refresh(scan_state)in_final_sync_redis_to_db()to prevent stale ORM values after raw SQL updates.tasks.py, removed trailing whitespace inscan_routes.py.models.py,auth.py,config.py,media_checker.py,scheduler.py,celery_config.py,version.py) into thepixelprobe/package. Root now only contains entry points (app.py,celery_worker.py) and build/config files. Updated 73+ import paths across 43+ files.database_migrations.py,init_db.py,operation_handlers.py,utils.py, entiremigrations/directory, broken shell scripts, obsolete patches, and outdated dev docs.Test plan
get_scan_progress_redis(),_final_sync_redis_to_db(), and_mark_scan_completed()from models import, etc.)ttlequals0/pixelprobe:2.5.67pushed to Docker Hub