Refactor GUI to modern HTML/CSS and improve Docker setup#2456
Refactor GUI to modern HTML/CSS and improve Docker setup#2456CesarPetrescu wants to merge 3 commits intoelebumm:masterfrom
Conversation
Major changes: - Add Qwen3 TTS provider with authentication support - Remove local pyttsx3 TTS (replaced with cloud TTS) - Add real-time progress GUI with WebSocket updates - Comprehensive Docker setup with docker-compose - Updated README with new documentation New features: - Qwen TTS: Supports multiple speakers and languages - Progress GUI: Live step-by-step tracking at http://localhost:5000 - Docker: Full containerization with environment variables - Config: Example config file for easy setup Files added: - TTS/qwen_tts.py - Qwen3 TTS provider - progress_gui.py - Flask/SocketIO progress server - utils/progress.py - Progress tracking module - GUI/progress.html - Progress dashboard template - GUI/static/css/progress.css - Progress GUI styles - GUI/static/js/progress.js - WebSocket client - docker-compose.yml - Docker orchestration - docker-entrypoint.sh - Container startup script - config.example.toml - Example configuration https://claude.ai/code/session_01HLLH3WjpmRzvaoY6eYSFAD
Major changes: - Remove PRAW dependency and Reddit API credentials - Add no-OAuth Reddit scraper using public .json endpoints - No Reddit API keys required - simpler setup! New scraper features: - Uses Reddit's public .json endpoints (www.reddit.com/r/subreddit.json) - Configurable rate limiting via request_delay setting - Automatic retry with exponential backoff - Fetches posts and comments without authentication Files changed: - reddit/scraper.py (new) - No-OAuth Reddit scraper - reddit/subreddit.py - Updated to use scraper instead of PRAW - requirements.txt - Removed praw dependency - utils/.config.template.toml - Removed Reddit credentials - config.example.toml - Updated with scraper settings - docker-entrypoint.sh - Updated for no-auth setup - docker-compose.yml - Removed Reddit credential env vars - main.py - Updated exception handling Limitations: - Subject to Reddit's rate limiting (configurable delay) - ~1000 post cap per subreddit listing - Some comments may be missing in large threads https://claude.ai/code/session_01HLLH3WjpmRzvaoY6eYSFAD
- Add dashboard page with quick actions and progress overview - Add settings page for Qwen TTS, Reddit scraper, and video configuration - Add backgrounds page for uploading custom video/audio backgrounds - Add videos page for viewing and downloading generated content - Add TTS connection test endpoint - Update docker-compose for standalone operation - Create unified CSS and JavaScript for consistent UI experience https://claude.ai/code/session_01HLLH3WjpmRzvaoY6eYSFAD
There was a problem hiding this comment.
Pull request overview
This PR modernizes the Reddit Video Maker Bot by replacing OAuth-based Reddit API access with a public JSON endpoint scraper, implementing a new web-based progress tracking GUI with WebSocket support, switching from pyttsx to Qwen TTS, and improving Docker containerization. While the goals are ambitious and the new features are valuable, the implementation has several critical security vulnerabilities and breaking changes that must be addressed before merging.
Changes:
- Replaced PRAW OAuth authentication with a no-auth Reddit JSON scraper to eliminate API key requirements
- Added comprehensive web GUI with real-time progress tracking via WebSocket, background management, and video generation controls
- Switched default TTS engine from pyttsx to Qwen TTS with support for multiple languages and voices
- Enhanced Dockerfile with proper dependency management, Playwright browser installation, and optimized layer caching
Reviewed changes
Copilot reviewed 24 out of 25 changed files in this pull request and generated 34 comments.
Show a summary per file
| File | Description |
|---|---|
| reddit/scraper.py | New no-auth Reddit scraper using public .json endpoints |
| reddit/subreddit.py | Refactored to use new scraper instead of PRAW |
| TTS/qwen_tts.py | New Qwen TTS engine implementation |
| TTS/pyttsx.py | Removed deprecated pyttsx engine |
| utils/progress.py | New progress tracking system with WebSocket callbacks |
| progress_gui.py | Flask server for web GUI with REST API and SocketIO |
| main.py | Updated with progress tracking integration and Qwen TTS validation |
| GUI/*.html | Modern standalone HTML pages with vanilla JavaScript |
| GUI/static/js/*.js | Client-side progress tracking and API utilities |
| GUI/static/css/*.css | Modern dark theme styling |
| requirements.txt | Updated dependencies (removed praw/pyttsx3, added Flask-SocketIO/gevent) |
| Dockerfile | Enhanced with proper environment variables and system dependencies |
| docker-compose.yml | New compose configuration for containerized deployment |
| docker-entrypoint.sh | Entrypoint script for environment-based configuration |
| config.example.toml | Updated config template with Qwen TTS and scraper settings |
| README.md | Comprehensive rewrite with Docker instructions and new features |
| utils/.config.template.toml | Removed Reddit OAuth fields, added scraper and Qwen TTS config |
| video_creation/voices.py | Updated to import QwenTTS instead of pyttsx |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| submission, similarity_score = _get_undone_post( | ||
| sorted_wrappers, subreddit_name, similarity_scores=similarity_scores | ||
| ) | ||
| else: | ||
| wrappers = [SubmissionWrapper(post) for post in posts] | ||
| submission = _get_undone_post(wrappers, subreddit_name) |
There was a problem hiding this comment.
There's a mismatch between _get_undone_post return type and how it's used. On line 110-112, the function is called and expected to return a tuple (submission, similarity_score), but on line 115 it's called and expects just a submission. However, on line 276, only post is returned when similarity_scores is not None and i < len(similarity_scores), which would cause unpacking errors. The logic should explicitly return a tuple on line 279.
| app.config['MAX_CONTENT_LENGTH'] = 500 * 1024 * 1024 # 500MB max upload | ||
|
|
||
| # Configure SocketIO for real-time updates | ||
| socketio = SocketIO(app, cors_allowed_origins="*", async_mode="gevent") |
There was a problem hiding this comment.
Missing CORS configuration: The Flask-SocketIO is configured with cors_allowed_origins="*" which allows connections from any origin. This is a security risk in production environments. Consider restricting CORS to specific trusted origins or making it configurable via environment variables.
|
|
||
| [settings.tts] | ||
| voice_choice = { optional = false, default = "tiktok", options = ["elevenlabs", "streamlabspolly", "tiktok", "googletranslate", "awspolly", "pyttsx", "OpenAI"], example = "tiktok", explanation = "The voice platform used for TTS generation. " } | ||
| voice_choice = { optional = false, default = "qwentts", options = ["elevenlabs", "streamlabspolly", "tiktok", "googletranslate", "awspolly", "qwentts", "OpenAI"], example = "qwentts", explanation = "The voice platform used for TTS generation." } |
There was a problem hiding this comment.
The conversion of pyttsx to QwenTTS changes the default TTS provider, but there's no migration path mentioned for existing users. The config template still references "pyttsx" in comments and the README should clearly document this breaking change and migration steps for existing installations.
| wrappers: List[SubmissionWrapper], | ||
| subreddit_name: str, | ||
| similarity_scores: Optional[List[float]] = None, | ||
| ) -> Optional[RedditPost] | Tuple[Optional[RedditPost], float]: |
There was a problem hiding this comment.
The return type annotation uses the deprecated pipe syntax Optional[RedditPost] | Tuple[Optional[RedditPost], float] which mixes Optional and union types incorrectly. This should be Union[Optional[RedditPost], Tuple[Optional[RedditPost], float]] or better yet, Optional[Union[RedditPost, Tuple[RedditPost, float]]] to properly express that the function returns either None, a RedditPost, or a tuple.
| if similarity_scores is not None and i < len(similarity_scores): | ||
| return post, similarity_scores[i] |
There was a problem hiding this comment.
The function _get_undone_post can return different types (RedditPost or tuple) based on whether similarity_scores is provided, but when the wrapper object is used for the check_done call on line 273, it's still a SubmissionWrapper. The return statement on line 281 should return the RedditPost from the wrapper, not None when no undone post is found. Consider returning the tuple consistently when similarity_scores is provided, or refactoring to have clearer return types.
| if similarity_scores is not None and i < len(similarity_scores): | |
| return post, similarity_scores[i] | |
| if similarity_scores is not None: | |
| score = similarity_scores[i] if i < len(similarity_scores) else 0.0 | |
| return post, score |
| import os | ||
| import json | ||
| import time | ||
| from dataclasses import dataclass, field, asdict |
There was a problem hiding this comment.
Import of 'asdict' is not used.
| from dataclasses import dataclass, field, asdict | |
| from dataclasses import dataclass, field |
| Complete web interface for configuration, video generation, and progress tracking. | ||
| """ | ||
| import os | ||
| import json |
There was a problem hiding this comment.
Import of 'json' is not used.
| import json |
| import shutil | ||
| from pathlib import Path | ||
| from datetime import datetime |
There was a problem hiding this comment.
Import of 'shutil' is not used.
| import shutil | |
| from pathlib import Path | |
| from datetime import datetime | |
| from pathlib import Path | |
| from datetime datetime |
| Note: This approach is subject to rate limiting and may be blocked by Reddit. | ||
| For production use, consider using the official Reddit API with OAuth. | ||
| """ | ||
| import json |
There was a problem hiding this comment.
Import of 'json' is not used.
| import json |
| from praw.models import MoreComments | ||
| from prawcore.exceptions import ResponseException | ||
|
|
||
| from reddit.scraper import get_scraper, RedditPost, RedditComment, RedditScraperError |
There was a problem hiding this comment.
Import of 'RedditComment' is not used.
| from reddit.scraper import get_scraper, RedditPost, RedditComment, RedditScraperError | |
| from reddit.scraper import get_scraper, RedditPost, RedditScraperError |
Description
This PR modernizes the GUI infrastructure and improves the Docker containerization of the Reddit Video Maker Bot. The changes include:
GUI Refactoring
Converted Jinja2 templates to standalone HTML: Replaced the Flask template inheritance system with self-contained HTML files that use modern vanilla JavaScript for dynamic content loading
Redesigned backgrounds.html: Completely restructured the backgrounds management page with:
Added new dashboard.html: New landing page featuring:
Added new progress.html: Dedicated progress tracking page with:
Docker Improvements
--no-install-recommendsand cleanup of apt cacheTechnical Improvements
Dependencies
Testing Notes
http://localhost:5000https://claude.ai/code/session_01HLLH3WjpmRzvaoY6eYSFAD