Skip to content

working upgrades #154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
soudaburger wants to merge 1 commit into
base: master
Choose a base branch
from
Open

working upgrades #154

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@
.idea
venv
backend/db.json
build.sh
version.txt
27 changes: 19 additions & 8 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:12 as build-stage
FROM node:18-alpine as build-stage

WORKDIR /frontend

Expand All @@ -9,19 +9,30 @@ ENV REACT_APP_BACKEND_URL="/"
RUN yarn install && yarn build


FROM tiangolo/uwsgi-nginx-flask:python3.10
FROM python:3.11-slim

ENV STATIC_INDEX 1
ENV CONFIG_DIR "/config"

COPY ./backend/requirements.txt/ /app
RUN pip install -r /app/requirements.txt
# Install Python dependencies
COPY ./backend/requirements.txt /app/
RUN pip install --no-cache-dir -r /app/requirements.txt

# Copy backend code
COPY ./backend /app

# Copy frontend build
COPY --from=build-stage /frontend/build /app/static
COPY --from=build-stage /frontend/build/static/css /app/static/css
COPY --from=build-stage /frontend/build/static/js /app/static/js

# Create config directory
RUN mkdir -p $CONFIG_DIR

# copied from here: https://github.com/se1exin/Cleanarr/issues/135#issuecomment-2091709103
RUN echo "buffer-size=32768" >> /app/uwsgi.ini
# Create startup script
RUN echo '#!/bin/bash\n\
set -e\n\
echo "Starting Flask app..."\n\
cd /app && FLASK_APP=main python3 -m flask run --host=0.0.0.0 --port=80' > /app/start.sh && chmod +x /app/start.sh

EXPOSE 80

CMD ["/app/start.sh"]
129 changes: 68 additions & 61 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,32 @@ A simple UI to help find and delete duplicate and sample files from your Plex se

> Note: At this time only Plex Content Libraries (TV/Movies) are supported.

## Recent Fixes (v1.1.1)

This version includes critical fixes for the delete functionality that was causing 500s and 502s:

### Backend Fixes

- **Fixed Delete Operation**: Resolved race conditions and database conflicts during media deletion
- **Improved Error Handling**: Better error handling and logging throughout the application
- **Database Thread Safety**: Added proper locking to prevent concurrent access issues
- **Updated Dependencies**: All Python packages updated to latest stable versions

### Frontend Fixes

- **Better Error Handling**: Improved API error handling and user feedback
- **Updated Dependencies**: All Node.js packages updated to latest stable versions
- **Enhanced UI**: Better error display and operation status feedback

### Infrastructure Improvements

- **Modern Docker Setup**: Updated to use Python 3.11 and Node.js 18
- **Nginx + uWSGI**: Replaced old uwsgi-nginx-flask with modern nginx + uWSGI setup
- **Health Checks**: Added health check endpoints for better monitoring

## Plex Setup
You need to check `Settings | Library | Allow media deletion` within your plex server’s settings

You need to check `Settings | Library | Allow media deletion` within your plex server's settings

You will need a Plex Token: [How to find your Plex Token](https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/)

Expand All @@ -17,17 +41,17 @@ This project is available as a docker container on [Docker Hub](https://hub.dock

You will need to set the correct parameters for your setup:

| Parameter | Function |
| ----- | --- |
| `-v /some/path/on/your/computer:/config` | (**required**) Volume mount for config directory |
| `-e PLEX_BASE_URL="plex_address"` | (**required**) Plex Server Address (e.g. http://192.169.1.100:32400) |
| `-e PLEX_TOKEN="somerandomstring"` | (**required**) A valid Plex token for your Plex Server ([How to find your Plex Token](https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/)) |
| `-e LIBRARY_NAMES="Movies"`| (**optional**) Name(s) of your Plex Libraries to search. Separate multiple library names with ";" character. E.g. `"Movies 1;Movies 2"`. Default value is **"Movies"** |
| `-e BYPASS_SSL_VERIFY=1` | (**optional**) Disable SSL certificate verification. Use this if your Plex Server has "Secure Connections: Required" and you are having issues connecting to it. (Thanks [@booksarestillbetter - #2](https://github.com/se1exin/cleanarr/issues/2)) |
| `-p 5000:80` | (**required**) Expose the UI via the selected port (in this case `5000`). Change `5000` to the port of your choosing, but don't change the number `80`. |
| `-e PAGE_SIZE=50` | (**optional**) To avoid plex timeouts, results are loaded in pages (or chunks). If you recieve Plex Timeout errors, try setting this parameter to a lower value. |
| `-e DEBUG=0` | (**optional**) To enable debug logging set `DEBUG` to `1` |
| `-e PLEX_TIMEOUT=7200` | (**optional**) modify the timeout for wrapper (Error : Failed to load content!) |
| Parameter | Function |
| ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-v /some/path/on/your/computer:/config` | (**required**) Volume mount for config directory |
| `-e PLEX_BASE_URL="plex_address"` | (**required**) Plex Server Address (e.g. http://192.169.1.100:32400) |
| `-e PLEX_TOKEN="somerandomstring"` | (**required**) A valid Plex token for your Plex Server ([How to find your Plex Token](https://support.plex.tv/articles/204059436-finding-an-authentication-token-x-plex-token/)) |
| `-e LIBRARY_NAMES="Movies"` | (**optional**) Name(s) of your Plex Libraries to search. Separate multiple library names with ";" character. E.g. `"Movies 1;Movies 2"`. Default value is **"Movies"** |
| `-e BYPASS_SSL_VERIFY=1` | (**optional**) Disable SSL certificate verification. Use this if your Plex Server has "Secure Connections: Required" and you are having issues connecting to it. (Thanks [@booksarestillbetter - #2](https://github.com/se1exin/cleanarr/issues/2)) |
| `-p 5000:80` | (**required**) Expose the UI via the selected port (in this case `5000`). Change `5000` to the port of your choosing, but don't change the number `80`. |
| `-e PAGE_SIZE=50` | (**optional**) To avoid plex timeouts, results are loaded in pages (or chunks). If you recieve Plex Timeout errors, try setting this parameter to a lower value. |
| `-e DEBUG=0` | (**optional**) To enable debug logging set `DEBUG` to `1` |
| `-e PLEX_TIMEOUT=7200` | (**optional**) modify the timeout for wrapper (Error : Failed to load content!) |

#### Example running directly with docker (with make)

Expand All @@ -46,6 +70,7 @@ CONFIG_MOUNT=/tmp/config make run
```

#### Example running directly with docker (manually)

```shell
# you can build and run manually
docker build -t=selexin/cleanarr:latest .
Expand All @@ -56,10 +81,11 @@ docker run \
-p 5000:80 \
-v /some/path/on/your/computer:/config \
selexin/cleanarr:latest

```

#### Example using Docker Compose

(Thanks @JesseWebDotCom - #8)

Note that environment variables should **not** be quoted when using docker-compose.yml format
Expand All @@ -85,82 +111,63 @@ services:
restart: unless-stopped
```


You can then access the UI in your browser at [http://localhost:5000/](http://localhost:5000/).

## Run from Source / Setup Development Environment
## Building from Source

To run from source you need to run two parts - the Python Backend and React Frontend.
### Prerequisites

First clone down this repo:
```
git clone https://github.com/se1exin/cleanarr
```
- Python 3.11+
- Node.js 18+
- Docker

## Backend
### Backend Setup

Requirements: `python3`

The backend is just a thin wrapper around the Python Plex API using `Flask` to serve requests. I recommend using a `virtualenv` to run.

You should change to the `backend` folder to get things running:
```
```bash
cd backend
```

Setup the python environment and dependencies:
```
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

Run the Backend:
```
PLEX_BASE_URL="plex_address" PLEX_TOKEN="somerandomstring" LIBRARY_NAMES="Movies" PLEX_TIMEOUT="7200" FLASK_APP=main python -m flask run
```

The backend will start and run from port `5000` on `localhost` (e.g. [http:localhost:5000](http:localhost:5000)).

If you are running on a remote server :
```
PLEX_BASE_URL="http://plex_address:32400" PLEX_TOKEN="somerandomstring" LIBRARY_NAMES="Movies" FLASK_APP=main python -m flask run --host=IP.remote.server
```
See [Flask's Docs for more run options (such as chaning the port)](https://flask.palletsprojects.com/en/1.1.x/cli/).

## Frontend

Requirements: `node`, `yarn`.
### Frontend Setup

You should change to the `frontend` folder to get things running:
```
```bash
cd frontend
npm install
npm run build
```

Setup the node environment and dependencies:
```
yarn install
```
### Docker Build

Run the Frontend development server:

>Note: change `REACT_APP_BACKEND_URL` to match where your backend is running at - **make sure to include the trailing slash!**
```
REACT_APP_BACKEND_URL="http://localhost:5000/" yarn start
```bash
docker build -t cleanarr:latest .
```

The frontend will now be available in your browser at [http:localhost:3000](http:localhost:3000).
## Troubleshooting

### Delete Operations Failing

## Screenshots
If you're still experiencing issues with delete operations:

![Demo of deleting duplicate movies](screenshots/demo.gif)
1. **Check Logs**: Enable debug logging by setting `DEBUG=1`
2. **Verify Permissions**: Ensure Plex has permission to delete files
3. **Check Disk Space**: Ensure there's sufficient disk space for database operations
4. **Database Issues**: If database corruption is suspected, delete the `db.json` file and restart

### Performance Issues

- Reduce `PAGE_SIZE` if experiencing timeouts
- Increase `PLEX_TIMEOUT` for large libraries
- Monitor memory usage during bulk operations

## Credits

Thanks to the following projects:

- [pkkid/python-plexapi](https://github.com/pkkid/python-plexapi)
- [tiangolo/uwsgi-nginx-flask-docker](https://github.com/tiangolo/uwsgi-nginx-flask-docker)

## License

MIT - see [LICENSE.md](https://github.com/se1exin/cleanarr/blob/master/LICENSE.md)
87 changes: 61 additions & 26 deletions backend/database.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import os
import threading

import time
from tinydb import TinyDB, where
from tinydb.table import Document

Expand All @@ -9,51 +9,86 @@
DELETED_SIZE_DOC_ID = 1
IGNORED_ITEMS_TABLE = 'ignored'


logger = get_logger(__name__)


class Database(object):
def __init__(self):
logger.debug("DB Init")
config_dir = os.environ.get("CONFIG_DIR", "") # Will be set by Dockerfile
self.local = threading.local()
self._lock = threading.RLock() # Add reentrant lock for thread safety
logger.debug("DB Init Success")

def get_db(self):
if not hasattr(self.local, 'db'):
config_dir = os.environ.get("CONFIG_DIR", "")
self.local.db = TinyDB(os.path.join(config_dir, 'db.json'))
db_path = os.path.join(config_dir, 'db.json')
try:
self.local.db = TinyDB(db_path)
logger.debug(f"Database initialized at {db_path}")
except Exception as e:
logger.error(f"Failed to initialize database: {e}")
raise
return self.local.db

def set_deleted_size(self, library_name, deleted_size):
logger.debug("library_name %s, deleted_size %s", library_name, deleted_size)
self.get_db().upsert(Document({
library_name: deleted_size
}, doc_id=DELETED_SIZE_DOC_ID))
with self._lock: # Use lock to prevent concurrent writes
try:
logger.debug("library_name %s, deleted_size %s", library_name, deleted_size)
db = self.get_db()
db.upsert(Document({
library_name: deleted_size
}, doc_id=DELETED_SIZE_DOC_ID))
logger.debug("Successfully updated deleted size for %s", library_name)
except Exception as e:
logger.error(f"Failed to set deleted size: {e}")
raise

def get_deleted_size(self, library_name):
logger.debug("library_name %s", library_name)
data = self.get_db().get(doc_id=DELETED_SIZE_DOC_ID)
if data is not None:
if library_name in data:
return data[library_name]
return 0
with self._lock: # Use lock for reads too to ensure consistency
try:
logger.debug("library_name %s", library_name)
db = self.get_db()
data = db.get(doc_id=DELETED_SIZE_DOC_ID)
if data is not None:
if library_name in data:
return data[library_name]
return 0
except Exception as e:
logger.error(f"Failed to get deleted size: {e}")
return 0 # Return 0 on error instead of crashing

def get_ignored_item(self, content_key):
logger.debug("content_key %s", content_key)
table = self.get_db().table(IGNORED_ITEMS_TABLE)
data = table.get(where('key') == content_key)
return data
with self._lock:
try:
logger.debug("content_key %s", content_key)
table = self.get_db().table(IGNORED_ITEMS_TABLE)
data = table.get(where('key') == content_key)
return data
except Exception as e:
logger.error(f"Failed to get ignored item: {e}")
return None

def add_ignored_item(self, content_key):
logger.debug("content_key %s", content_key)
table = self.get_db().table(IGNORED_ITEMS_TABLE)
table.insert({
'key': content_key
})
with self._lock:
try:
logger.debug("content_key %s", content_key)
table = self.get_db().table(IGNORED_ITEMS_TABLE)
table.insert({
'key': content_key
})
logger.debug("Successfully added ignored item: %s", content_key)
except Exception as e:
logger.error(f"Failed to add ignored item: {e}")
raise

def remove_ignored_item(self, content_key):
logger.debug("content_key %s", content_key)
table = self.get_db().table(IGNORED_ITEMS_TABLE)
table.remove(where('key') == content_key)
with self._lock:
try:
logger.debug("content_key %s", content_key)
table = self.get_db().table(IGNORED_ITEMS_TABLE)
table.remove(where('key') == content_key)
logger.debug("Successfully removed ignored item: %s", content_key)
except Exception as e:
logger.error(f"Failed to remove ignored item: {e}")
raise
Loading