Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .github/workflows/backend-hermetic-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,18 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
cache-dependency-path: |
backend/requirements.txt
backend/testing/e2e/requirements.txt
python-version-file: backend/.python-version

- name: Set up uv
# astral-sh/setup-uv v7, pinned to an immutable commit for CI supply-chain stability.
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78
with:
enable-cache: true
cache-dependency-glob: backend/pylock.toml

- name: Install backend and e2e dependencies
working-directory: backend
run: |
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
python -m pip install -r testing/e2e/requirements.txt
run: uv pip sync pylock.toml --system

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep e2e installs aligned with Docker dependencies

Until the backend Dockerfiles are moved to pylock.toml, this workflow no longer tests the dependency set that is actually deployed: I checked backend/Dockerfile and it still builds from backend/requirements.txt via pip install -r /tmp/requirements.txt. In any PR that changes requirements.txt without the lock, or where an open requirement resolves differently under pip, hermetic e2e can pass here while the production image uses different package versions, so either keep this CI path on the same requirements inputs as Docker for now or switch Docker/enforcement to the lock in the same rollout.

Useful? React with 👍 / 👎.


- name: Prewarm tokenizer cache
working-directory: backend
Expand Down
1 change: 1 addition & 0 deletions backend/.python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.11.15
11 changes: 9 additions & 2 deletions backend/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,21 @@ Inherits all rules from the root `../AGENTS.md`. This file adds backend-specific

## Setup

Python 3.11 required (not 3.12+ — Dockerfile pins 3.11). Also needs FFmpeg, Opus (`opuslib`), Redis (optional).
Python 3.11 is required (not 3.12+ — Dockerfile pins 3.11). Backend local dev pins the exact interpreter in `.python-version` and uses `uv` for reproducible dependency sync. Also needs FFmpeg, Opus (`opuslib`), Redis (optional).

```bash
cp .env.template .env # Fill in required values (see .env.template for full list)
pip install -r requirements.txt
./scripts/sync-python-deps.sh # creates .venv from .python-version + pylock.toml
source .venv/bin/activate
uvicorn main:app --host 0.0.0.0 --port 8080
```

When intentionally changing backend Python dependencies, edit the relevant `requirements*.txt` input file and refresh the lock:

```bash
./scripts/update-python-lock.sh
```

Key env vars: `OPENAI_API_KEY` (LLM calls — not `OPENAI_ADMIN_KEY` which is billing-only), `DEEPGRAM_API_KEY` (STT), `ENCRYPTION_SECRET` (required for tests), `REDIS_DB_HOST` (cache/rate-limiting, fail-open without it), `ADMIN_KEY` (local dev auth bypass via token `ADMIN_KEY<uid>`), `SERVICE_ACCOUNT_JSON` (Firestore/GCS credentials).

## Directory Structure
Expand Down
1,612 changes: 1,612 additions & 0 deletions backend/pylock.toml

Large diffs are not rendered by default.

20 changes: 20 additions & 0 deletions backend/scripts/sync-python-deps.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
# Sync the backend local virtualenv from the checked-in uv pylock.
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT_DIR"

PYTHON_VERSION="$(tr -d '[:space:]' < .python-version)"
VENV_PATH="${VENV_PATH:-.venv}"

if ! command -v uv >/dev/null 2>&1; then
echo "uv is required. Install it from https://docs.astral.sh/uv/getting-started/installation/" >&2
exit 1
fi

uv python install "$PYTHON_VERSION"
uv venv --python "$PYTHON_VERSION" "$VENV_PATH"
uv pip sync pylock.toml --python "$VENV_PATH/bin/python"

echo "Backend dependencies synced into $ROOT_DIR/$VENV_PATH"
22 changes: 22 additions & 0 deletions backend/scripts/update-python-lock.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Refresh the backend uv pylock from the human-maintained requirements files.
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT_DIR"

PYTHON_VERSION="$(tr -d '[:space:]' < .python-version)"

if ! command -v uv >/dev/null 2>&1; then
echo "uv is required. Install it from https://docs.astral.sh/uv/getting-started/installation/" >&2
exit 1
fi

uv python install "$PYTHON_VERSION"
uv pip compile \
requirements.txt \
testing/e2e/requirements.txt \
--format pylock.toml \
--python "$PYTHON_VERSION" \
--output-file pylock.toml \
--custom-compile-command 'backend/scripts/update-python-lock.sh'
27 changes: 13 additions & 14 deletions docs/doc/developer/backend/Backend_Setup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -270,34 +270,33 @@ OAuth is required for user authentication. You need to configure both Google and
</Step>
<Step title="Set Up Python Virtual Environment" icon="python">
<Tip>
Using a virtual environment is strongly recommended to avoid dependency conflicts.
The backend uses `uv`, `.python-version`, and `pylock.toml` for reproducible local and CI installs.
</Tip>

```bash
# Create virtual environment with Python 3.11
python --version
python -m venv venv
```

<Tabs>
<Tab title="macOS/Linux">
```bash
source venv/bin/activate
# From backend/
./scripts/sync-python-deps.sh
source .venv/bin/activate
```
</Tab>
<Tab title="Windows">
<Tab title="Windows PowerShell">
```powershell
venv\Scripts\activate
# From backend/
bash ./scripts/sync-python-deps.sh
.venv\Scripts\Activate.ps1
```
</Tab>
</Tabs>

You should see `(venv)` at the beginning of your command prompt.
You should see `(.venv)` at the beginning of your command prompt.
</Step>
<Step title="Install Python Dependencies" icon="download">
<Step title="Update Python Dependencies" icon="download">
When intentionally changing backend dependencies, edit `requirements.txt` or `testing/e2e/requirements.txt`, then refresh the lock:

```bash
pip install PyOgg
pip install -r requirements.txt
./scripts/update-python-lock.sh
```
</Step>
<Step title="Create Environment File" icon="file">
Expand Down
Loading