Dev

CHANGELOG_EXTRA.md

@@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.8.9.1] - 2026.03.08
+
+### Changed
+
+- 合并官方 0.8.9 改动
+
 ## [0.8.8.1] - 2026.03.03
 
 ### Changed

backend/open_webui/config.py

@@ -319,6 +319,13 @@ def __getattr__(self, key):
     os.environ.get("ENABLE_OAUTH_SIGNUP", "False").lower() == "true",
 )
 
+OAUTH_REFRESH_TOKEN_INCLUDE_SCOPE = PersistentConfig(
+    "OAUTH_REFRESH_TOKEN_INCLUDE_SCOPE",
+    "oauth.refresh_token_include_scope",
+    os.environ.get("OAUTH_REFRESH_TOKEN_INCLUDE_SCOPE", "False").lower() == "true",
+)
+
+
 OAUTH_MERGE_ACCOUNTS_BY_EMAIL = PersistentConfig(
     "OAUTH_MERGE_ACCOUNTS_BY_EMAIL",
     "oauth.merge_accounts_by_email",
@@ -1921,7 +1928,7 @@ class BannerModel(BaseModel):
 - Only suggest follow-ups that make sense given the chat content and do not repeat what was already covered.
 - If the conversation is very short or not specific, suggest more general (but relevant) follow-ups the user might ask.
 - Use the conversation's primary language; default to English if multilingual.
-- Response must be a JSON array of strings, no extra text or formatting.
+- Response must be a JSON object with a "follow_ups" key containing an array of strings, no extra text or formatting.
 ### Output:
 JSON format: { "follow_ups": ["Question 1?", "Question 2?", "Question 3?"] }
 ### Chat History:
@@ -2242,20 +2249,36 @@ class BannerModel(BaseModel):
 ]
 
 DEFAULT_CODE_INTERPRETER_PROMPT = """
-#### Tools Available
-
-1. **Code Interpreter**: `<code_interpreter type="code" lang="python"></code_interpreter>`
-   - You have access to a Python shell that runs directly in the user's browser, enabling fast execution of code for analysis, calculations, or problem-solving.  Use it in this response.
-   - The Python code you write can incorporate a wide array of libraries, handle data manipulation or visualization, perform API calls for web-related tasks, or tackle virtually any computational challenge. Use this flexibility to **think outside the box, craft elegant solutions, and harness Python's full potential**.
-   - To use it, **you must enclose your code within `<code_interpreter type="code" lang="python">` XML tags** and stop right away. If you don't, the code won't execute. 
-   - When writing code in the code_interpreter XML tag, Do NOT use the triple backticks code block for markdown formatting, example: ```py # python code ``` will cause an error because it is markdown formatting, it is not python code.
-   - When coding, **always aim to print meaningful outputs** (e.g., results, tables, summaries, or visuals) to better interpret and verify the findings. Avoid relying on implicit outputs; prioritize explicit and clear print statements so the results are effectively communicated to the user.  
-   - After obtaining the printed output, **always provide a concise analysis, interpretation, or next steps to help the user understand the findings or refine the outcome further.**  
-   - If the results are unclear, unexpected, or require validation, refine the code and execute it again as needed. Always aim to deliver meaningful insights from the results, iterating if necessary.  
-   - **If a link to an image, audio, or any file is provided in markdown format in the output, ALWAYS regurgitate word for word, explicitly display it as part of the response to ensure the user can access it easily, do NOT change the link.**
-   - All responses should be communicated in the chat's primary language, ensuring seamless understanding. If the chat is multilingual, default to English for clarity.
-
-Ensure that the tools are effectively utilized to achieve the highest-quality analysis for the user."""
+#### Code Interpreter
+
+You have access to a Python code interpreter via: `<code_interpreter type="code" lang="python"></code_interpreter>`
+
+- The Python shell runs directly in the user's browser for fast execution of analysis, calculations, or problem-solving. Use it in this response.
+- You can use a wide array of libraries for data manipulation, visualization, API calls, or any computational task. Think outside the box and harness Python's full potential.
+- **You must enclose your code within `<code_interpreter type="code" lang="python">` XML tags** and stop right away. If you don't, the code won't execute.
+- Do NOT use triple backticks (```py ... ```) inside the XML tags — that is markdown formatting, not executable Python code.
+- **Always print meaningful outputs** (results, tables, summaries, visuals). Avoid implicit outputs; use explicit print statements.
+- After obtaining output, **provide a concise analysis, interpretation, or next steps** to help the user understand the findings.
+- If results are unclear or unexpected, refine the code and re-execute. Iterate until you deliver meaningful insights.
+- **If a link to an image, audio, or any file appears in the output, display it exactly as-is** in your response so the user can access it. Do not modify the link.
+- Respond in the chat's primary language. Default to English if multilingual.
+
+Ensure the code interpreter is effectively utilized to achieve the highest-quality analysis for the user."""
+
+# Appended to the code interpreter prompt only when engine is pyodide (not jupyter)
+CODE_INTERPRETER_PYODIDE_PROMPT = """
+
+##### Pyodide Environment
+
+- This Python environment runs via Pyodide in the browser. **Do not install packages** — `pip install`, `subprocess`, and `micropip.install()` are not available.
+- If a required library is unavailable, use an alternative approach with available modules. Do not attempt to install anything.
+
+##### Persistent File System
+
+- User-uploaded files are available at `/mnt/uploads/`. When the user asks you to work with their files, read from this directory.
+- You can also write output files to `/mnt/uploads/` so the user can access and download them from the file browser.
+- The file system persists across code executions within the same session.
+- Use `import os; os.listdir('/mnt/uploads')` to discover available files."""
 
 ####################################
 # Vector Database

backend/open_webui/env.py

@@ -790,6 +790,16 @@ def parse_section(section):
 except ValueError:
     WEBSOCKET_SERVER_PING_INTERVAL = 25
 
+WEBSOCKET_EVENT_CALLER_TIMEOUT = os.environ.get("WEBSOCKET_EVENT_CALLER_TIMEOUT", "")
+
+if WEBSOCKET_EVENT_CALLER_TIMEOUT == "":
+    WEBSOCKET_EVENT_CALLER_TIMEOUT = None
+else:
+    try:
+        WEBSOCKET_EVENT_CALLER_TIMEOUT = int(WEBSOCKET_EVENT_CALLER_TIMEOUT)
+    except ValueError:
+        WEBSOCKET_EVENT_CALLER_TIMEOUT = 300
+
 
 REQUESTS_VERIFY = os.environ.get("REQUESTS_VERIFY", "True").lower() == "true"
 

backend/open_webui/internal/db.py

@@ -102,11 +102,30 @@ def create_sqlcipher_connection():
         conn.execute(f"PRAGMA key = '{database_password}'")
         return conn
 
-    engine = create_engine(
-        "sqlite://",  # Dummy URL since we're using creator
-        creator=create_sqlcipher_connection,
-        echo=False,
-    )
+    # The dummy "sqlite://" URL would cause SQLAlchemy to auto-select
+    # SingletonThreadPool, which non-deterministically closes in-use
+    # connections when thread count exceeds pool_size, leading to segfaults
+    # in the native sqlcipher3 C library. Use NullPool by default for safety,
+    # or QueuePool if DATABASE_POOL_SIZE is explicitly configured.
+    if isinstance(DATABASE_POOL_SIZE, int) and DATABASE_POOL_SIZE > 0:
+        engine = create_engine(
+            "sqlite://",
+            creator=create_sqlcipher_connection,
+            pool_size=DATABASE_POOL_SIZE,
+            max_overflow=DATABASE_POOL_MAX_OVERFLOW,
+            pool_timeout=DATABASE_POOL_TIMEOUT,
+            pool_recycle=DATABASE_POOL_RECYCLE,
+            pool_pre_ping=True,
+            poolclass=QueuePool,
+            echo=False,
+        )
+    else:
+        engine = create_engine(
+            "sqlite://",
+            creator=create_sqlcipher_connection,
+            poolclass=NullPool,
+            echo=False,
+        )
 
     log.info("Connected to encrypted SQLite database using SQLCipher")
 

backend/open_webui/main.py

@@ -1457,46 +1457,52 @@ async def dispatch(self, request: Request, call_next):
 app.add_middleware(SecurityHeadersMiddleware)
 
 
-class APIKeyRestrictionMiddleware(BaseHTTPMiddleware):
-    async def dispatch(self, request: Request, call_next):
-        auth_header = request.headers.get("Authorization")
-        token = None
-
-        if auth_header:
-            parts = auth_header.split(" ", 1)
-            if len(parts) == 2:
-                token = parts[1]
-
-        # Only apply restrictions if an sk- API key is used
-        if token and token.startswith("sk-"):
-            # Check if restrictions are enabled
-            if request.app.state.config.ENABLE_API_KEYS_ENDPOINT_RESTRICTIONS:
-                allowed_paths = [
-                    path.strip()
-                    for path in str(
-                        request.app.state.config.API_KEYS_ALLOWED_ENDPOINTS
-                    ).split(",")
-                    if path.strip()
-                ]
-
-                request_path = request.url.path
-
-                # Match exact path or prefix path
-                is_allowed = any(
-                    request_path == allowed or request_path.startswith(allowed + "/")
-                    for allowed in allowed_paths
-                )
-
-                if not is_allowed:
-                    return JSONResponse(
-                        status_code=status.HTTP_403_FORBIDDEN,
-                        content={
-                            "detail": "API key not allowed to access this endpoint."
-                        },
+class APIKeyRestrictionMiddleware:
+    def __init__(self, app):
+        self.app = app
+
+    async def __call__(self, scope, receive, send):
+        if scope["type"] == "http":
+            request = Request(scope)
+            auth_header = request.headers.get("Authorization")
+            token = None
+
+            if auth_header:
+                parts = auth_header.split(" ", 1)
+                if len(parts) == 2:
+                    token = parts[1]
+
+            # Only apply restrictions if an sk- API key is used
+            if token and token.startswith("sk-"):
+                # Check if restrictions are enabled
+                if app.state.config.ENABLE_API_KEYS_ENDPOINT_RESTRICTIONS:
+                    allowed_paths = [
+                        path.strip()
+                        for path in str(
+                            app.state.config.API_KEYS_ALLOWED_ENDPOINTS
+                        ).split(",")
+                        if path.strip()
+                    ]
+
+                    request_path = request.url.path
+
+                    # Match exact path or prefix path
+                    is_allowed = any(
+                        request_path == allowed
+                        or request_path.startswith(allowed + "/")
+                        for allowed in allowed_paths
                     )
 
-        response = await call_next(request)
-        return response
+                    if not is_allowed:
+                        await JSONResponse(
+                            status_code=status.HTTP_403_FORBIDDEN,
+                            content={
+                                "detail": "API key not allowed to access this endpoint."
+                            },
+                        )(scope, receive, send)
+                        return
+
+        await self.app(scope, receive, send)
 
 
 app.add_middleware(APIKeyRestrictionMiddleware)
@@ -2269,6 +2275,7 @@ async def get_app_config(request: Request):
                 "user_count": user_count,
                 "code": {
                     "engine": app.state.config.CODE_EXECUTION_ENGINE,
+                    "interpreter_engine": app.state.config.CODE_INTERPRETER_ENGINE,
                 },
                 "audio": {
                     "tts": {

backend/open_webui/migrations/versions/8452d01d26d7_add_chat_message_table.py

@@ -21,6 +21,39 @@
 branch_labels: Union[str, Sequence[str], None] = None
 depends_on: Union[str, Sequence[str], None] = None
 
+BATCH_SIZE = 5000
+
+
+def _flush_batch(conn, table, batch):
+    """
+    Insert a batch of messages, falling back to row-by-row on error.
+
+    Tries a single bulk insert first (fast path). If that fails (e.g. due to
+    a duplicate key), falls back to individual inserts wrapped in savepoints
+    so the rest of the batch can still succeed.
+    """
+    savepoint = conn.begin_nested()
+    try:
+        conn.execute(sa.insert(table), batch)
+        savepoint.commit()
+        return len(batch), 0
+    except Exception:
+        savepoint.rollback()
+        # Batch failed - insert one-by-one to isolate the bad row(s)
+        inserted = 0
+        failed = 0
+        for msg in batch:
+            sp = conn.begin_nested()
+            try:
+                conn.execute(sa.insert(table).values(**msg))
+                sp.commit()
+                inserted += 1
+            except Exception as e:
+                sp.rollback()
+                failed += 1
+                log.warning(f"Failed to insert message {msg['id']}: {e}")
+        return inserted, failed
+
 
 def upgrade() -> None:
     # Step 1: Create table
@@ -88,18 +121,21 @@ def upgrade() -> None:
         sa.column("updated_at", sa.BigInteger()),
     )
 
-    # Fetch all chats (excluding shared chats which have user_id starting with 'shared-')
-    chats = conn.execute(
-        sa.select(chat_table.c.id, chat_table.c.user_id, chat_table.c.chat).where(
-            ~chat_table.c.user_id.like("shared-%")
-        )
-    ).fetchall()
+    # Stream rows instead of loading all into memory:
+    # - yield_per: fetches rows in chunks via cursor.fetchmany() (all backends)
+    # - stream_results: enables server-side cursors on PostgreSQL (no-op on SQLite)
+    result = conn.execute(
+        sa.select(chat_table.c.id, chat_table.c.user_id, chat_table.c.chat)
+        .where(~chat_table.c.user_id.like("shared-%"))
+        .execution_options(yield_per=1000, stream_results=True)
+    )
 
     now = int(time.time())
-    messages_inserted = 0
-    messages_failed = 0
+    messages_batch = []
+    total_inserted = 0
+    total_failed = 0
 
-    for chat_row in chats:
+    for chat_row in result:
         chat_id = chat_row[0]
         user_id = chat_row[1]
         chat_data = chat_row[2]
@@ -139,39 +175,49 @@ def upgrade() -> None:
             if timestamp < 1577836800 or timestamp > now + 86400:
                 timestamp = now
 
-            # Use savepoint to allow individual insert failures without aborting transaction
-            savepoint = conn.begin_nested()
-            try:
-                conn.execute(
-                    sa.insert(chat_message_table).values(
-                        id=f"{chat_id}-{message_id}",
-                        chat_id=chat_id,
-                        user_id=user_id,
-                        role=role,
-                        parent_id=message.get("parentId"),
-                        content=message.get("content"),
-                        output=message.get("output"),
-                        model_id=message.get("model"),
-                        files=message.get("files"),
-                        sources=message.get("sources"),
-                        embeds=message.get("embeds"),
-                        done=message.get("done", True),
-                        status_history=message.get("statusHistory"),
-                        error=message.get("error"),
-                        created_at=timestamp,
-                        updated_at=timestamp,
-                    )
+            messages_batch.append(
+                {
+                    "id": f"{chat_id}-{message_id}",
+                    "chat_id": chat_id,
+                    "user_id": user_id,
+                    "role": role,
+                    "parent_id": message.get("parentId"),
+                    "content": message.get("content"),
+                    "output": message.get("output"),
+                    "model_id": message.get("model"),
+                    "files": message.get("files"),
+                    "sources": message.get("sources"),
+                    "embeds": message.get("embeds"),
+                    "done": message.get("done", True),
+                    "status_history": message.get("statusHistory"),
+                    "error": message.get("error"),
+                    "usage": message.get("usage"),
+                    "created_at": timestamp,
+                    "updated_at": timestamp,
+                }
+            )
+
+            # Flush batch when full
+            if len(messages_batch) >= BATCH_SIZE:
+                inserted, failed = _flush_batch(
+                    conn, chat_message_table, messages_batch
                 )
-                savepoint.commit()
-                messages_inserted += 1
-            except Exception as e:
-                savepoint.rollback()
-                messages_failed += 1
-                log.warning(f"Failed to insert message {message_id}: {e}")
-                continue
+                total_inserted += inserted
+                total_failed += failed
+                if total_inserted % 50000 < BATCH_SIZE:
+                    log.info(
+                        f"Migration progress: {total_inserted} messages inserted..."
+                    )
+                messages_batch.clear()
+
+    # Flush remaining messages
+    if messages_batch:
+        inserted, failed = _flush_batch(conn, chat_message_table, messages_batch)
+        total_inserted += inserted
+        total_failed += failed
 
     log.info(
-        f"Backfilled {messages_inserted} messages into chat_message table ({messages_failed} failed)"
+        f"Backfilled {total_inserted} messages into chat_message table ({total_failed} failed)"
     )