Skip to content

agrawal-2005/PyRuntimeForge

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

13 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

PyRuntime Forge

Python 3.9+ Flask SocketIO Kubernetes MongoDB Atlas

A browser-based Python runtime platform that gives each user an isolated Kubernetes container, persistent filesystem state, and interactive stdin support.

Features β€’ Screenshots β€’ Tech Stack β€’ Project Structure β€’ Installation and Setup β€’ Environment Variables β€’ API Endpoints β€’ How It Works β€’ Known Limitations


Overview

PyRuntime Forge lets users register, log in, and run Python code inside their own Kubernetes pod.

The current execution model is:

command_to_exec = ["python", "-c", command]

So the platform is:

  • filesystem-stateful
  • container-stateful
  • not Python-RAM-stateful across separate executions

This means:

  • files created in a user's pod remain available between runs
  • interactive programs using input() now work through the browser UI
  • Python variables such as x = 10 do not survive a new click on Execute

Features

  • Per-user isolated Python runtime in Kubernetes
  • User registration and login backed by MongoDB Atlas
  • Python code execution directly from the browser
  • Interactive stdin support for input()-based programs
  • Streaming stdout and stderr output using Socket.IO
  • Persistent filesystem inside the user's pod
  • File-based data science workflows across multiple executions
  • Pre-built user pod image with NumPy, Pandas, scikit-learn, and Matplotlib
  • Test-case suite for validating project behavior
  • Documentation comparing PyRuntime Forge with Jupyter

Screenshots

Login And Registration

Users can switch between sign up and login from the same page.

Login Page

Signup Page

Command Console

After login, users can execute Python code and send stdin when the running program calls input().

Main Console


Tech Stack

Layer Technology
Backend Python 3.9+, Flask, Flask-SocketIO, Eventlet
Frontend HTML, CSS, JavaScript, Socket.IO client
Database MongoDB Atlas
Orchestration Kubernetes on Docker Desktop
Runtime Isolation Per-user Kubernetes Deployment + Pod
User Pod Image jupyter/datascience-notebook:latest
Custom Pod Base python:3.9-slim with NumPy, Pandas, scikit-learn, Matplotlib (see Dockerfile)
Execution Model python -c "<submitted code>" inside the user container
Configuration .env with python-dotenv
Test Assets Ready-to-run Python scripts in test-cases/
Documentation Markdown docs in docs/

Project Structure

PyRuntimeForge/
β”œβ”€β”€ docs/
β”‚   └── jupyter-vs-pyruntime-forge.md   # Comparison: PyRuntime Forge vs Jupyter
β”œβ”€β”€ screenshots/
β”‚   β”œβ”€β”€ Login_page.png
β”‚   β”œβ”€β”€ Main.png
β”‚   └── Signup_Page.png
β”œβ”€β”€ templates/
β”‚   └── index.html                      # Single-page frontend (auth + code console)
β”œβ”€β”€ test-cases/
β”‚   β”œβ”€β”€ README.md                       # How to run the test cases
β”‚   β”œβ”€β”€ data_science_demo_step1.py      # Saves a CSV dataset to the pod
β”‚   β”œβ”€β”€ data_science_demo_step2.py      # Loads and summarises the saved dataset
β”‚   β”œβ”€β”€ filesystem_statefulness_step1.py
β”‚   β”œβ”€β”€ filesystem_statefulness_step2.py
β”‚   β”œβ”€β”€ interactive_guess_game.py       # Tests interactive stdin support
β”‚   β”œβ”€β”€ ram_statefulness_step1.py
β”‚   └── ram_statefulness_step2.py
β”œβ”€β”€ .env                                # MongoDB URI (not committed)
β”œβ”€β”€ .gitignore
β”œβ”€β”€ Dockerfile                          # Flask server image (used by Render)
β”œβ”€β”€ pod.Dockerfile                      # Custom user-pod image (Python 3.9-slim + data science libs)
β”œβ”€β”€ LICENSE
β”œβ”€β”€ README.md
β”œβ”€β”€ requirements.txt                    # Server-side Python dependencies
└── server.py                          # Flask + Socket.IO backend

About the Dockerfile

The Dockerfile defines a lightweight custom image for user pods:

FROM python:3.9-slim
RUN pip install --no-cache-dir numpy pandas scikit-learn matplotlib

It is an alternative to jupyter/datascience-notebook:latest if you want a smaller image without the full Jupyter stack. To use it, build and push it to a registry, then replace the image value in the deployment manifest inside server.py.


Installation and Setup

Prerequisites

Make sure you have:

  1. Python 3.9 or newer
  2. Docker Desktop installed
  3. Kubernetes enabled in Docker Desktop
  4. A MongoDB Atlas cluster
  5. Your current public IP allowed in Atlas network access

1. Clone the repository

git clone <your-repository-url>
cd PyRuntimeForge

2. Create and activate a virtual environment

python3 -m venv .venv
source .venv/bin/activate

3. Install dependencies

pip install -r requirements.txt

This installs Flask, Flask-SocketIO, Eventlet, PyMongo, the Kubernetes client, and python-dotenv.

4. Verify Kubernetes

kubectl config current-context
kubectl get nodes

Expected context:

docker-desktop

5. Create .env

See the next section for the exact format.

6. Run the app

python server.py

Default URL:

http://127.0.0.1:5000

macOS port 5000 note

On some macOS systems, AirPlay / Control Center already uses port 5000.

If that happens, run:

python -c "import server; server.socketio.run(server.app, debug=True, port=5001)"

Then open:

http://127.0.0.1:5001

Environment Variables

Create a .env file in the project root:

MONGODB_URI="mongodb+srv://<username>:<password>@<cluster>/<database>?retryWrites=true&w=majority&appName=Cluster0"

Current database usage in the code:

  • database: cloud
  • collection: users

Example:

MONGODB_URI="mongodb+srv://user:password@cluster0.example.mongodb.net/cloud?retryWrites=true&w=majority&appName=Cluster0"

API Endpoints

HTTP Endpoints

GET /

Loads the frontend UI.

POST /register

Registers a user, stores that user in MongoDB Atlas, and creates a per-user Kubernetes deployment.

Example request:

{
  "username": "alice",
  "email": "alice@example.com"
}

Example response:

{
  "message": "User alice registered successfully."
}

POST /login

Looks up the user by email and returns the sanitized container identifier.

Example request:

{
  "email": "alice@example.com"
}

Example response:

{
  "container_id": "alice"
}

Socket.IO Events

Frontend -> Backend

  • execute_command Starts Python execution inside the user's pod.
  • send_stdin Sends one input line to the running process.

Backend -> Frontend

  • command_started Signals that execution has started.
  • command_output Streams stdout and stderr back to the browser.
  • command_complete Signals that execution has finished.

How It Works

Simple Execution Flow

flowchart TD
    A["User opens PyRuntime Forge"] --> B["Register or Login"]
    B --> C["Flask server validates user"]
    C --> D["MongoDB Atlas stores / fetches user"]
    C --> E["Kubernetes creates or finds user pod"]
    E --> F["User submits Python code from browser"]
    F --> G["Backend opens exec session in the user's pod"]
    G --> H["python -c runs the submitted code"]
    H --> I["stdout / stderr streamed back through Socket.IO"]
    I --> J["UI renders output in the console"]

    H --> K{"Program uses input()?"}
    K -- Yes --> L["User types in stdin box"]
    L --> M["Frontend emits send_stdin"]
    M --> H
    K -- No --> I
Loading

End-To-End Workflow

sequenceDiagram
    participant U as User
    participant UI as Browser UI
    participant S as Flask + Socket.IO Server
    participant DB as MongoDB Atlas
    participant K as Kubernetes API
    participant P as User Pod

    U->>UI: Open app
    UI->>S: GET /
    S-->>UI: Render page

    U->>UI: Register
    UI->>S: POST /register
    S->>DB: Insert user
    S->>K: Create deployment
    K-->>S: Deployment created
    S-->>UI: Registration success

    U->>UI: Login
    UI->>S: POST /login
    S->>DB: Find user
    DB-->>S: Return container_id
    S-->>UI: Login success

    U->>UI: Submit Python code
    UI->>S: execute_command
    S->>K: Find pod by label
    K-->>S: Pod name
    S->>P: exec python -c "submitted code"
    P-->>S: stdout/stderr
    S-->>UI: command_output

    alt input() is called
        U->>UI: Type reply in stdin box
        UI->>S: send_stdin
        S->>P: write to stdin
        P-->>S: more output
        S-->>UI: command_output
    end

    P-->>S: process exits
    S-->>UI: command_complete
Loading

Stateful vs Non-Stateful Behavior

PyRuntime Forge remembers:

  • files saved in the container
  • datasets downloaded inside the pod
  • generated CSV files, plots, and outputs written to disk

PyRuntime Forge does not remember:

  • Python variables between separate Execute clicks
  • in-memory pandas dataframes between separate runs
  • imported modules in RAM between separate runs

Example:

Execution 1:

x = 12345
print("Stored x =", x)

Execution 2:

print(x)

Result:

NameError: name 'x' is not defined

This is expected because every execution starts a fresh Python process.


Known Limitations

  • No password authentication. Registration requires only a username and email. This is intentional for a demo/sandbox environment; do not use this in production without adding proper password hashing and session management.
  • No RAM statefulness. Each Execute click runs python -c, which starts a fresh interpreter. Python variables and imported modules are not preserved between clicks. Use file I/O to pass data between executions.
  • Pod startup delay. When a user registers for the first time, Kubernetes must pull the container image. The first execution may fail or stall until the pod reaches the Running state. Check pod status with kubectl get pods.
  • Single namespace. All user pods are deployed to the default namespace. Usernames must be unique and are sanitized to valid Kubernetes resource names (lowercase alphanumeric and hyphens only).
  • No resource limits. The deployment manifest does not set CPU or memory limits on user pods. In a production deployment, resource quotas should be configured.

Useful Commands

Kubernetes status

kubectl cluster-info
kubectl get nodes

Deployments and pods

kubectl get deployments
kubectl get pods
kubectl get pods -o wide

Inspect a pod

kubectl describe pod <pod-name>

Open a shell in a pod

kubectl exec -it <pod-name> -- /bin/bash

If bash is unavailable:

kubectl exec -it <pod-name> -- /bin/sh

Delete a user runtime

kubectl delete deployment <username>-deployment

Documentation and Test Cases

Additional documentation:

Runnable test cases:


πŸ“„ License

This project is licensed under the MIT License β€” see the LICENSE file for details.


πŸ‘¨β€πŸ’» Author

Prashant Agrawal

GitHub


⭐ If you find this project useful, please give it a star! ⭐

About

A cloud-native "Jupyter-as-a-Service" platform. Dynamically provisions isolated, stateful Python runtime environments using Kubernetes, Flask, and WebSockets.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors