This is the repo for ProxiTalk OS. ProxiTalk is a custom operating system designed for the ProxiTalk "platform", which is a handheld communication and gaming device.
Browse, favorite, and launch every installed experience on the device.
Flagship TTS communicator for typing and speaking phrases fast.
Toggle app visibility and tweak how entries appear in the launcher.
Pick which engines are active, adjust voices, and tune speech defaults.
Scan for nearby networks and connect directly from the device.
Soft-restart the runtime when you need a quick clean slate.
Chat with The Garden forums directly from your ProxiTalk handheld.
Minimalist browser tailored to the 128x64 screen for quick lookups.
Proof-of-concept monthly calendar for jotting down upcoming events.
Displays current time/date and offers simple timer capability.
Lightweight editor with syntax highlighting and basic file tools.
Browse monochrome previews of locally stored images.
Check the public repo for updates and sync apps/overlays in place.
4-track, 16-step music tracker with synthesised tones.
Drop in an MP4 and play it back on the ProxiTalk display.
Retro raycasting FPS demo lovingly inspired by DOOM.
Drop-block puzzle classic adapted to the 1-bit panel.
A life countdown overlay that displays approximate seconds remaining for you to live. Can be turned off in app settings if you want less existential dread.
Configure screen brightness, speaker volume, and more without leaving the app you're in.
ProxiTalk/
├── apps/ # Installed apps (auto-discovered by the launcher)
├── overlays/ # Background overlay apps (run on top of any app)
├── config/ # All configuration files
├── tts/ # TTS engine binaries and voice models
├── assets/ # Fonts and emulator icons (do not modify)
├── files/ # User files: images, videos, etc.
└── proxitalk.py # Entry point
ProxiTalk uses two path config files depending on where it is running:
config/emulator/paths.py— used on Windows (emulator)config/paths.py— used on Linux (real hardware)
When setting up a new install, update the relevant file for your platform. The key paths to set are:
| Variable | What it points to |
|---|---|
PIPER_BIN |
Path to the piper or piper.exe binary |
MODEL_PATH |
Default Piper .onnx voice model |
VOICEVOX_BIN |
Path to the VoiceVox executable (if using VoiceVox) |
VOICEVOX_HOST / VOICEVOX_PORT |
VoiceVox server address (default localhost:50021) |
OPENJTALK_HTSVOICE_DIR |
Folder containing OpenJTalk .htsvoice files |
CACHE_DIR |
Where Piper caches synthesised audio |
FILES_DIR |
Where user files (images, video) are stored |
The other paths (APPS_DIR, OVERLAY_DIR, ICON_DIR, FONT_PATH, etc.) should follow naturally from your repo root and typically don't need manual edits.
tts/
├── piper/ # Piper binary + .onnx voice models
└── openjtalk/ # OpenJTalk .htsvoice files
Drop additional Piper .onnx model files into tts/piper/ to make them available in TTS Settings. VoiceVox is an external application that must be installed and running separately.
Generated automatically on first run. You can edit it directly or use the in-app settings screens. Key fields:
| Field | Default | Description |
|---|---|---|
tts_engine |
"piper" |
Active TTS engine: "piper", "voicevox", or "openjtalk" |
disabled_tts_engines |
[] |
Engines to hide even if installed |
piper_model |
null |
Override the default Piper model path |
voicevox_speaker_id |
2 |
VoiceVox speaker/character ID |
auto_sleep_minutes |
5 |
Minutes of inactivity before the display sleeps (0 disables) |
pinned_apps |
[] |
Apps pinned to the top of the launcher |
hidden_apps |
[] |
Apps hidden from the launcher |
disabled_overlays |
[] |
Overlays that should not run |
keyboard_device_path |
null |
Override the keyboard input device path (Linux; e.g. /dev/input/event0) |
debug_piper_wav |
false |
Write .wav files alongside cached .raw audio for debugging |
A Python dict that corrects TTS mispronunciations. Add entries as needed:
word_map = {
'pidge': 'piddge', # key: what you type, value: what the TTS pronounces
}Plain text file with one word per line, used by the Proxi communicator for autocomplete suggestions. A separate config/autocomplete_words_japanese.txt is used when Japanese input is active. Edit either file to add or remove suggestions.
Required if you use the Discourse Chat app. The file is excluded from git — create or edit it with your forum credentials:
USERNAME=your@email.com
PASSWORD=yourpassword
EMAIL=your@email.com
SESSION_TOKEN= # optional, leave blank
CHAT_URL=https://your-forum.discourse.group/chat/c/channel-name/IDThe gallery, video player, and other apps read from this folder. Drop images (.png, .jpg, .gif) and videos (.mp4) here to make them available in those apps.
- Python 3.7+
- PIL (Pillow) for image processing
- pygame (for Windows emulation)
- keyboard (for Windows input handling)
python proxitalk.pyOn Windows, this will start the emulated display. On Linux, it will run on actual hardware.
Two optional assets live in the repo root:
startup.wav– plays once through the speaker stack or emulator when the OS boots.startup.gif– 128×64 (or smaller) monochrome animation that displays while the audio runs.
If you delete either file, that portion of the startup sequence is skipped (no errors are raised). Provide your own versions by dropping replacements with the same filenames next to proxitalk.py. Animated GIF frames are automatically centered and dithered, so you can work in grayscale and let the runtime handle the 1‑bit conversion.
Every ProxiTalk app follows this structure:
apps/
└── your_app_name/
├── main.py # Required: Contains the App class
├── metadata.json # Required: App metadata
├── icon.png # Required: App icon (26x26 recommended)
├── icon_selected.png # Required: Selected state icon
└── assets/ # Optional: App-specific assets
Create apps/your_app_name/main.py:
from interfaces import AppBase
from PIL import Image, ImageDraw
class App(AppBase):
def __init__(self, context):
"""Initialize the app with context"""
super().__init__(context)
def start(self):
"""Called when the app starts"""
# Set a simple text screen using the built-in method
self.set_screen("My App", "Hello, ProxiTalk!")
def update(self):
"""Called every frame (20fps/hz by default)"""
# Update game logic, animations, etc.
pass
def onkeydown(self, keycode):
"""Handle key press events"""
if keycode == "KEY_ESC":
# Return to launcher
self.context["app_manager"].swap_app_async(
"your_app_name", "launcher", update_rate_hz=20.0, delay=0.1
)
elif keycode == "KEY_SPACE":
self.set_screen("My App", "Space pressed!")
def onkeyup(self, keycode):
"""Handle key release events"""
pass
def stop(self):
"""Called when the app stops"""
# Cleanup state or resources if needed
passProxiTalk provides several built-in methods for displaying content:
# Simple text screen with title and body
self.set_screen("Title", "Body text content")
self.set_screen_with_cursor("Editor", "Type here: [highlighted text]")
# Temporary message screen
self.show_message("Success", "File saved!", duration=2)
# Error screen
self.show_error("File not found")
# Loading screen
self.show_loading("Processing...")For custom graphics, you can use the context drawing methods:
def draw_custom_screen(self):
# Create a 128x64 monochrome image (the size of the ProxiTalk display)
img = Image.new("1", (128, 64), 0) # 0 = black background
draw = ImageDraw.Draw(img)
# Draw shapes
draw.rectangle([10, 10, 50, 30], fill=1) # White rectangle
draw.ellipse([60, 20, 80, 40], outline=1) # White circle outline
# Draw text on the image
font = self.context["fonts"]["small"]
draw.text((10, 45), "Hello World!", font=font, fill=1)
# Clear display and draw the image
self.context["drawing"]["clear_screen"]()
self.context["drawing"]["draw_image"](img, 0, 0)While you can use almost all standard key codes, here are some commonly used ones:
KEY_ESC- Escape keyKEY_SPACE- SpacebarKEY_UP,KEY_DOWN,KEY_LEFT,KEY_RIGHT- Arrow keysKEY_W,KEY_A,KEY_S,KEY_D- WASD keysKEY_ENTER- Enter keyKEY_P- P key (commonly used for pause)KEY_R- R key (commonly used for restart)
Use the context drawing methods for efficient rendering:
# Clear the display
self.context["drawing"]["clear_screen"]()
# Draw a PIL image at position (x, y)
self.context["drawing"]["draw_image"](img, x, y)
# Draw text directly
self.context["drawing"]["draw_text"]("text", x, y, font)
# Or with custom fill color
self.context["drawing"]["draw_text"]("text", x, y, font, fill=255)
# Clear a specific area
self.context["drawing"]["clear_area"](x, y, width, height)
# Draw a filled area
self.context["drawing"]["draw_area"](x, y, width, height, fill=255)# Draw on the overlay layer (appears on top of apps)
self.context["drawing"]["draw_overlay_text"]("text", x, y, font)
self.context["drawing"]["draw_overlay_image"](img, x, y)
self.context["drawing"]["clear_overlay_area"](x, y, width, height)
self.context["drawing"]["draw_overlay_area"](x, y, width, height, fill=255)# Batch multiple drawing operations for better performance
self.context["drawing"]["begin_batch"]()
self.context["drawing"]["draw_text"]("Player 1", 10, 10)
self.context["drawing"]["draw_text"]("Score: 100", 10, 20)
self.context["drawing"]["draw_area"](50, 50, 10, 10, fill=255)
self.context["drawing"]["end_batch"]() # All drawing happens at onceThe built-in set_screen() method supports:
- Automatic text wrapping
- Highlighted text using
[brackets] - Proper spacing and layout
# Play sound effects (place audio files in your app directory)
self.context["audio"]["play_sfx"](self.context["app_path"] + "sound.wav")
# Play background music (looped)
self.context["audio"]["play_music"](self.context["app_path"] + "music.wav", loop=True)
# Text-to-speech
self.context["tts"]["run"]("Hello, this will be spoken!", background=True)Create apps/your_app_name/metadata.json:
{
"name": "My Custom App",
"version": "1.0",
"description": "A description of what your app does",
"author": "Your Name",
"type": "app"
}For overlay apps (run in background):
{
"name": "My Overlay",
"version": "1.0",
"description": "A description of what your overlay does",
"author": "Your Name",
"type": "overlay"
}from interfaces import AppBase
from PIL import Image, ImageDraw
import random
class App(AppBase):
def __init__(self, context):
super().__init__(context)
# Game state
self.player_x = 64
self.player_y = 32
self.score = 0
self.needs_redraw = True
def start(self):
self.needs_redraw = True
def update(self):
if self.needs_redraw:
self.draw_game()
self.needs_redraw = False
def draw_game(self):
# Use batching for better performance
self.context["drawing"]["begin_batch"]()
# Clear screen
self.context["drawing"]["clear_screen"]()
# Draw player as a filled rectangle
self.context["drawing"]["draw_area"](
self.player_x-2, self.player_y-2, 4, 4, fill=255
)
# Draw score
font = self.context["fonts"]["small"]
self.context["drawing"]["draw_text"](
f"Score: {self.score}", 5, 5, font
)
# Execute all drawing at once
self.context["drawing"]["end_batch"]()
def onkeydown(self, keycode):
moved = False
if keycode == "KEY_LEFT" and self.player_x > 5:
self.player_x -= 5
moved = True
elif keycode == "KEY_RIGHT" and self.player_x < 123:
self.player_x += 5
moved = True
elif keycode == "KEY_UP" and self.player_y > 5:
self.player_y -= 5
moved = True
elif keycode == "KEY_DOWN" and self.player_y < 59:
self.player_y += 5
moved = True
elif keycode == "KEY_ESC":
self.context["app_manager"].swap_app_async(
"your_game", "launcher", update_rate_hz=20.0, delay=0.1
)
if moved:
self.needs_redraw = TrueAfter creating your app, it will automatically be discovered by the ProxiTalk system. Make sure to include appropriate icon files.
- Use
print()statements for console debugging - Check the console output when running
python proxitalk.py - The emulator window shows the actual display output
- Use try/catch blocks around PIL operations to handle errors gracefully
ProxiTalk includes a built-in debug overlay system to help developers optimize their drawing operations and understand how the display system works:
In the emulator (Windows):
- Press
F1while the emulator window is focused to toggle the debug overlay on/off - When enabled, you'll see red rectangles highlighting areas of the screen that are being updated
- The console will show:
[Debug] Region overlay: ONor[Debug] Region overlay: OFF
- Red rectangles appear whenever a drawing operation updates part of the screen
- Each rectangle shows the exact region that was modified by a drawing call
- Rectangles fade out over time (about 5 frames) to show the update sequence
- Semi-transparent overlay allows you to see both the content and the update regions
# Example: Inefficient drawing (many small updates)
def bad_draw_example(self):
for i in range(10):
self.context["drawing"]["draw_text"](f"Line {i}", 10, i*10, font)
# Debug overlay will show 10 separate red rectangles
# Example: Efficient drawing (batched updates)
def good_draw_example(self):
self.context["drawing"]["begin_batch"]()
for i in range(10):
self.context["drawing"]["draw_text"](f"Line {i}", 10, i*10, font)
self.context["drawing"]["end_batch"]()
# Debug overlay will show 1 red rectangle covering the entire area- Too many small rectangles: Consider using
begin_batch()andend_batch() - Large rectangles for small changes: You might be clearing/redrawing too much
- Overlapping rectangles: Multiple drawing calls affecting the same area
- Constant flickering: Indicates unnecessary redraws every frame
- Batch related operations: Group drawing calls that happen together
- Minimize cleared areas: Only clear the specific regions that changed
- Use dirty flags: Only redraw when something actually changed
- Separate static/dynamic content: Use overlay layer for frequently changing elements
The debug overlay only works in the Windows emulator and helps identify inefficient drawing patterns that could impact performance on actual hardware.