From 284f8760d57af873fbbff3536e4635fe8f27ffec Mon Sep 17 00:00:00 2001 From: Seung Hyun Kim Date: Sat, 11 Apr 2026 18:00:06 -0500 Subject: [PATCH 1/2] Enhance add VR README for the VR directory, detailing setup, usage, and publishing instructions --- .gitignore | 19 +++++++++++++++++++ VR/README.md | 23 +++++++++++++++++++++++ 2 files changed, 42 insertions(+) create mode 100644 VR/README.md diff --git a/.gitignore b/.gitignore index cfdd63c..0451668 100644 --- a/.gitignore +++ b/.gitignore @@ -1,12 +1,31 @@ # rendered files *.png *.mov +*.pkl +*.npz +*.mp4 +*.json +*.jsonl +*.npy + +# deprecated files +_old* + +# VR directory +VR/node_modules/ +!VR/package.json +!VR/package-lock.json # Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] *$py.class +research +model +logs +evaluations + # C extensions *.so *.swp diff --git a/VR/README.md b/VR/README.md new file mode 100644 index 0000000..d89e66a --- /dev/null +++ b/VR/README.md @@ -0,0 +1,23 @@ +# VR Prototype Interface + +This folder contains the Quest WebXR client and startup scripts for the multi-user soft-arm VR prototype powered by the `virtual_field` package. +Deployment model: +- Linux desktop starts and hosts the VR services. +- Clients join over URLs: + - Headgear client opens HTTPS URL in Quest Browser. + - Object-publisher client connects to WSS URL from Python CLI. + +## Quick start + +Use `npx` or `bunx`, depending on your preference. + +### Serve the WebXR client over HTTPS + +Serve the WebXR client over HTTPS: + +```bash +# In VR directory +npx serve client -l 8443 --ssl-cert certs/dev-cert.pem --ssl-key certs/dev-key.pem +# or +bunx serve client -l 8443 --ssl-cert certs/dev-cert.pem --ssl-key certs/dev-key.pem +``` \ No newline at end of file From 24b3272a1a084a52e09c8f1f469fefbb10ded1f3 Mon Sep 17 00:00:00 2001 From: Seung Hyun Kim Date: Sun, 12 Apr 2026 13:50:12 -0500 Subject: [PATCH 2/2] Update VR README --- VR/README.md | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 74 insertions(+), 2 deletions(-) diff --git a/VR/README.md b/VR/README.md index d89e66a..1d1b84d 100644 --- a/VR/README.md +++ b/VR/README.md @@ -7,7 +7,18 @@ Deployment model: - Headgear client opens HTTPS URL in Quest Browser. - Object-publisher client connects to WSS URL from Python CLI. -## Quick start +## Join the VR environment + +Before joining, use the top-right panel: +- Choose `Character` (`Demo-spline`, `Two CR`, ...). +- Click `Join Field`. + - You can join as a spectator by clicking `Join as Spectator`. + - You can joint arm camera view by clicking `Join Arm Camera`. +- Click `Enter VR` to enter the VR environment. + - This will start the VR session. +- If you are using a headgear, you can click `Enter VR` on the bottom to enter the immersion mode. + +## Quick Setup Use `npx` or `bunx`, depending on your preference. @@ -20,4 +31,65 @@ Serve the WebXR client over HTTPS: npx serve client -l 8443 --ssl-cert certs/dev-cert.pem --ssl-key certs/dev-key.pem # or bunx serve client -l 8443 --ssl-cert certs/dev-cert.pem --ssl-key certs/dev-key.pem -``` \ No newline at end of file +``` +### Start runtime server + +> This requires a python environment with all packages installed. + +```bash +./VR/scripts/start_server.sh +``` + +### Open on Quest Browser (replace `YOUR_SERVER_IP` to your IP or domain name): + +> Note: You can also open the client URL in your browser. +> I have not tested, but some VR immerse emulator plugin exists. + +```text +https://YOUR_SERVER_IP:8443/ +``` + +The client now auto-derives websocket endpoint from page host: +- `https://HOST:8443` -> `wss://HOST:8765` +- `http://HOST:...` -> `ws://HOST:8765` + +> You can still override with `?ws=...` or custom port via `?ws_port=...`. + +## Character modes: + +- `Demo-spline`: current spline demo behavior. +- `Two CR`: always 2 arms, base is fixed near body center, and base movement controls are disabled. Controller targets are sent to the server, and the server runs `PyElastica` to compute the rod posture. + +## Python mesh publisher (remote LAN machine) + +Another machine on the same LAN (for example a Macbook) can publish mesh/scenery files into the shared VR scene by connecting to the Linux host WSS URL. + +> TODO + +1. Install python-packages on the publisher machine. +2. Run: + + +## System diagram + +```mermaid +flowchart LR + subgraph L["Linux Desktop (Server Host)"] + C["HTTPS static client :8443"] + W["Virtual Field runtime :8765 (WSS)"] + E["Simulation backend (PyElastica in physics-driven modes)"] + end + + Q["Quest / web browsers"] + P["Python publisher clients"] + + Q -->|"Open client page (see Quick Setup; may be / or /client/)"| C + Q -->|"hello; then xr_input"| W + W -->|"scene_state (periodic broadcast); hello_ack + asset_manifest on join"| Q + + P -->|"hello(role=publisher); mesh / overlay / etc."| W + W -->|"mesh_ack, overlay_ack, etc."| P + + W -->|"teleop / step"| E + E -->|"arm state, meshes, overlays"| W +```