Skip to content

Alemusica/jsom

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

66 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

jsOM 🎨

License: Source Available GitHub stars GitHub Sponsors

jsOM - The Definitive Designer Platform, LLM Friendly

Draw UI visually → Export precise JSON specs → Any LLM generates production code. The bridge between human creativity and AI code generation.


⚖️ LICENSE CHANGE NOTICE

IMPORTANT: As of January 28, 2026, jsOM is licensed under a Source-Available License.

Version License Status
≤ v1.4.0-beta (tag: v1.4.0-beta-MIT-LAST) MIT ⚠️ DEPRECATED - No updates, no support
≥ v1.5.0 Source-Available ACTIVE - All new features here

⚠️ MIT Versions Are DEPRECATED

The MIT version (v1.4.0-beta and earlier) is officially deprecated:

  • No security updates
  • No bug fixes
  • No new features
  • No support
  • Architecturally diverging (incompatible soon)

Using the MIT fork = Using abandonware.

📋 Read the Ethics Statement

127 people cloned jsOM under MIT.

If you're one of them: Read MIT-ETHICS.md

TL;DR: You have legal rights. But parasitizing a solo dev's work for commercial gain is morally bankrupt and the community will know.

For Users of Pre-v1.5.0 (MIT versions)

If you downloaded jsOM under MIT before Jan 28, 2026:

Legitimate Use (learning, internal tools, contributions):

  • You're fine, keep learning!

⚠️ Commercial Use (competitor products, SaaS, rebrand):

Parasitic Use (silent competitor, zero attribution):

  • See MIT-ETHICS.md
  • Your fork will be listed publicly
  • Community judgment is permanent

Note: v1.6.0+ will use AGPL-licensed dependencies, making MIT forks legally incompatible with new code.

Branches

Branch Status Description
master Main Stable release, production-ready
core Dev Development branch
feature/* Experimental Feature development branches

jsOM Logo
👉 Try Live Demo


💖 Support This Project

If jsOM helps your workflow, consider supporting its development:

GitHub Sponsors Ko-fi

Sponsor Tiers

Tier Perks
Supporter (€5/mo) Name in README, Discord access
🚀 Builder (€10/mo) Early access to features, priority support
💎 Patron (€25/mo) 30min monthly consulting, feature requests

🎯 What is this?

UI Canvas MCP is a visual design tool that:

  1. Lets you draw UI visually - Like Figma/Keynote, with rectangles, text, buttons, etc.
  2. AI understands your design - Multi-agent pipeline analyzes layout, style, hierarchy
  3. Generates precise specs - Export universal format any LLM can understand
  4. MCP integration - AI agents can read and manipulate your designs

The key insight: Your drawing is the source of truth. AI doesn't guess layouts - it uses YOUR precise design.


🆕 What's New (v1.4.0-beta - Jan 2026)

Fan Pivot - Gestural Fan Arrangement

  • Keychain Metaphor - Arrange elements like keys on a keychain, pinned at a pivot point
  • Click & Drag - Click to place pivot, drag to control spread and rotation
  • Progressive Rotation - Each element rotates more than the previous for natural fan effect
  • Physics Settling - Elements settle with inertia after gesture release
  • Polar Control - Angle controls rotation, distance controls spread
  • Access via 🌀 Fan icon in toolbar (requires 2+ selected elements)

PDD - Physics Driven Design (World Paradigm)

  • Enter the World - Instead of "simulate", you step into a physics environment
  • Spring Constraints - Connect elements with elastic springs (Soft, Medium, Stiff, Bouncy presets)
  • Real-time Physics - 60fps simulation via Konva direct manipulation
  • Tension Visualization - See constraint forces in real-time
  • Commit on Exit - Leave the world to commit final positions
  • Access via ⚛️ Atom icon in toolbar

Rotation Curve System (Beta)

  • Intelligent Rotation Distribution - Apply easing curves to control rotation across multiple elements
  • Preset Curves - Linear, Ease In/Out, Logarithmic, Exponential
  • Freehand Mode - Draw custom curves directly on canvas
  • Real-time Preview - See rotation changes as you draw
  • Keyboard Controls - Enter to confirm, ESC to cancel

TIP Lite Architecture

  • Layered Code Structure - archetipi (pure math) → molecole (domain rules) → organismi (orchestration)
  • World Paradigm - Enter/exit instead of start/stop for immersive interactions
  • Applied to Fan Pivot and PDD modules

What's New (v1.3.0 - Jan 2026)

UILayoutSpec v1.1 (LLM-Optimized Export)

  • Direct LLM Instructions - llmContext field with code generation guidance
  • Dual Type System - Both type (raw) and semanticType (inferred) for transparency
  • Token Optimization - Stripped default values based on fig2json research
  • Research-Backed - Based on Addy Osmani's LLM workflow best practices

Precise Mode & Snap Grid

  • Shift+Drag - DAW/synth-style fine movement (1/10 precision)
  • Alt+Drag - Temporary grid snapping (Apple Keynote-style)
  • Live Overlay - Coordinates display near cursor during precise mode

Typography Harmonization System

  • Font Database - 15+ fonts with classification metadata
  • Brand Archetypes - clean, classy, friendly, quirky, techie
  • Compatibility Scores - Algorithmic pairing analysis
  • Type Scales - Mobile, desktop, editorial presets

Test Suite & Code Quality

  • Vitest - 26 unit tests for canvasStore
  • DOMPurify - XSS prevention for user input
  • Quadtree - O(log n) marquee selection
  • Scoped Logging - Environment-aware debug utilities

📜 Previous Updates (v1.2.0)

Blender-Style Smooth Zoom

  • Exponential Scaling - Perceptually linear zoom using Math.pow(2, -deltaY * ZOOM_SENSITIVITY)
  • Trackpad Friendly - No more "shooting off" with two-finger zoom gestures
  • Consistent Feel - Same zoom behavior regardless of scroll speed
  • Wide Range - 0.1x to 10x zoom levels

TOON Export Format

  • 40% Fewer Tokens - Optimized notation for LLM context windows
  • Tabular Format - Elements, events, voice notes in compact tables
  • Element Markers - Voice notes reference elements with @id syntax
  • Export Integration - Available alongside JSON, YAML, Markdown in Export Panel

Extended Device Support

  • 24+ Device Presets - iPhone 16 Pro/Max, Pixel 9, Samsung S24, iPad Air, Galaxy Tab S9, MacBook Pro 14", iMac 24", and more
  • Device-Aware Blueprints - Automatically scale UI elements based on device type (mobile/tablet/desktop)
  • Modern Devices - Latest 2024-2025 device dimensions

Semantic Layers System

  • HTML Structure Organization - Organize elements into semantic sections (navigation, header, hero, footer, etc.)
  • Dual View Mode - Toggle between Classic (z-index) and Semantic (hierarchy) views
  • Quick Assign - One-click element assignment to sections
  • CSS Hints - Each section includes layout hints (flexbox, grid, positioning)

Multi-LLM Support

  • Gemini 2.0 Flash - Use Google's latest model for faster, smarter generation
  • Ollama Integration - Local models with qwen2.5:7b and minicpm-v
  • Provider Switching - Easily switch between cloud (Gemini) and local (Ollama)

Style Blueprints

  • 9 Complete Design Systems - Bauhaus, Swiss, Glassmorphism, Brutalist, Material, Neomorphism, Retro80s, Japanese, Minimal
  • 6 UI Component Blueprints - Hero, Navbar, Landing Page, Dashboard, Product Card, Profile
  • LLM-Optimized - Each blueprint includes compositional rules, color palettes, and element specs
  • Device-Aware Scaling - Blueprints adapt to mobile/tablet/desktop automatically
  • Semantic Integration - Blueprints map to semantic sections automatically

Multi-Agent AI Pipeline

  • LangGraph Integration - Stateful agent orchestration with conditional routing
  • Smart Mode - One-click AI generation with vision analysis
  • Knowledge Base - Design intelligence powered by SurrealDB
  • PHI Algorithms - Golden ratio layouts, spacing, and color harmony

Enhanced Canvas

  • Marquee Selection - Drag to select multiple elements
  • Voice Notes - Record and transcribe design notes (Whisper support)
  • Storyboard Recording - Track element movements for animations

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                        UI Canvas MCP                                 │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  ┌──────────────┐    ┌─────────────────────────────────────────┐   │
│  │   Canvas     │    │         Multi-Agent Pipeline            │   │
│  │   Editor     │    │  ┌─────────────────────────────────┐   │   │
│  │  (React +    │───▶│  │        SimpleOrchestrator       │   │   │
│  │   Konva)     │    │  │   (Browser-compatible runner)   │   │   │
│  └──────────────┘    │  └─────────────────────────────────┘   │   │
│         │            │              │                          │   │
│         │            │  ┌───────────▼───────────┐             │   │
│         ▼            │  │     RouterAgent       │             │   │
│  ┌──────────────┐    │  │  (Intent detection)   │             │   │
│  │  Properties  │    │  └───────────┬───────────┘             │   │
│  │    Panel     │    │              │                          │   │
│  │  + Notes     │    │      ┌───────┴───────┐                 │   │
│  │  + Voice     │    │      ▼               ▼                 │   │
│  └──────────────┘    │  ┌────────┐    ┌───────────┐          │   │
│                      │  │ Vision │    │  Prompt   │          │   │
│  ┌──────────────┐    │  │ Agent  │    │ Engineer  │          │   │
│  │   Toolbar    │    │  │minicpm │    │  Agent    │          │   │
│  │  + PHI       │    │  └────┬───┘    └─────┬─────┘          │   │
│  │  Algorithms  │    │       │              │                 │   │
│  └──────────────┘    │       └──────┬───────┘                 │   │
│                      │              ▼                          │   │
│                      │  ┌───────────────────┐                 │   │
│                      │  │  ExecutorAgent    │                 │   │
│                      │  │ (Grid + Collision │                 │   │
│                      │  │   + Z-Index)      │                 │   │
│                      │  └───────────────────┘                 │   │
│                      └─────────────────────────────────────────┘   │
│                                     │                              │
│  ┌──────────────┐    ┌──────────────▼──────────────┐              │
│  │  MCP Server  │◀──▶│       Knowledge Base        │              │
│  │  (stdio)     │    │  - Style Systems            │              │
│  └──────────────┘    │  - Layout Patterns          │              │
│                      │  - Component Patterns       │              │
│                      │  - Design Rules             │              │
│                      └─────────────────────────────┘              │
│                                     │                              │
│                      ┌──────────────▼──────────────┐              │
│                      │         SurrealDB           │              │
│                      │   (Projects + Elements)     │              │
│                      └─────────────────────────────┘              │
└─────────────────────────────────────────────────────────────────────┘

🤖 Multi-Agent System

Agent Pipeline

Agent Model Role
RouterAgent qwen2.5:7b Classifies intent (create/modify/vision)
VisionAgent minicpm-v Analyzes screenshots, extracts UI elements
PromptEngineerAgent qwen2.5:7b Transforms requests into design specs
ExecutorAgent qwen2.5:7b Generates canvas elements with positioning

Smart Mode Features

// Enable Smart Mode in LLMPanel
const orchestrator = new SimpleOrchestrator({ model: 'qwen2.5:7b' });

const result = await orchestrator.generate(prompt, device, {
  screenshot: imageData,           // Optional: analyze existing UI
  existingElements: elements,      // Context for modifications
  visionModel: 'minicpm-v',       // Vision model for screenshots
  onProgress: (step, progress) => console.log(step, progress),
});

LangGraph Workflow (Server-side)

import { UIGenerationGraph } from './orchestration';

const graph = new UIGenerationGraph();
const result = await graph.invoke({
  userPrompt: "Create a login form with email and password",
  device: { name: 'iPhone 14', width: 390, height: 844 },
  screenshot: base64Image,  // Optional
});

🧠 Knowledge Base

The Knowledge Base provides design intelligence to agents:

Style Systems

  • Swiss International - Bold typography, grid-based, minimal color
  • Material Design - Card-based, shadows, 8dp grid
  • Glassmorphism - Frosted glass, transparency, soft shadows
  • Brutalist - Raw, monospace, harsh borders
  • Japanese Minimalism - Extreme whitespace, earth tones

Layout Patterns

  • Hero Section - Full-width header with CTA
  • Dashboard - Grid-based cards and metrics
  • Form Layout - Labeled inputs with validation

Design Rules

  • Spacing guidelines (8px base, PHI scale)
  • Typography hierarchy
  • Color accessibility (WCAG AA)
  • Responsive breakpoints

🌀 PHI (Golden Ratio) Algorithms

Built-in golden ratio utilities for harmonious designs:

import {
  phiSpacing,        // PHI-based spacing scale
  phiColorScale,     // Golden angle color harmony
  phiLayerPalette,   // Layer-aware color palette
  phiGridLayout,     // Golden rectangle grid
  sunflowerPattern,  // Vogel spiral arrangement
  ensureContrast,    // WCAG contrast checking
} from './shared/phi';

// Generate PHI spacing: 16, 26, 42, 68, 110...
const spacing = phiSpacingScale(16, 5);

// Generate harmonious colors using golden angle
const colors = phiColorScale(210, 5); // 5 colors from blue

// Layer-aware palette for UI hierarchy
const palette = phiLayerPalette(220, true); // Dark theme
// { background, surface, content, interactive, overlay }

Toolbar Actions

  • PHI Spacing - Harmonize element spacing
  • PHI Colors - Apply golden angle color distribution
  • PHI Spiral - Arrange elements in Fibonacci spiral
  • Sunflower Pattern - Vogel spiral arrangement
  • PHI Sizes - Scale elements using golden ratio

🚀 Quick Start

1. Install dependencies

git clone https://github.com/Alemusica/jsom.git
cd jsom
npm install

2. Start SurrealDB

surreal start --user root --pass root file:./data/ui-canvas.db

3. Configure LLM Provider

Option A: Ollama (Local)

# Text generation model
ollama pull qwen2.5:7b

# Vision model for screenshot analysis
ollama pull minicpm-v

Option B: Gemini 2.0 Flash (Cloud)

# Set your Gemini API key
export VITE_GEMINI_API_KEY=your_api_key_here

4. Run the app

npm run dev

Open http://localhost:5173


☁️ Vercel Deployment

Deploy the canvas app to Vercel:

1. Install Vercel CLI

npm i -g vercel

2. Configure Environment

Create .env.production:

VITE_GEMINI_API_KEY=your_gemini_api_key
VITE_DEFAULT_PROVIDER=gemini

3. Deploy

vercel --prod

Note: Vercel deployment runs the canvas app only (client-side). For full MCP server functionality, deploy to a Node.js-compatible platform.


📦 Project Structure

jsom/
├── src/
│   ├── canvas/
│   │   ├── components/
│   │   │   ├── CanvasStage.tsx    # Konva canvas with marquee
│   │   │   ├── Toolbar.tsx        # Tools + PHI buttons
│   │   │   ├── PropertiesPanel.tsx # Element properties
│   │   │   ├── NotesPanel.tsx     # Voice notes + transcription
│   │   │   ├── LLMPanel.tsx       # Smart Mode UI
│   │   │   ├── HelpPanel.tsx      # In-app manual
│   │   │   └── FisicaPanel.tsx    # Physics controls
│   │   ├── fan-pivot/             # Fan Pivot (TIP Lite)
│   │   │   ├── archetipi.ts       # Pure math functions
│   │   │   ├── molecole.ts        # Domain rules
│   │   │   ├── FanPivotWorld.ts   # World orchestration
│   │   │   └── useFanPivot.ts     # React hook
│   │   ├── pdd/                   # Physics Driven Design
│   │   │   ├── world/             # World paradigm
│   │   │   ├── constraints/       # Spring, rope, rigid
│   │   │   └── usePDDWorld.ts     # React hook
│   │   ├── services/
│   │   │   ├── agents/            # Multi-agent system
│   │   │   │   ├── BaseAgent.ts
│   │   │   │   ├── RouterAgent.ts
│   │   │   │   ├── VisionAgent.ts
│   │   │   │   ├── PromptEngineerAgent.ts
│   │   │   │   └── ExecutorAgent.ts
│   │   │   ├── orchestration/     # LangGraph + SimpleOrchestrator
│   │   │   │   ├── UIGenerationGraph.ts
│   │   │   │   └── SimpleOrchestrator.ts
│   │   │   ├── knowledge/         # Design Knowledge Base
│   │   │   │   ├── DesignKnowledgeBase.ts
│   │   │   │   └── types.ts
│   │   │   ├── llm.ts             # Ollama integration
│   │   │   └── toon/              # TOON format encoder
│   │   └── store/
│   │       ├── canvasStore.ts     # Zustand state
│   │       └── storyboardStore.ts # Animation recording
│   ├── mcp-server/
│   │   ├── index.ts               # MCP Server entry
│   │   └── storage.ts             # SurrealDB integration
│   └── shared/
│       ├── types.ts               # Shared types
│       └── phi.ts                 # Golden ratio utilities
├── docs/
│   └── assets/                    # Screenshots and demos
├── LICENSE                        # MIT License
└── README.md

🎮 Keyboard Shortcuts

Key Action
V Select tool
R Rectangle
T Text
C Circle
L Line
P Path/Bezier
I Image
B Button
N Input
Delete/Backspace Delete selected
⌘D Duplicate
⌘⇧D Duplicate x5 with offset
⌘Z Undo
⌘⇧Z Redo
⌘⇧H Distribute horizontal
⌘⇧J Distribute vertical
⌘⇧G Distribute grid
⌘⇧R Progressive rotation
Scroll Zoom (Blender-style smooth)
Shift+Scroll Pan canvas
F Fit to screen
Drag on empty Marquee selection

🛠️ MCP Server Tools

Tool Description
create_project Create new canvas project
add_element Add element at exact position
get_elements Get all elements with positions
update_element Modify element properties
delete_element Remove element
export_layout_spec Export universal UI spec (YAML/JSON/TOON)
export_code Generate React/HTML code

MCP Configuration

{
  "mcpServers": {
    "ui-canvas": {
      "command": "node",
      "args": ["/path/to/jsom/dist/mcp-server/index.js"]
    }
  }
}

📱 Device Presets

Mobile iOS: iPhone 16 Pro, iPhone 16 Pro Max, iPhone 15, iPhone 14, iPhone 14 Pro Max, iPhone SE

Mobile Android: Pixel 9, Pixel 7, Samsung S24, Samsung S23

Tablet iOS: iPad Mini, iPad Air, iPad Pro 11", iPad Pro 12.9"

Tablet Android: Galaxy Tab S9, Pixel Tablet

Desktop: HD 1080p, 2K, 4K, MacBook Air 13", MacBook Pro 14", MacBook Pro 16", Surface Laptop, iMac 24"


🔧 Development

# Run everything (canvas + MCP server)
npm run dev

# Build for production
npm run build

# Run MCP server only
npm run mcp

# Canvas only
npm run dev:canvas

📚 Documentation


🤝 Contributing

Contributions are welcome! Please read our contributing guidelines and submit PRs.

  1. Fork the repo
  2. Create your feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

🧩 Tech Stack

  • Frontend: React 18, Konva.js, Zustand, TailwindCSS
  • AI: LangGraph, LangChain, Ollama (local), Gemini 2.0 Flash (cloud)
  • Vision: minicpm-v, Gemini 2.0 (UI screenshot analysis)
  • Database: SurrealDB
  • Protocol: MCP (Model Context Protocol)
  • Deployment: Vercel, Docker

📄 License

jsom Source Available License v1.0

This project is source-available but NOT open source.

  • Free for personal, educational, and non-commercial use
  • View, modify, contribute to the source code
  • NO commercial use without permission
  • NO competing products or rebranding
  • NO redistribution under different names

See LICENSE for full terms.

📜 On MIT Versions

If you downloaded jsOM under MIT (before Jan 27, 2026), you have legal rights to that version. But please read MIT-ETHICS.md on the moral expectations of open source.

For commercial licensing: alessioivoycazzaniga@gmail.com


🙏 Acknowledgments


Built with ❤️ by @FluturArt for the agentic future.

Sponsor

About

Visual UI designer that exports LLM-readable specs. Draw → Export → AI builds it. Semantic layers, 9 design systems, PHI algorithms, MCP server integration.

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Packages

 
 
 

Contributors