Create hilarious lip-synced meme videos by animating cartoon mouths on any image! Upload a character image and audio, and watch the magic happen.
-
Two Animation Modes
- 🎭 Meme Mode - Volume-based mouth animation (instant, works with any audio)
- 🎤 Precise Mode - Phoneme-accurate lip sync using Rhubarb Lip Sync
-
Smart Audio Handling
- Upload separate vocal track for precise analysis while playing full song
- Automatic MP3 to WAV conversion
- Lyrics support for improved accuracy
-
Easy Positioning
- Click and drag to position the mouth
- Adjustable size slider
- Real-time preview
-
Export Options
- MP4 (H.264/AAC - universal playback)
- WebM (VP9 - smaller file size)
- Node.js 18+
- ffmpeg (for MP3 conversion and MP4 export)
- Rhubarb Lip Sync (for Precise mode)
-
Clone the repository
git clone https://github.com/akrowczyk/meme-lipsync.git cd meme-lipsync -
Install dependencies
npm install
-
Download Rhubarb Lip Sync (required for Precise mode)
Download the latest release for your OS from Rhubarb releases and extract to
server/bin/:# macOS example cd server/bin curl -L https://github.com/DanielSWolf/rhubarb-lip-sync/releases/download/v1.14.0/Rhubarb-Lip-Sync-1.14.0-macOS.zip -o rhubarb.zip unzip rhubarb.zip rm rhubarb.zip
-
Set up environment (optional)
cp .env.example .env # Edit .env if needed -
Start the app
# Start both frontend and backend npm run dev:full # Or start separately: npm run server # Backend on http://localhost:3001 npm run dev # Frontend on http://localhost:5173
-
Open in browser
Navigate to http://localhost:5173
- Upload a character image (PNG recommended)
- Upload any audio file (MP3, WAV, OGG)
- Select Meme Mode 🎭
- Click Start Meme Mode
- Position the mouth by clicking on the preview
- Adjust size with the slider
- Press play to preview
- Export as MP4 or WebM
- Upload a character image
- Upload your audio (full song with music)
- (Optional) Upload an isolated vocal track for better accuracy
- (Recommended) Paste the lyrics
- Select Precise Mode 🎤
- Click Analyze Lip Sync
- Position and preview
- Export
Tip: For songs with background music, use vocalremover.org to extract vocals for the analysis track.
The default cartoon mouth sprites are in public/mouths/. To use custom mouths:
- Create 9 PNG images (A.png through H.png + X.png) on a bright green (#00FF00) background
- The green will be automatically removed (chroma key)
- Place them in
public/mouths/
| Shape | Sound | Description |
|---|---|---|
| A | P, B, M | Closed lips |
| B | EE | Teeth showing |
| C | EH, AE | Open oval |
| D | AH | Wide open |
| E | OH | Rounded |
| F | OO, W | Puckered |
| G | F, V | Teeth on lip |
| H | L | Tongue visible |
| X | Rest | Neutral/closed |
- Frontend: React + Vite
- Backend: Node.js + Express
- Lip Sync: Rhubarb Lip Sync
- Audio/Video: ffmpeg, Web Audio API, MediaRecorder API
meme-lipsync/
├── public/
│ └── mouths/ # Mouth sprite images
├── server/
│ ├── bin/ # Rhubarb binary (not in repo)
│ ├── uploads/ # Temporary uploaded files
│ ├── index.js # Express server
│ └── rhubarbService.js # Rhubarb CLI wrapper
├── src/
│ ├── components/ # React components
│ ├── App.jsx # Main app component
│ └── index.css # Global styles
├── .env.example # Environment template
└── package.json
Contributions are welcome! Please feel free to submit a Pull Request.
- Fork the repository
- Create your feature branch (
git checkout -b feature/AmazingFeature) - Commit your changes (
git commit -m 'Add some AmazingFeature') - Push to the branch (
git push origin feature/AmazingFeature) - Open a Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.
- Rhubarb Lip Sync by Daniel Wolf
- Inspired by classic lip sync meme videos
Made with ❤️ and way too many late nights