The idea of this project came after noticing that I have a plenty of shopping receipts lying down on my desk. Initially, I was going to dump them all in a bin, but I realized these could help me build a project, so I came up with a shopping receipt scanner using OCR and LLM to extract information, then visualize them on dashboard. At first, I wanted to just use OpenClaw, but since I only use Ollama cloud, the token usage may hit the ceiiling in no time, so I decided to use an OCR combined with LLM. I have been aware of the challenges, but that is why I want to dive into AI Engineering. So far, the PoC is done and I can use it for my daily life. I will also be adding some more features to improve my current project.
This project is using Cloudflare as the deployment, Supabase as the database, Telegram as the bot, and Ollama as the LLM thus a Cloudflare account, Supabase account, Telegram Bot Token, and Ollama account are required.
- cd to frontend
cd frontend - add .env
VITE_BACKEND_URL=your_backend_url VITE_STORAGE_KEY=your_randomized_openssl_rand
- install the dependencies
bun install && bun run dev - cd to backend
cd backend - add .dev.vars
SUPABASE_URL=yout_supabase_url SUPABASE_KEY=your_supabase_key ML_SERVICE=your_ml_service_url JWT_SECRET=your_generated_jwt_secret BOT_TOKEN=your_telegram_bot_token
- install the dependencies
bun install && bun run dev - cd to ml-service
uv sync && uvicorn app.main:app --reload - add .env
TELEGRAM_BOT_TOKEN=your_telegram_bot_token OLLAMA_API_KEY=your_ollama_api_key BACKEND_URL=your_backend_url
Temporarily the development of the ml-service is using polling as I am in the progress of deploying it to my homelab server. As soon as I do that, I will change the ml-service to run using a webhook instead.
- Receipt History allowing users to see the receipts data
- HITL (Human-In-The-Loop) enabling edit mode in the dashboard
- Status System -> Verified, Action Required
- Dashboard with graphs to track outcome / Analytics
- Category Breakdown