Overview
Receipt Scanner is a progressive web app that lets users photograph paper receipts and extract line items, totals, and merchant data using OCR.
It is designed for personal expense tracking with an offline-first workflow: receipts are processed locally and synchronized with the backend when connectivity returns.
Architecture
The Next.js frontend is deployed as a PWA with service-worker caching. Its camera capture pipeline uses the MediaStream API with a custom viewfinder overlay.
OCR runs client-side through Tesseract.js WebAssembly workers, avoiding an additional server round trip. Extracted data is normalized into a structured schema and persisted to Supabase PostgreSQL with row-level security for multi-user isolation.
The bottleneck
OCR accuracy on crumpled or faded thermal paper was initially below 60%. I implemented a multi-pass preprocessing pipeline using adaptive thresholding, deskew correction through Hough transforms, and contrast normalization.
A post-processing step cross-references extracted amounts against the calculated total. Together, these changes improved accuracy to roughly 92% on real-world receipts.
Offline flow
- Capture and preprocess the receipt locally.
- Run OCR inside a WebAssembly worker.
- Normalize the extracted merchant, items, tax, and total.
- Save the pending record locally.
- Synchronize it with Supabase when the connection is restored.