feat: bulk import validation & preview improvements (#115)#473
Open
qiridigital wants to merge 1 commit intorohitdash08:mainfrom
Open
feat: bulk import validation & preview improvements (#115)#473qiridigital wants to merge 1 commit intorohitdash08:mainfrom
qiridigital wants to merge 1 commit intorohitdash08:mainfrom
Conversation
Enhances the expense import flow with per-row validation, explicit data corrections, and actionable warnings before the user commits transactions. Backend changes: - services/expense_import.py: New validate_import_rows() function - Per-row status: valid | warning | warning | invalid - Warnings: future/old dates, large amounts, duplicates, unknown category - Corrections: date format, amount normalisation, truncation, inferred type - Summary counts: total, valid, warnings, invalid, duplicates - routes/expenses.py: New POST /expenses/import/validate endpoint - Accepts same multipart file upload as /import/preview - Loads user's existing expenses for duplicate detection - Loads user's category IDs for category validation - Returns full validation result (rows + summary) - Original /import/preview and /import/commit unchanged Frontend changes: - api/importValidate.ts: TypeScript client with full typed interfaces - pages/BulkImport.tsx: Two-phase import page with summary bar, filter tabs (All/Valid/Warning/Invalid), row cards with expandable detail, sticky commit button, post-commit success banner - App.tsx: route /import added - Navbar.tsx: Import nav link added Tests: 14 pytest tests in test_import_validate.py Closes rohitdash08#115
There was a problem hiding this comment.
Pull request overview
Adds a bulk import validation/preview flow (issue #115) with per-row validation, warnings, corrections, and duplicate detection before committing transactions.
Changes:
- New
validate_import_rows()service function and/expenses/import/validateendpoint for per-row validation with status, warnings, and corrections - New React
BulkImportpage with upload, summary tiles, filter tabs, row cards, and commit flow - 14 pytest tests covering the validation endpoint
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
packages/backend/app/services/expense_import.py |
New validate_import_rows() with date/amount/description/category validation, duplicate detection, and correction reporting |
packages/backend/app/routes/expenses.py |
New POST /expenses/import/validate endpoint wiring up the validation service |
app/src/api/importValidate.ts |
TypeScript client with typed interfaces for validate and commit APIs |
app/src/pages/BulkImport.tsx |
Two-phase import page with upload, summary, filtering, and commit |
app/src/App.tsx |
Added /import route |
app/src/components/layout/Navbar.tsx |
Added "Import" nav link |
packages/backend/tests/test_import_validate.py |
14 tests for the validation endpoint |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| summary – counts (total, valid, warnings, invalid, duplicates) | ||
| """ | ||
| today = date.today() | ||
| far_future = date(today.year, today.month, today.day) |
| def validate_import_rows( | ||
| raw_rows: list[dict[str, Any]], | ||
| *, | ||
| existing_descriptions: set[str] | None = None, |
Comment on lines
+177
to
+178
|
|
||
| import pytest |
| raw_rows: list[dict[str, Any]], | ||
| *, | ||
| existing_descriptions: set[str] | None = None, | ||
| existing_amounts: dict[str, list[Decimal]] | None = None, |
Comment on lines
+309
to
+313
| existing = ( | ||
| db.session.query(Expense.notes, Expense.spent_at) | ||
| .filter_by(user_id=uid) | ||
| .all() | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enhances the expense import flow with per-row validation, explicit data corrections, and actionable warnings — allowing users to review all issues and auto-corrections before committing transactions to the database.
What's included
Backend (
packages/backend/)app/services/expense_import.py— Newvalidate_import_rows()function:status:valid|warning|invalidsummary:{total, valid, warnings, invalid, duplicates}app/routes/expenses.py— NewPOST /expenses/import/validateendpoint:/import/previewcategory_idfields{rows: [...], summary: {...}}/import/previewand/import/commitunchanged (backward compatible)Frontend (
app/src/)api/importValidate.ts— TypeScript client with full typed interfacespages/BulkImport.tsx— Two-phase import page:App.tsx—/importroute addedNavbar.tsx— "Import" nav link addedTests
tests/test_import_validate.py— 14 pytest tests: auth, no-file, valid CSV, response structure, bad date, bad amount, empty description, future date, old date, big amount, duplicate detection, corrections in response, summary counts, unknown file typeCloses
Closes #115