From cac6919ab8b64848e2b209750066cecb340b4b46 Mon Sep 17 00:00:00 2001 From: Ramesh Padmanabhaiah Date: Tue, 16 Jun 2026 01:49:16 -0700 Subject: [PATCH] Consolidate financial intelligence design docs --- .ai-context/ARCHITECTURE.md | 11 +- .ai-context/STATUS.md | 5 +- CHANGELOG.md | 2 + README.md | 5 +- docs/assetbuddy_design.md | 837 --------- ...=> financial_intelligence_architecture.md} | 51 +- ...inancial_intelligence_architecture_spec.md | 1569 ----------------- docs/financial_intelligence_open_questions.md | 137 ++ docs/financial_intelligence_vision.md | 28 - 9 files changed, 170 insertions(+), 2475 deletions(-) delete mode 100644 docs/assetbuddy_design.md rename docs/{financial_intelligence_architecture_review.md => financial_intelligence_architecture.md} (95%) delete mode 100644 docs/financial_intelligence_architecture_spec.md create mode 100644 docs/financial_intelligence_open_questions.md diff --git a/.ai-context/ARCHITECTURE.md b/.ai-context/ARCHITECTURE.md index abb41f1..5c342b2 100644 --- a/.ai-context/ARCHITECTURE.md +++ b/.ai-context/ARCHITECTURE.md @@ -11,10 +11,13 @@ - `base_manifest.yaml` is the Base project contract. It declares BankBuddy as a uv-managed Python project and routes selected project commands through Base's `runner: uv` support. -- `bank_buddy_spec.md` is the broader design and architecture specification. -- `docs/financial_intelligence_architecture_review.md` is the current v2 - architecture direction for evolving BankBuddy into a local-first personal - financial intelligence platform. +- `bank_buddy_spec.md` is the original banking-focused design specification. +- `docs/financial_intelligence_vision.md` is the product north star for the + broader local-first personal financial intelligence platform. +- `docs/financial_intelligence_architecture.md` is the current v2 architecture + direction. +- `docs/financial_intelligence_open_questions.md` tracks unresolved design + topics. ## Runtime Model diff --git a/.ai-context/STATUS.md b/.ai-context/STATUS.md index a50a395..3a4417d 100644 --- a/.ai-context/STATUS.md +++ b/.ai-context/STATUS.md @@ -22,9 +22,10 @@ section is `Unreleased`. ## Recent Major Changes -- Financial intelligence v2 architecture review, keeping the `bankbuddy` name +- Financial intelligence v2 architecture docs, keeping the `bankbuddy` name while moving the target model toward documents, entities, observations, and - relationships. + relationships. Active design docs are the vision, architecture, and open + questions documents. - Prospective relicensing from MIT to AGPL-3.0-or-later. - Canonical data-home layout with `database/`, `bank/`, and `tax/` directories. - First TaxBuddy CLI slice and `tax_documents` metadata index. diff --git a/CHANGELOG.md b/CHANGELOG.md index e30111f..ea02a8e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,8 @@ and versions are tracked in the repo-root `VERSION` file. ### Changed +- Consolidated the financial intelligence design docs into a current vision, + canonical architecture, and open-questions set. - Declared BankBuddy as a uv-managed Base project and routed Base test/run commands through Base's uv runner contract. - Moved the README license notice out of the opening project summary. diff --git a/README.md b/README.md index 8dab6e5..26a3f87 100644 --- a/README.md +++ b/README.md @@ -7,10 +7,9 @@ Local-first personal finance tracking. BankBuddy is evolving from a banking-focused CLI into a local-first personal financial intelligence platform. The current direction is documented in: -- [Financial intelligence architecture review](docs/financial_intelligence_architecture_review.md) - [Financial intelligence vision](docs/financial_intelligence_vision.md) -- [Financial intelligence architecture source spec](docs/financial_intelligence_architecture_spec.md) -- [AssetBuddy design discussion notes](docs/assetbuddy_design.md) +- [Financial intelligence architecture](docs/financial_intelligence_architecture.md) +- [Financial intelligence open questions](docs/financial_intelligence_open_questions.md) ## Development diff --git a/docs/assetbuddy_design.md b/docs/assetbuddy_design.md deleted file mode 100644 index 4126371..0000000 --- a/docs/assetbuddy_design.md +++ /dev/null @@ -1,837 +0,0 @@ -# AssetBuddy — Design, Architecture & Roadmap - -**Version:** 1.0 -**Status:** Design phase — ready for architecture refinement and implementation -**Evolved from:** BankBuddy (bank-focused CLI, transaction tracking) -**Date:** June 2026 - ---- - -## 1. What Is AssetBuddy? - -AssetBuddy is a **document-first, privacy-first personal financial command center** for -technically capable individuals. It tracks every dimension of a person's financial life — -bank accounts, investments, properties, tax documents, employer records — across multiple -currencies and jurisdictions, without ever connecting to a bank or external institution. - -### The Core Insight - -Most personal finance tools (Mint, Personal Capital, Quicken) work by connecting to your -bank via OAuth and pulling data automatically. This creates: - -- Privacy exposure — your credentials or tokens live on a third-party server -- Dependency — when the service shuts down (as Mint did in 2023), your data and - history disappear -- Complexity — maintaining connections to dozens of institutions is hard engineering -- Fragility — any API change at any bank breaks the product - -AssetBuddy takes the opposite approach: **the user provides all data manually by -downloading documents from institutions and feeding them to the system.** The system -makes meaning from those documents. - -This is not a limitation — it is a feature. The target user is someone who: -- Knows how to log into their bank and download a statement -- Is comfortable running a CLI command -- Values data ownership and privacy over convenience -- Has a complex financial life (multiple countries, currencies, asset types) - ---- - -## 2. Design Principles - -### 1. Document-First -Every piece of data in the system comes from a document the user explicitly provides — -PDF, spreadsheet, or scanned image. No API connections. No OAuth. No automatic syncing. -The document is the source of truth. - -### 2. Privacy by Design -No bank credentials stored. No cloud service holds financial data. Data lives locally on -the user's machine, with optional iCloud sync for family access. The threat model is -"someone gains access to your Mac" — not "a cloud server is breached." - -### 3. Asset-Centric, Not Bank-Centric -The original BankBuddy was transaction-focused and bank-specific. AssetBuddy models -the complete financial picture: bank accounts, credit cards, mortgages, properties, -investment accounts, employer relationships, tax obligations. Every financial entity -is an **asset**. - -### 4. Fail Loudly, Not Silently -If a document cannot be confidently routed to the correct asset, the system errors -out with a clear message rather than guessing. Bad data in the transactions table is -worse than a failed import. - -### 5. Infer From Documents, Not From Manual Entry -The system should extract as much metadata as possible from the documents themselves — -institution name, account type, account suffix, date range, form type, jurisdiction. -The user's setup burden should be minimal. - -### 6. Never Pollute the Transactions Table -All parsing and validation happens in a staging area. Only clean, validated, -deduplicated data is committed to `bb_transaction`. A bad import that rolls back -is infinitely better than corrupted transaction history. - -### 7. All Computations On the Fly -No precomputed summaries or derived data stored in the database. Spending reports, -budget analysis, and trend computation all query raw transactions at runtime. -This keeps the schema clean and handles retroactive imports (historical data) -correctly without needing cache invalidation. - -### 8. Multi-Currency, Multi-Jurisdiction by Default -No assumption that the user is in one country or uses one currency. Currency is -stored per transaction. Jurisdiction is stored per document and per asset. Budgets -are per-currency. The schema makes no US-centric or India-centric assumptions. - -### 9. Households, Not Just Individuals -The design should accommodate joint accounts, family members with separate assets, -and shared financial visibility. This is deferred to a later phase but should not -be architecturally precluded. - -### 10. Build for Yourself First -Ship when it works for you. Share when friends ask. Commercialize when strangers -want it. No premature scaling. - ---- - -## 3. Evolution: BankBuddy → AssetBuddy - -### What BankBuddy Already Has (Keep) - -- Import pipeline: PDF and spreadsheet ingestion with file-level deduplication - (SHA-256 hash of file contents) -- Row-level deduplication: SHA-256 hash of (asset + date + amount + description) - prevents duplicate rows even from overlapping spreadsheets -- Staging area: parsed data validated in memory before committing to database -- Folder structure: `inbox/` for new files, `processed//` for - archived files after import -- Bank inference from document content — not from filename -- Parsers: Bank of America (CSV, PDF), HDFC Bank (XLS), ICICI Bank (XLS), - Apple Card (PDF) -- Transaction categorization: rule-based with user-correction learning -- Budget tracking: per category, per currency, monthly or annual -- CLI-first interface using `click` -- SQLite database (local) -- Base-managed project with `base_manifest.yaml` - -### What Changes in AssetBuddy - -- **Old model:** `bank` → `account` → `transaction` -- **New model:** `bb_institution` → `bb_asset` → `bb_transaction` -- Banks become institutions (broader concept) -- Accounts become assets (much broader: includes properties, mortgages, employer records) -- Documents become first-class entities (`bb_document`) -- Parsers are decoupled from assets and linked via a mapping table -- Fingerprinting introduced for routing documents to the correct asset -- Asset attributes stored in a flexible key-value table (`bb_asset_attribute`) - -### What Stays the Same - -- Design philosophy (document-first, local, private) -- Import flow (scan inbox, hash check, parse, validate, stage, commit, archive) -- Deduplication logic (file-level and row-level) -- Categorization engine (rule-based → ML → LLM fallback, phased) -- Budget design (per category, per currency, monthly/annual, min/max bounds) -- CLI-first development -- Python + SQLite + `click` stack - ---- - -## 4. Naming Conventions - -### Table Prefix -All tables prefixed with `bb_` to respect the BankBuddy origin and avoid -naming conflicts if the schema is embedded in a larger system. - -### Table Names -All **singular** (not plural). `bb_asset`, not `bb_assets`. - -### Full Table List -``` -bb_institution -bb_asset_type -bb_asset -bb_asset_attribute_type -bb_asset_attribute -bb_asset_parser -bb_asset_to_parser -bb_document -bb_import_activity -bb_transaction -bb_category -bb_category_rule -bb_budget -bb_tax_source -bb_tax_profile -bb_tax_form_gap -``` - ---- - -## 5. Data Model - -All tables include `created_at DATETIME NOT NULL` and `updated_at DATETIME NOT NULL`. - ---- - -### `bb_institution` - -Registry of all entities that issue documents or hold assets. Replaces the old -`banks` table but is much broader. - -| Column | Type | Notes | -|---|---|---| -| `institution_id` | INTEGER PK | Surrogate key | -| `institution_name` | TEXT NOT NULL UNIQUE | "Bank of America", "Fidelity", "Apple Inc", "IRS", "Alameda County Assessor", "Bangalore City Corporation" | -| `institution_type` | TEXT NOT NULL | "bank", "brokerage", "employer", "government", "utility", "insurance", "other" | -| `country` | TEXT NOT NULL | "US", "India", "UK" — no country assumptions | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - ---- - -### `bb_asset_type` - -Reference table defining valid asset types. Extensible — add new types as discovered. - -| Column | Type | Notes | -|---|---|---| -| `asset_type_id` | INTEGER PK | Surrogate key | -| `asset_type_name` | TEXT NOT NULL UNIQUE | "bank_account", "credit_card", "investment_account", "mortgage_account", "property", "employer_record", "vehicle", "other" | -| `description` | TEXT | Human explanation of what this type represents | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - -**Seed data:** -- `bank_account` — checking, savings, CD accounts -- `credit_card` — credit card accounts -- `investment_account` — brokerage, retirement (401k, IRA, NPS) -- `mortgage_account` — home loan accounts -- `property` — real estate (residential, rental, commercial, land) -- `employer_record` — W-2/salary slip source -- `vehicle` — car loans (future) -- `other` — catch-all for unclassified assets - ---- - -### `bb_asset` - -The unified root entity. Every financial entity the user tracks is a row here. - -| Column | Type | Notes | -|---|---|---| -| `asset_id` | INTEGER PK | Surrogate key — universal key across the system | -| `asset_type_id` | INTEGER FK | References `bb_asset_type.asset_type_id` | -| `institution_id` | INTEGER FK | References `bb_institution.institution_id` | -| `asset_name` | TEXT NOT NULL | User-friendly label: "My BofA Checking", "Sunnyvale House", "Fidelity Brokerage" | -| `fingerprint_json` | TEXT | JSON metadata for document routing. Example: `{"holder_name":"Ramesh K", "account_suffix":"1234", "account_type":"checking"}` | -| `status` | TEXT NOT NULL | "active", "closed", "archived" | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - -**Key design decision:** `asset_id` is the universal foreign key throughout the schema. -Transactions, documents, and import activities all reference `asset_id`, not a -bank-specific account ID. - ---- - -### `bb_asset_attribute_type` - -Whitelist of valid attribute types. Enforces consistency across all assets. -Extensible — add new types as you encounter them in documents. - -| Column | Type | Notes | -|---|---|---| -| `attr_type_id` | INTEGER PK | Surrogate key | -| `attr_type_name` | TEXT NOT NULL UNIQUE | "account_number", "address", "email", "phone", "loan_number", "account_suffix", "holder_name", "property_type", "purchase_date", "purchase_price", "rental_days_per_year", "depreciation_basis", "tax_id", "employer_ein" | -| `description` | TEXT | What this attribute means | -| `applicable_asset_types` | TEXT | JSON array of asset types that typically have this attr. Null = applies to any | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - ---- - -### `bb_asset_attribute` - -Flexible key-value store for asset-specific attributes. - -| Column | Type | Notes | -|---|---|---| -| `attr_id` | INTEGER PK | Surrogate key | -| `asset_id` | INTEGER FK | References `bb_asset.asset_id` | -| `attr_type_id` | INTEGER FK | References `bb_asset_attribute_type.attr_type_id` | -| `attr_value` | TEXT NOT NULL | The actual value: "123 Main St, Sunnyvale CA", "me@email.com", "1234" (last 4 of account) | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - -**Why this pattern instead of typed columns?** -Every asset type has different attributes. A bank account has an account number. -A property has an address and purchase date. A mortgage has a loan number and lender. -Rather than adding columns to `bb_asset` for every possible attribute (which bloats -the table and breaks for new asset types), we store attributes in a separate table. -Query: `SELECT attr_value FROM bb_asset_attribute a JOIN bb_asset_attribute_type t -ON a.attr_type_id = t.attr_type_id WHERE asset_id = 5 AND t.attr_type_name = 'address'` - ---- - -### `bb_asset_parser` - -Registry of available document parsers. A parser is a parsing strategy for a specific -document format — decoupled from any particular asset instance. - -| Column | Type | Notes | -|---|---|---| -| `parser_id` | INTEGER PK | Surrogate key | -| `parser_name` | TEXT NOT NULL UNIQUE | "bof_a_checking_statement", "bof_a_savings_statement", "bof_a_mortgage_statement", "apple_card_statement", "hdfc_savings_xls", "icici_checking_xls", "form_1099_int", "form_1099_div", "form_w2", "property_tax_statement_ca", "property_tax_statement_india" | -| `asset_type_id` | INTEGER FK | Which asset type does this parser produce data for? | -| `institution_id` | INTEGER FK | Usually tied to one institution (nullable if format-generic) | -| `description` | TEXT | How to identify documents this parser handles | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - ---- - -### `bb_asset_to_parser` - -Links an asset to one or more parsers. One asset can have multiple parsers -(e.g., BofA checking has both PDF and CSV parsers). - -| Column | Type | Notes | -|---|---|---| -| `mapping_id` | INTEGER PK | Surrogate key | -| `asset_id` | INTEGER FK | References `bb_asset.asset_id` | -| `parser_id` | INTEGER FK | References `bb_asset_parser.parser_id` | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - ---- - -### `bb_document` - -First-class entity. Every document the user feeds to the system — bank statement, -tax form, property tax bill, mortgage statement — is tracked here. - -| Column | Type | Notes | -|---|---|---| -| `doc_id` | INTEGER PK | Surrogate key | -| `asset_id` | INTEGER FK | References `bb_asset.asset_id` — which asset does this belong to? Null if routing failed | -| `parser_id` | INTEGER FK | References `bb_asset_parser.parser_id` — which parser was used? | -| `doc_type` | TEXT NOT NULL | "bank_statement", "credit_card_statement", "1099_int", "1099_div", "1099_nec", "w2", "property_tax_statement", "mortgage_statement", "prior_tax_return", "salary_slip" | -| `jurisdiction` | TEXT | "US", "India", "UK" — for geographic and tax classification | -| `tax_year` | INTEGER | For tax documents: 2024, 2025, etc. | -| `file_name` | TEXT NOT NULL | Standardized name: `----.`. Example: `Ramesh-2025-1099INT-BofA-x1234.pdf` | -| `file_path` | TEXT NOT NULL | Local or iCloud path | -| `file_hash` | TEXT NOT NULL UNIQUE | SHA-256 of file contents — prevents re-importing same file | -| `fingerprint_json` | TEXT | Metadata extracted from document: `{"holder_name":"Ramesh K", "date_range":"2025-01-01/2025-01-31", "account_suffix":"1234", "total_amount":"5678.90"}` | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - ---- - -### `bb_import_activity` - -Complete import history. Every import attempt is logged here — success or failure. - -| Column | Type | Notes | -|---|---|---| -| `import_id` | INTEGER PK | Surrogate key | -| `doc_id` | INTEGER FK | References `bb_document.doc_id` — which document triggered this? | -| `asset_id` | INTEGER FK | References `bb_asset.asset_id` — where did the data go? | -| `parser_id` | INTEGER FK | References `bb_asset_parser.parser_id` — which parser was used? | -| `import_status` | TEXT NOT NULL | "success", "failed", "partial", "pending_review" | -| `rows_total` | INTEGER | Total rows parsed from document | -| `rows_imported` | INTEGER | New rows added to `bb_transaction` | -| `rows_duplicate` | INTEGER | Rows skipped as duplicates | -| `rows_error` | INTEGER | Rows that failed validation | -| `error_message` | TEXT | If status is "failed" or "partial" — why? | -| `import_date` | DATETIME NOT NULL | Timestamp of import attempt | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - ---- - -### `bb_transaction` - -The heart of the system. Every financial transaction from every asset. - -| Column | Type | Notes | -|---|---|---| -| `transaction_id` | INTEGER PK | Surrogate key | -| `asset_id` | INTEGER FK | References `bb_asset.asset_id` — which asset is this from? | -| `category_id` | INTEGER FK | References `bb_category.category_id` — defaults to Uncategorized | -| `import_id` | INTEGER FK | References `bb_import_activity.import_id` — which import brought this in? | -| `transaction_date` | DATE NOT NULL | | -| `amount` | REAL NOT NULL | Positive = inflow/credit. Negative = outflow/debit | -| `currency` | TEXT NOT NULL | "USD", "INR", "GBP" etc. — stored per transaction, not per asset | -| `description` | TEXT | Transaction remark from source document | -| `check_number` | TEXT | Optional | -| `transfer_pair_id` | TEXT | UUID linking two legs of an internal transfer between assets | -| `transaction_hash` | TEXT NOT NULL UNIQUE | SHA-256 of (asset_id + date + amount + currency + description) — row-level deduplication | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - -**Key changes from BankBuddy:** -- `asset_id` instead of `account_id` — works for any asset type -- `currency` per transaction — a single asset could have multi-currency transactions -- `transfer_pair_id` links any two assets, not just bank accounts - ---- - -### `bb_category` - -| Column | Type | Notes | -|---|---|---| -| `category_id` | INTEGER PK | Surrogate key | -| `category_name` | TEXT NOT NULL UNIQUE | "Salary", "Groceries", "Travel", "Transfer", "Uncategorized" | -| `is_system` | BOOLEAN NOT NULL | System categories cannot be deleted | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - -**Seed categories:** -Income: Salary, Interest, Dividends, Rental Income, Other Income -Expense: Groceries, Dining, Utilities, Travel, Healthcare, Shopping, Entertainment, -Education, Insurance, Rent/Mortgage, Property Tax, Other Expense -Special: Transfer (excluded from all reports), Uncategorized (fallback, undeletable) - ---- - -### `bb_category_rule` - -| Column | Type | Notes | -|---|---|---| -| `rule_id` | INTEGER PK | Surrogate key | -| `pattern` | TEXT NOT NULL | String or regex matched against transaction description | -| `category_id` | INTEGER FK | Target category | -| `priority` | INTEGER NOT NULL | Higher = wins over lower-priority rules | -| `is_user_defined` | BOOLEAN NOT NULL | True if created from a user correction | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - ---- - -### `bb_budget` - -| Column | Type | Notes | -|---|---|---| -| `budget_id` | INTEGER PK | Surrogate key | -| `category_id` | INTEGER FK | References `bb_category.category_id` | -| `currency` | TEXT NOT NULL | "USD", "INR" etc. — budget is per-currency | -| `budget_type` | TEXT NOT NULL | "monthly" or "annual" — not both for same category+currency | -| `min_amount` | REAL | Optional lower bound | -| `max_amount` | REAL | Optional upper bound | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - -**Unique constraint:** `(category_id, currency)` — one budget per category per currency. -Switching between monthly and annual requires deleting the old budget first. - ---- - -### `bb_tax_source` - -Inferred registry of entities expected to issue tax documents. - -| Column | Type | Notes | -|---|---|---| -| `source_id` | INTEGER PK | Surrogate key | -| `asset_id` | INTEGER FK | References `bb_asset.asset_id` — which asset is this source linked to? | -| `source_name` | TEXT NOT NULL | "Fidelity Brokerage", "Bank of America Savings", "ACME Corp" | -| `source_type` | TEXT NOT NULL | "bank", "brokerage", "employer", "property", "other" | -| `expected_forms` | TEXT NOT NULL | JSON array: `["1099-INT", "1099-DIV"]` | -| `jurisdiction` | TEXT NOT NULL | "US", "India" | -| `active` | BOOLEAN NOT NULL | False if this relationship is closed/ended | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - ---- - -### `bb_tax_profile` - -High-level snapshot of the user's tax situation per year. - -| Column | Type | Notes | -|---|---|---| -| `profile_id` | INTEGER PK | Surrogate key | -| `tax_year` | INTEGER NOT NULL UNIQUE | e.g. 2025 | -| `filing_status` | TEXT | "single", "married_filing_jointly", "married_filing_separately" | -| `has_rental_property` | BOOLEAN | | -| `has_investments` | BOOLEAN | | -| `has_self_employment` | BOOLEAN | | -| `has_foreign_income` | BOOLEAN | Income from outside primary country | -| `jurisdictions` | TEXT | JSON array: `["US", "India"]` | -| `notes` | TEXT | Free text | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - ---- - -### `bb_tax_form_gap` - -Tracks expected tax forms and whether they've been received. - -| Column | Type | Notes | -|---|---|---| -| `gap_id` | INTEGER PK | Surrogate key | -| `tax_year` | INTEGER NOT NULL | | -| `source_id` | INTEGER FK | References `bb_tax_source.source_id` | -| `expected_form_type` | TEXT NOT NULL | "1099-INT", "1099-DIV", "W-2", "1098", "property_tax_statement" | -| `expected_by_date` | DATE | Typical deadline for this form to arrive | -| `received_date` | DATE | Null if not yet received | -| `doc_id` | INTEGER FK | References `bb_document.doc_id` when received | -| `status` | TEXT NOT NULL | "pending", "received", "waived" | -| `created_at` | DATETIME NOT NULL | | -| `updated_at` | DATETIME NOT NULL | | - ---- - -## 6. Document Routing — The Fingerprint System - -This is the most important new architectural concept in AssetBuddy. - -### Problem - -Different documents identify the account/asset they belong to in different ways: -- Bank of America checking statement → contains full account number -- Apple Card statement → contains NO account number, not even last 4 digits -- Property tax statement → identified by property address + parcel number -- 1099-INT → identified by institution name + payer EIN + possibly account suffix - -Without a smart routing mechanism, you can't know which `bb_asset` a document belongs to. - -### Solution: Two-Level Routing - -**Level 1: Parser identification** -Read the top portion of every document to identify which parser handles it. -Pattern match on known signatures: -- "Bank of America" + column headers → `bof_a_checking_statement` -- "Apple Card" + "Goldman Sachs" → `apple_card_statement` -- "Form 1099-INT" → `form_1099_int` -- "COUNTY OF SANTA CLARA" + "Secured Property Tax" → `property_tax_statement_ca` - -**Level 2: Asset routing via fingerprint** -Once the parser is identified, look up `bb_asset_to_parser` to find candidate assets. - -``` -candidates = SELECT a.* FROM bb_asset a - JOIN bb_asset_to_parser ap ON a.asset_id = ap.asset_id - JOIN bb_asset_parser p ON ap.parser_id = p.parser_id - WHERE p.parser_name = -``` - -Then apply fingerprint matching: -- If exactly 1 candidate → route there (no fingerprint needed) -- If 0 candidates → error: "No asset configured for this parser" -- If multiple candidates → match document fingerprint against asset fingerprints - -**Fingerprint matching logic:** -- Extract key fields from document: holder name, account suffix, address, date range, totals -- Compare against `bb_asset.fingerprint_json` for each candidate -- If exactly 1 match → route there -- If still ambiguous → error: "Multiple assets matched — please clarify" -- If no match → error: "No asset fingerprint matched — please update asset fingerprint" - -### Fingerprint Examples - -**BofA Checking:** -```json -{"holder_name": "Ramesh K", "account_suffix": "1234", "account_type": "checking"} -``` - -**Apple Card:** -```json -{"holder_name": "Ramesh K", "card_member_since": "2020"} -``` - -**Property Tax Statement:** -```json -{"parcel_number": "111-22-333", "property_address": "123 Main St, Sunnyvale CA 94087"} -``` - -**1099-INT:** -```json -{"payer_name": "Bank of America", "payer_ein": "94-1687665", "account_suffix": "1234"} -``` - ---- - -## 7. Import Flow - -### Folder Structure -``` -~/AssetBuddy/ -├── inbox/ ← User drops all documents here -└── processed/ - ├── Bank of America/ - ├── Apple Card/ - ├── Fidelity/ - ├── Alameda County Assessor/ - └── IRS/ -``` - -Subdirectories named after the institution. Created automatically on first import. - -### Step-by-Step Import Process - -1. **Scan `inbox/`** for new files (PDF or spreadsheet) -2. **Compute SHA-256 hash** of the file -3. **Check `bb_document.file_hash`** — if exists, skip with message: "Already imported" -4. **Identify parser** from document content — if not confident, fail loudly -5. **Extract document fingerprint** from document metadata -6. **Route to asset** using parser + fingerprint matching (see Section 6) - - If routing fails (ambiguous or no match), log to `bb_import_activity` with - status "failed" and leave file in inbox -7. **Decrypt PDF** if password-protected (password stored in `bb_asset_attribute` - with `attr_type_name = "pdf_password"`) -8. **Parse document** using identified parser into staging area (in-memory) -9. **Validate staged data:** - - Required fields present (date, amount, description) - - Dates sensible (not future, not before 2000) - - Amounts are numeric - - No intra-batch duplicates -10. **Row-level deduplication:** Compute `transaction_hash` for each row. - Skip rows where hash already exists in `bb_transaction`. - Count and report skipped rows. -11. **Detect internal transfers:** Match debits and credits of same amount across - assets within a 2-day window. Assign shared `transfer_pair_id` UUID. -12. **Auto-categorize** using `bb_category_rule` ordered by priority. - Uncategorized → default "Uncategorized" category. -13. **Create `bb_document` record** with standardized filename and fingerprint -14. **Commit rows to `bb_transaction`** — only new, validated rows -15. **Create `bb_import_activity` record** with full summary -16. **Move original file** to `processed//` - -### On Failure (Steps 4–11) -- Log to `bb_import_activity` with status "failed" and error message -- Do NOT write to `bb_transaction` -- Leave original file in `inbox/` -- Print clear error to user - -### Import Summary Output -``` -File: hdfc_savings_nov2025.pdf -Institution: HDFC Bank | Asset: My HDFC Savings | Parser: hdfc_savings_xls -Total rows parsed: 87 -New rows imported: 54 -Duplicate rows skipped: 33 -Transfer pairs detected: 2 -Categories assigned: 51 (3 Uncategorized) -``` - ---- - -## 8. Categorization Engine - -### Phase 1: Rule-Based (Current) -- Match description against `bb_category_rule.pattern` ordered by `priority` descending -- Highest-priority matching rule wins -- No match → "Uncategorized" -- User manual override → auto-creates new rule with `is_user_defined = true` - -### Phase 2: ML-Assisted (scikit-learn) -- TF-IDF vectorization of transaction descriptions -- Naive Bayes or Logistic Regression classifier -- Training data: accumulated user-corrected transactions (~300+ examples to start) -- Model serialized to local file via `joblib` -- Low-confidence → fall back to "Uncategorized" rather than guess -- Retrain periodically as corrections accumulate - -### Phase 3: LLM Fallback -- Transactions below confidence threshold → call Claude API -- Send description + category list, get best match -- Handles novel/unusual descriptions gracefully - ---- - -## 9. CLI Command Surface - -### `asset-buddy` (or keep `bank-buddy` during transition) - -```bash -# Import -asset-buddy import # Process all new files in inbox/ -asset-buddy import --file FILE # Import specific file -asset-buddy import --status # Show import history -asset-buddy import --retry IMPORT_ID # Retry a failed import - -# Assets (replaces account management) -asset-buddy asset list # List all assets -asset-buddy asset add # Add asset (interactive) -asset-buddy asset show ASSET_ID # Show asset details and attributes -asset-buddy asset set-attr ASSET_ID TYPE VALUE # Set asset attribute -asset-buddy asset link-parser ASSET_ID PARSER # Link parser to asset -asset-buddy asset deactivate ASSET_ID # Mark asset as closed/archived - -# Transactions -asset-buddy tx list # List recent transactions -asset-buddy tx list --asset ASSET_ID -asset-buddy tx list --from DATE --to DATE -asset-buddy tx list --category CATEGORY -asset-buddy tx list --currency USD -asset-buddy tx list --uncategorized -asset-buddy tx list --transfers -asset-buddy tx categorize TX_ID CATEGORY # Override category - -# Reporting -asset-buddy report spending --year YEAR -asset-buddy report spending --year YEAR --month M -asset-buddy report income --year YEAR -asset-buddy report trend --category CAT --years 3 -asset-buddy report budget --year YEAR -asset-buddy report net-worth # Total across all assets - -# Budget -asset-buddy budget list -asset-buddy budget set CATEGORY CURRENCY TYPE MIN MAX -asset-buddy budget delete CATEGORY CURRENCY - -# Categories -asset-buddy category list -asset-buddy category add NAME -asset-buddy category rules list -asset-buddy category rules add PATTERN CATEGORY PRIORITY -asset-buddy category train # Retrain ML model - -# Tax -asset-buddy tax import # Import tax documents from inbox/ -asset-buddy tax profile --year YEAR # Show inferred tax profile -asset-buddy tax gaps --year YEAR # Show missing/pending forms -asset-buddy tax docs list --year YEAR -asset-buddy tax docs show DOC_ID -asset-buddy tax summary --year YEAR # Full tax picture - -# Setup -asset-buddy setup institution add # Add new institution -asset-buddy setup parser list # List available parsers -asset-buddy status # System overview -``` - ---- - -## 10. Product Positioning - -### Who This Is For - -**Primary user:** An engineer, accountant, or financially sophisticated person who: -- Has income or assets in more than one country -- Has multiple institution relationships (banks, brokerages, employer, properties) -- Values owning their data and doesn't want to share credentials with third parties -- Is comfortable downloading files and running commands -- Is frustrated with the complexity of tax season ("do I have all my 1099s?") - -**Secondary user:** The non-technical spouse or family member who needs read-only -visibility via a web dashboard. They should never need to touch the CLI. - -### What It Is NOT - -- Not a Mint replacement for the general public -- Not a tax filing tool (it is a tax *preparation awareness* tool) -- Not a tool that connects to banks via API -- Not a cloud service -- Not an App Store product (at least not initially) - -### Market Gap - -No consumer product today: -1. Unifies banking + tax document awareness in one tool -2. Is document-centric rather than OAuth-based -3. Handles dual-jurisdiction (e.g. India + US) tax tracking for NRIs -4. Runs fully locally with zero cloud dependency -5. Covers all asset types (not just bank accounts) - ---- - -## 11. Phased Roadmap - -### Phase 0 — Migration: BankBuddy → AssetBuddy -- [ ] Rename tables to `bb_` prefix with new singular names -- [ ] Create `bb_institution`, `bb_asset_type`, `bb_asset`, `bb_asset_attribute_type`, - `bb_asset_attribute` tables -- [ ] Create `bb_asset_parser` and `bb_asset_to_parser` tables -- [ ] Create `bb_document` and `bb_import_activity` tables -- [ ] Migrate existing BankBuddy data into new schema -- [ ] Update all existing parsers to work with new routing layer -- [ ] Update CLI commands to use asset-centric terminology -- [ ] Write migration script for existing users - -### Phase 1 — AssetBuddy Core (CLI) -- [ ] Fingerprint-based document routing -- [ ] Support for all existing parsers (BofA, HDFC, ICICI, Apple Card) -- [ ] New parsers: mortgage statements, 1099-INT, 1099-DIV, W-2 -- [ ] Property asset type with address, purchase price, rental days attributes -- [ ] Multi-currency transaction support -- [ ] Transfer detection between any two assets (not just bank accounts) -- [ ] Full CLI command surface as defined above -- [ ] Budget and categorization migrated to new schema - -### Phase 2 — Tax Layer -- [ ] `bb_tax_source`, `bb_tax_profile`, `bb_tax_form_gap` tables -- [ ] Tax document ingestion and routing -- [ ] Tax profile inference from documents -- [ ] Gap detection: "You're missing 1099-DIV from Fidelity" -- [ ] Annual tax readiness summary -- [ ] India tax document support (Form 26AS, salary slips) -- [ ] `tax-buddy` CLI (or `asset-buddy tax` subcommands) - -### Phase 3 — Intelligence & Automation -- [ ] ML-based categorization (scikit-learn) -- [ ] LLM fallback categorization (Claude API) -- [ ] File watcher daemon (`watchdog`) for auto-import -- [ ] macOS `launchd` integration for daemon at login -- [ ] macOS native notifications on import completion -- [ ] iCloud document storage for family sharing - -### Phase 4 — Web Interface -- [ ] Read-only web dashboard (Node.js) -- [ ] Spending trends, income trends, net worth over time -- [ ] Budget vs actuals visualization -- [ ] Tax document status and gap view -- [ ] Multi-year trend views -- [ ] Accessible to non-technical family members -- [ ] Full management capability for primary user - -### Phase 5 — Mobile & Beyond -- [ ] iOS app (local install via Xcode — not App Store initially) -- [ ] Read-only viewer: transactions, budgets, tax gaps -- [ ] Document upload from phone (drag documents from email) -- [ ] Multi-user / household support -- [ ] User authentication for web UI - ---- - -## 12. Open Questions for Implementation - -- [ ] **Migration strategy:** How do existing BankBuddy users migrate? Is there a - `asset-buddy migrate` command that reads old schema and transforms it? -- [ ] **Household / multi-user:** Should `bb_asset` have an `owner_id` field now - or is this deferred? Better to add it now than to bolt it on later. -- [ ] **Fingerprint storage format:** JSON in a single column vs separate - `bb_asset_fingerprint` table. JSON is simpler; table is more queryable. -- [ ] **PDF password storage:** Currently plaintext in `bb_asset_attribute`. - Future: encrypt using macOS Keychain. -- [ ] **iCloud path vs local path:** `bb_document.file_path` — should this be an - abstract path (resolved at runtime) or an absolute path? -- [ ] **India tax document types:** Form 26AS, AIS, ITR acknowledgment — need - parser definitions for these. -- [ ] **Property depreciation calculation:** How much does the app compute vs - just store the inputs and let a tax professional handle it? -- [ ] **CLI naming:** Keep `bank-buddy` as the CLI name during transition, or - rename to `asset-buddy` immediately? -- [ ] **AGPL v3 license:** All repos (base, bank-buddy, banyan-labs) to be - relicensed from MIT to AGPL v3. - ---- - -## 13. License - -All AssetBuddy/BankBuddy code is (or will be) licensed under -**GNU Affero General Public License v3.0 (AGPL v3)**. - -This means: -- Anyone can use, modify, and distribute the code -- If you modify it and run it as a network service, you must open source your changes -- Commercial use requires either complying with AGPL or negotiating a separate license - -The author is the sole contributor. No external contributions complicate the -relicensing from MIT to AGPL v3. - ---- - -*Document prepared June 2026. Ready for architecture review and phased implementation.* diff --git a/docs/financial_intelligence_architecture_review.md b/docs/financial_intelligence_architecture.md similarity index 95% rename from docs/financial_intelligence_architecture_review.md rename to docs/financial_intelligence_architecture.md index 1f1054a..45f4768 100644 --- a/docs/financial_intelligence_architecture_review.md +++ b/docs/financial_intelligence_architecture.md @@ -1,21 +1,17 @@ -# BankBuddy Financial Intelligence Architecture Review +# BankBuddy Financial Intelligence Architecture **Date:** 2026-06-15 -**Status:** Architecture review and implementation proposal -**Inputs reviewed:** +**Status:** Current v2 architecture direction +**Companion documents:** - `docs/financial_intelligence_vision.md` -- `docs/financial_intelligence_architecture_spec.md` -- `docs/assetbuddy_design.md` -- `bank_buddy_spec.md` -- `.ai-context/ARCHITECTURE.md` -- Current `src/bankbuddy/` modules, migrations, and CLI surfaces +- `docs/financial_intelligence_open_questions.md` -## Executive Recommendation +## Executive Summary BankBuddy should evolve into the broader personal financial intelligence -platform described in the new design documents, but it should remain in the -existing BankBuddy repository. +platform described in the product vision, while remaining in the existing +BankBuddy repository and keeping the `bankbuddy` CLI name. The right target architecture is not a banking app with more tables. It is a local-first evidence graph: @@ -34,9 +30,9 @@ data can be discarded, the cleanest path is a deliberate schema reset into a versioned v2 model, not a long compatibility migration that preserves every old bank/account assumption. -## Review Decisions +## Architecture Decisions -The following decisions were accepted after review: +The following decisions define the current v2 direction: - Keep the product and CLI name as `bankbuddy`. - Use `Document`, `Entity`, `Observation`, and `Relationship` as the conceptual @@ -1286,7 +1282,7 @@ sensitive. Mitigation: classify attribute sensitivity, avoid raw text in normal logs, mask CLI output, and defer durable text indexing until encryption is designed. -## 15. Resolved Decisions and Remaining Topics +## 15. Resolved Decisions Resolved decisions: @@ -1302,25 +1298,16 @@ Resolved decisions: encryption/keychain design as later hardening. 8. Start net-worth reporting with native-currency buckets only. -Remaining discussion topics: - -1. Whether `infer` needs to be user-facing, or whether import plus inspect is - sufficient for the early product. -2. Whether raw extracted text should be stored by default, stored only for - selected document types, or stored only as an optional debug/audit feature. -3. The exact managed storage layout for successful, failed, duplicate, and - review-needed documents. -4. The initial set of `BB_ENTITY_ATTRIBUTE_TYPE`, `BB_RELATIONSHIP_TYPE`, and - `BB_OBSERVATION_TYPE` seed values. +Open design topics are tracked in +`docs/financial_intelligence_open_questions.md`. ## 16. Bottom Line -The new design direction is sound, but it should not be implemented as -"BankBuddy plus more tables." The right move is to preserve the proven local -runtime and import mechanics while replacing the domain center with documents, -entities, observations, and relationships. +The new design direction should not be implemented as "BankBuddy plus more +tables." The right move is to preserve the proven local runtime and import +mechanics while replacing the domain center with documents, entities, +observations, and relationships. -I recommend approving a v2 architecture effort before implementing issue #100. -Issue #100 should either be reframed as the first Tax Readiness slice on the new -document/entity foundation, or postponed until the generic document and -observation model exists. +The next architecture implementation slice is the v2 foundation schema. TaxBuddy +issue #100 stays paused until the generic document/entity/observation model +exists, then it can be reframed as a Tax Readiness projection over that model. diff --git a/docs/financial_intelligence_architecture_spec.md b/docs/financial_intelligence_architecture_spec.md deleted file mode 100644 index 4e6c682..0000000 --- a/docs/financial_intelligence_architecture_spec.md +++ /dev/null @@ -1,1569 +0,0 @@ -# BankBuddy Financial Intelligence Platform -## Comprehensive Architecture Specification - -**Status:** Draft architecture specification -**Audience:** Codex, future maintainers, product/design review -**Source:** Design discussion expanding BankBuddy from a banking tool into a local-first personal financial intelligence platform. - ---- - -# 1. Executive Summary - -BankBuddy began as a personal banking application focused on accounts, balances, and transactions. - -The product direction has now expanded. - -The new vision is to evolve BankBuddy into a **Personal Financial Intelligence Platform**: a local-first, privacy-preserving system that helps individuals understand their complete financial life across documents, accounts, assets, liabilities, jurisdictions, currencies, and time. - -This is not a tax filing product. - -This is not a CPA replacement. - -This is not a professional financial advisory tool. - -The goal is to help ordinary individuals gain clarity over their financial world by organizing source documents, extracting facts, inferring relationships, detecting gaps, and producing understandable views of their financial position. - -The most important new capability is the distinction between: - -- **Import** — bring documents and structured files into the system. -- **Infer** — derive financial facts and relationships from documents. -- **Inspect** — determine completeness, gaps, inconsistencies, and year-over-year changes. - ---- - -# 2. Product Vision - -## 2.1 Mission - -BankBuddy should help a user answer: - -- What financial accounts do I have? -- What assets do I own? -- What liabilities do I owe? -- What documents prove or explain these things? -- What changed between last year and this year? -- What documents are missing? -- What accounts or assets appear in one year but not another? -- What institutions am I financially connected to? -- What is my total picture across countries and currencies? - -The intended feeling for the user is: - -> “I finally understand my financial picture.” - ---- - -## 2.2 Non-Goals - -BankBuddy should not initially attempt to: - -- File tax returns. -- Replace a CPA. -- Provide legal advice. -- Provide tax advice. -- Recommend investments. -- Optimize tax strategy. -- Connect to live bank feeds before the local model is solid. -- Become enterprise accounting software. - -The platform is for personal understanding, organization, and insight. - ---- - -# 3. Design Principles - -## 3.1 Local First - -The primary architecture must be local-first. - -All core functionality should run on the user's machine: - -- Data ingestion -- Document storage references -- Metadata extraction -- SQLite database -- Inference engine -- Inspection reports -- CLI workflows -- Future local UI - -No cloud service should be required for the product to function. - ---- - -## 3.2 Privacy First - -Financial documents are deeply sensitive. - -The default posture: - -- Do not upload user documents. -- Do not require remote processing. -- Do not store personal financial data outside the local machine. -- Treat document contents, extracted facts, and inferred observations as private user data. - -Any future cloud sync must be explicit, optional, and user-controlled. - ---- - -## 3.3 Cloud Optional - -Future synchronization can be considered through user-controlled platforms such as: - -- Apple CloudKit -- iCloud private containers -- Local network sync -- User-owned encrypted storage - -Cloud sync must not become a requirement. - ---- - -## 3.4 AI-Assisted, Not AI-Dependent - -AI may assist with: - -- Document classification -- Field extraction -- Account/entity recognition -- Relationship inference -- Summarization -- Missing-document detection -- Natural-language explanations - -But the system should retain explicit structured data models and deterministic inspection logic. - -AI output should be treated as proposed observations, not unquestioned truth. - ---- - -## 3.5 Traceability - -Every inferred fact should be traceable back to source material. - -A user should be able to ask: - -- Where did this account come from? -- Which document showed this amount? -- Why did the system infer this institution? -- What evidence supports this observation? - -This implies a strong provenance model. - ---- - -## 3.6 Refactoring Is Allowed - -Existing BankBuddy code should not constrain the correct long-term design. - -Allowed: - -- Schema redesign -- Entity redesign -- Package restructuring -- CLI redesign -- Service boundary redesign -- Test reorganization - -Backward compatibility is not a primary concern at this stage. - -Existing data can be discarded if necessary. - ---- - -# 4. Conceptual Product Expansion - -## 4.1 Original BankBuddy Scope - -The original application was banking-oriented. - -Likely concepts: - -- Bank accounts -- Transactions -- Balances -- Institutions -- Imports -- Reports - -This remains valuable but becomes one part of a larger model. - ---- - -## 4.2 Expanded Scope - -BankBuddy should support the following domains. - -### Banking - -- Checking accounts -- Savings accounts -- Certificates of deposit -- Money market accounts -- Interest income - -### Investments - -- Brokerage accounts -- Stocks -- ETFs -- Mutual funds -- Bonds -- Dividends -- Capital gains documents - -### Retirement - -- 401(k) -- IRA -- Roth IRA -- Pension accounts -- Retirement statements - -### Real Estate - -- Primary residence -- Rental properties -- Property tax documents -- Mortgage documents -- Rental income records - -### Liabilities - -- Mortgages -- HELOCs -- Credit cards -- Personal loans -- Auto loans - -### Insurance - -- Health insurance -- Life insurance -- Auto insurance -- Property insurance -- COBRA / ACA documents - -### Tax Documents - -- W-2 -- 1099-INT -- 1099-DIV -- 1099-B -- 1099-R -- K-1 -- Property tax forms -- Foreign income documents -- India tax documents -- Brokerage tax packages - -### International Assets - -- US accounts -- India accounts -- Foreign real estate -- Foreign rental income -- Multi-currency balances - ---- - -# 5. Core Architectural Vocabulary - -The foundational architecture should be built around three primary abstractions: - -```text -Document -Entity -Observation -``` - -These three concepts allow the system to remain general, extensible, and traceable. - ---- - -## 5.1 Document - -A **Document** is an imported artifact. - -Examples: - -- PDF tax return -- Bank statement -- Brokerage statement -- Spreadsheet -- CSV export -- Property tax bill -- Rental income statement -- Insurance policy -- Loan statement - -Documents are source material. - -Documents should be treated as immutable once imported. - -Possible document properties: - -- Document ID -- File path -- Original filename -- File hash -- MIME type -- Document type -- Source institution -- Jurisdiction -- Tax year -- Statement period start -- Statement period end -- Import timestamp -- Processing status - ---- - -## 5.2 Entity - -An **Entity** is a real-world financial object discovered or manually entered. - -Examples: - -- Person -- Institution -- Account -- Property -- Loan -- Insurance policy -- Investment holding -- Tax form issuer -- Currency -- Jurisdiction - -Entities are the durable objects in the user's financial graph. - -Possible entity types: - -- `person` -- `institution` -- `account` -- `asset` -- `liability` -- `property` -- `investment_holding` -- `insurance_policy` -- `tax_form` -- `currency` -- `jurisdiction` - ---- - -## 5.3 Observation - -An **Observation** is a fact asserted or inferred from a document. - -Examples: - -- “This document contains a 1099-INT from Capital One.” -- “Interest income of $3,200 was reported for 2025.” -- “Account ending in 1234 existed in 2025.” -- “A rental property generated income in 2025.” -- “A mortgage liability existed during the statement period.” -- “This Indian bank account had INR-denominated funds.” - -Observations should have provenance. - -Possible observation properties: - -- Observation ID -- Observation type -- Subject entity ID -- Related entity ID -- Source document ID -- Tax year -- Amount -- Currency -- Date or date range -- Confidence score -- Extraction method -- Human review status -- Raw extracted text reference -- Notes - -Observation confidence is important because AI-generated extraction may be uncertain. - ---- - -# 6. Import → Infer → Inspect Architecture - -The system should explicitly separate these workflows. - ---- - -## 6.1 Import - -Import answers: - -> “What files or structured records have entered the system?” - -Responsibilities: - -- Accept files from CLI. -- Compute file hash. -- Store document metadata. -- Classify document type if possible. -- Avoid duplicate imports. -- Record source path. -- Optionally copy files into a managed local document store. -- Extract basic text and metadata. - -Example CLI: - -```bash -bankbuddy import ~/Documents/Taxes/2025/1099-capitalone.pdf -bankbuddy import ~/Downloads/fidelity-2025-tax-package.pdf --tax-year 2025 -bankbuddy import ~/Documents/India/icici-statement.xlsx --jurisdiction IN --currency INR -``` - ---- - -## 6.2 Infer - -Infer answers: - -> “What can we conclude from the available documents?” - -Responsibilities: - -- Parse document contents. -- Extract candidate entities. -- Extract candidate observations. -- Match observations to existing entities. -- Detect new entities. -- Detect relationships. -- Compare across years. -- Produce inference reports. - -Examples: - -- A 1099-INT implies a bank relationship. -- A 1099-DIV implies brokerage or investment holdings. -- Rental income implies rental property. -- Mortgage interest implies mortgage liability. -- Foreign bank statement implies foreign financial account. -- A prior-year account missing in current year implies a possible missing document or closed account. - -Example CLI: - -```bash -bankbuddy infer --tax-year 2025 -bankbuddy infer --document DOC123 -bankbuddy infer --compare-years 2024 2025 -``` - ---- - -## 6.3 Inspect - -Inspect answers: - -> “What do we have, what is missing, and what needs attention?” - -Responsibilities: - -- Show imported documents by year. -- Show documents by institution. -- Show accounts with missing statements. -- Show tax years with incomplete coverage. -- Show inferred entities needing review. -- Show confidence warnings. -- Show year-over-year changes. - -Examples: - -```bash -bankbuddy inspect --tax-year 2025 -bankbuddy inspect documents --tax-year 2025 -bankbuddy inspect gaps --compare-years 2024 2025 -bankbuddy inspect entities --unreviewed -``` - ---- - -# 7. Domain Model - -## 7.1 Person - -Represents a human connected to the financial graph. - -Examples: - -- User -- Spouse -- Dependent -- Joint account holder - -Suggested fields: - -- `id` -- `display_name` -- `role` -- `notes` -- `created_at` -- `updated_at` - ---- - -## 7.2 Institution - -Represents an organization. - -Examples: - -- Bank of America -- Fidelity -- ICICI Bank -- UnitedHealthcare -- County tax office - -Suggested fields: - -- `id` -- `name` -- `institution_type` -- `country_code` -- `website` -- `notes` - ---- - -## 7.3 Account - -Represents a financial account. - -Examples: - -- Checking account -- Savings account -- Brokerage account -- 401(k) -- IRA -- NRE account -- NRO account - -Suggested fields: - -- `id` -- `institution_id` -- `account_type` -- `display_name` -- `masked_account_number` -- `currency_code` -- `jurisdiction_code` -- `opened_date` -- `closed_date` -- `status` -- `notes` - ---- - -## 7.4 Asset - -Represents something owned. - -Examples: - -- Bank balance -- Brokerage portfolio -- Real estate -- Stock position -- Retirement balance - -Suggested fields: - -- `id` -- `asset_type` -- `display_name` -- `currency_code` -- `jurisdiction_code` -- `owner_person_id` -- `related_account_id` -- `notes` - ---- - -## 7.5 Liability - -Represents something owed. - -Examples: - -- Mortgage -- HELOC -- Credit card balance -- Auto loan - -Suggested fields: - -- `id` -- `liability_type` -- `display_name` -- `institution_id` -- `currency_code` -- `jurisdiction_code` -- `related_asset_id` -- `opened_date` -- `closed_date` -- `status` -- `notes` - ---- - -## 7.6 Property - -Real estate deserves explicit modeling. - -Suggested fields: - -- `id` -- `display_name` -- `property_type` -- `country_code` -- `region` -- `city` -- `currency_code` -- `ownership_type` -- `notes` - -Avoid storing precise street addresses unless the user explicitly wants them. - ---- - -## 7.7 Document - -Covered earlier. - ---- - -## 7.8 Observation - -Covered earlier. - ---- - -## 7.9 Relationship - -Relationships connect entities. - -Examples: - -- Person owns account. -- Account belongs to institution. -- Property has mortgage liability. -- Document supports account. -- Observation references asset. -- Institution issued document. - -Suggested fields: - -- `id` -- `source_entity_id` -- `relationship_type` -- `target_entity_id` -- `source_document_id` -- `confidence` -- `valid_from` -- `valid_to` -- `notes` - ---- - -# 8. Proposed ER Model - -Initial conceptual ER structure: - -```text -Person - └── owns/manages ── Account - └── held_at ── Institution - -Person - └── owns ── Asset - └── may_be_backed_by ── Account - -Person - └── owes ── Liability - └── owed_to ── Institution - -Property - └── secured_by ── Liability - -Document - └── produces ── Observation - └── references ── Entity - -Observation - └── supported_by ── Document - -Entity - └── related_to ── Entity -``` - -The key is not to overfit early. - -The `Entity` and `Relationship` abstraction allows broad modeling without requiring a separate table for every possible real-world object immediately. - ---- - -# 9. Suggested SQLite Schema Direction - -This is a starting point, not final DDL. - -## 9.1 Core Tables - -```sql -documents -document_texts -entities -entity_attributes -relationships -observations -observation_evidence -imports -processing_runs -``` - ---- - -## 9.2 Reference Tables - -```sql -currencies -jurisdictions -institution_aliases -document_type_patterns -entity_type_definitions -observation_type_definitions -``` - ---- - -## 9.3 Financial Tables - -Depending on the current BankBuddy structure, retain or redesign: - -```sql -accounts -transactions -balances -assets -liabilities -properties -holdings -``` - -These can either be first-class tables or specialized entity profiles. - ---- - -## 9.4 Recommended Hybrid Model - -Use both: - -1. Generic graph-like tables: - - `entities` - - `relationships` - - `observations` - -2. Domain-specific tables: - - `accounts` - - `transactions` - - `balances` - - `assets` - - `liabilities` - - `properties` - -This gives flexibility and queryability. - ---- - -# 10. Observation Model - -Observation types should be explicit. - -Examples: - -```text -account_existence -account_balance -interest_income -dividend_income -capital_gain -rental_income -mortgage_interest -insurance_coverage -tax_form_received -foreign_account_presence -property_ownership -liability_existence -``` - -Each observation should include: - -- Type -- Source document -- Subject entity -- Amount if applicable -- Currency if applicable -- Period -- Confidence -- Extraction method -- Review status - -Review statuses: - -```text -new -accepted -rejected -needs_review -superseded -``` - ---- - -# 11. Inference Engine Design - -## 11.1 Pipeline - -```text -Document - -> classify - -> extract text - -> extract fields - -> detect entities - -> generate observations - -> reconcile with existing graph - -> produce inference report -``` - ---- - -## 11.2 Rule-Based Inference - -Start with deterministic rules. - -Examples: - -```text -If document type is 1099-INT: - create/confirm institution - create interest_income observation - create account_existence observation if account identifier is present - -If document type is mortgage statement: - create/confirm liability - create mortgage_interest observation - link liability to property if known - -If document contains rental income: - create/confirm property asset - create rental_income observation -``` - ---- - -## 11.3 AI-Assisted Inference - -AI can be added behind explicit interfaces. - -Example interface: - -```python -class DocumentInterpreter: - def classify(document) -> DocumentClassification - def extract_entities(document) -> list[EntityCandidate] - def extract_observations(document) -> list[ObservationCandidate] -``` - -Possible implementations: - -- Regex/rule-based interpreter -- Local ML model interpreter -- Optional LLM interpreter -- Manual interpreter - -The rest of the architecture should not care which interpreter generated the candidates. - ---- - -## 11.4 Confidence and Human Review - -Every inferred item should have: - -- Confidence score -- Evidence reference -- Review status - -High-confidence deterministic facts can be auto-accepted. - -Low-confidence AI facts should require review. - ---- - -# 12. Inspect Engine Design - -Inspect should be deterministic and report-oriented. - -Examples of inspection reports: - -## 12.1 Document Completeness Report - -Questions: - -- Which documents exist for tax year 2025? -- Which institutions have documents? -- Which expected documents are missing? - -## 12.2 Account Continuity Report - -Questions: - -- Which accounts existed in 2024? -- Which accounts exist in 2025? -- Which disappeared? -- Which are new? - -## 12.3 Jurisdiction Report - -Questions: - -- Which assets are US-based? -- Which assets are India-based? -- Which documents support each jurisdiction? - -## 12.4 Currency Exposure Report - -Questions: - -- What assets are denominated in USD? -- What assets are denominated in INR? -- What is the converted base-currency value? - -## 12.5 Review Queue - -Questions: - -- Which observations need human review? -- Which entities were inferred but not accepted? -- Which documents failed parsing? - ---- - -# 13. Multi-Year Timeline Architecture - -Time is central to the platform. - -The system should model: - -- Calendar year -- Tax year -- Statement period -- Observation date -- Account open/close range -- Asset valuation date -- Liability balance date - -Useful commands: - -```bash -bankbuddy timeline --entity ACCOUNT123 -bankbuddy compare --years 2024 2025 -bankbuddy inspect gaps --years 2024 2025 -``` - ---- - -# 14. Multi-Currency Architecture - -Currencies should be first-class. - -Required fields: - -- Native currency -- Base reporting currency -- FX rate date -- FX source -- Converted amount - -Suggested tables: - -```sql -currencies -exchange_rates -amounts -``` - -Do not overwrite native values. - -Always preserve: - -- Original amount -- Original currency -- Converted amount -- Conversion rate -- Conversion date - ---- - -# 15. Multi-Jurisdiction Architecture - -Jurisdiction should be metadata, not hard-coded tax logic. - -Track: - -- Country code -- State/region if needed -- Institution jurisdiction -- Asset jurisdiction -- Account jurisdiction -- Document jurisdiction -- Tax year - -Start simple: - -```text -US -IN -``` - -Avoid building a tax engine initially. - -Instead, produce organization and completeness insights. - ---- - -# 16. CLI Architecture - -The CLI should remain the first product interface. - -Suggested command groups: - -```bash -bankbuddy init -bankbuddy import -bankbuddy documents -bankbuddy infer -bankbuddy inspect -bankbuddy entities -bankbuddy accounts -bankbuddy assets -bankbuddy liabilities -bankbuddy properties -bankbuddy observations -bankbuddy review -bankbuddy report -bankbuddy db -``` - ---- - -## 16.1 Example CLI Workflows - -### Import a document - -```bash -bankbuddy import ~/Documents/Taxes/2025/fidelity-tax-package.pdf --tax-year 2025 -``` - -### Infer from imported documents - -```bash -bankbuddy infer --tax-year 2025 -``` - -### Inspect current tax year completeness - -```bash -bankbuddy inspect --tax-year 2025 -``` - -### Compare two years - -```bash -bankbuddy inspect gaps --years 2024 2025 -``` - -### Review AI-generated observations - -```bash -bankbuddy review observations --status needs_review -``` - -### Accept an observation - -```bash -bankbuddy review accept OBS123 -``` - ---- - -# 17. Package / Module Structure - -Suggested Python package structure: - -```text -bankbuddy/ - __init__.py - cli/ - __init__.py - main.py - import_cmd.py - infer_cmd.py - inspect_cmd.py - review_cmd.py - report_cmd.py - - core/ - __init__.py - config.py - paths.py - errors.py - types.py - - db/ - __init__.py - connection.py - migrations.py - schema.sql - repositories/ - - domain/ - __init__.py - documents.py - entities.py - observations.py - relationships.py - accounts.py - assets.py - liabilities.py - properties.py - currencies.py - jurisdictions.py - - importers/ - __init__.py - base.py - pdf_importer.py - csv_importer.py - spreadsheet_importer.py - - interpreters/ - __init__.py - base.py - rule_based.py - llm_optional.py - - inference/ - __init__.py - engine.py - rules/ - tax_forms.py - banking.py - investments.py - real_estate.py - insurance.py - - inspection/ - __init__.py - engine.py - reports/ - documents.py - gaps.py - continuity.py - currency.py - jurisdiction.py - - review/ - __init__.py - workflow.py - - reporting/ - __init__.py - markdown.py - json.py - tables.py - -tests/ - unit/ - integration/ - fixtures/ -``` - ---- - -# 18. Testing Strategy - -Testing should include: - -## 18.1 Unit Tests - -- Document classification -- Entity creation -- Observation creation -- Rule inference -- Currency conversion -- Relationship creation - -## 18.2 Integration Tests - -- Import sample document -- Run inference -- Inspect output -- Verify database state - -## 18.3 Fixture Documents - -Use synthetic fixture documents. - -Do not commit real financial documents. - -Create artificial examples: - -- Fake 1099-INT -- Fake bank statement -- Fake mortgage statement -- Fake rental summary -- Fake India bank statement - ---- - -# 19. Security and Privacy - -## 19.1 Data Storage - -Use local SQLite. - -Default database location might be: - -```text -~/.bankbuddy/bankbuddy.db -``` - -Documents can remain in place or be copied into: - -```text -~/.bankbuddy/documents/ -``` - -If copied, preserve original filename and hash. - ---- - -## 19.2 Sensitive Data Handling - -Avoid logging: - -- Account numbers -- Full document text -- SSNs -- Tax IDs -- Precise addresses -- Full financial details - -Use masked identifiers in CLI output. - ---- - -## 19.3 Optional Encryption - -Future: - -- SQLCipher -- macOS Keychain integration -- Encrypted document store - -Not required for first architecture pass, but leave room for it. - ---- - -# 20. Future UI Architecture - -## 20.1 Local Web UI - -Possible stack: - -- Python backend -- FastAPI -- SQLite -- React or simple server-rendered UI - -Advantages: - -- Cross-platform -- Easier rapid iteration -- Works with existing Python backend - ---- - -## 20.2 Native macOS App - -Possible stack: - -- Swift -- SwiftUI -- SQLite access -- Local service bridge to Python engine if needed - -Advantages: - -- Native Apple experience -- Better document handling -- Better iCloud / CloudKit future path -- Better iPhone/iPad extension later - ---- - -## 20.3 iOS / iPadOS App - -Possible only after the domain and local data model are stable. - -Potential uses: - -- Read-only dashboard -- Document capture -- Review queue -- Notifications -- iCloud sync - ---- - -## 20.4 CloudKit Future - -CloudKit can be considered for private user sync. - -Design requirement: - -- Sync should be optional. -- User data should remain private. -- Local-first architecture must remain valid without sync. - ---- - -# 21. Migration / Refactoring Strategy - -Since existing data can be discarded, prefer clean architecture. - -Suggested approach: - -## Phase 0: Architecture Review - -Codex should inspect current code and produce: - -- Existing module map -- Existing schema -- Current CLI commands -- Current domain model -- Current test coverage -- Reusable components -- Components to discard - -No implementation in this phase. - ---- - -## Phase 1: New Core Domain - -Introduce: - -- Document -- Entity -- Observation -- Relationship - -Add migrations and repositories. - ---- - -## Phase 2: Import System - -Implement: - -- File import -- Hashing -- Metadata capture -- Duplicate detection -- Document listing - ---- - -## Phase 3: Rule-Based Inference - -Implement simple inference rules for synthetic fixtures. - -Start with: - -- Bank statement -- 1099-INT-like document -- Brokerage statement -- Mortgage statement - ---- - -## Phase 4: Inspect Reports - -Implement: - -- Document completeness -- Entity listing -- Observation review queue -- Year-over-year gaps - ---- - -## Phase 5: Multi-Currency and Jurisdiction - -Add: - -- Currency metadata -- Exchange rate table -- Jurisdiction metadata -- Base currency reporting - ---- - -## Phase 6: Review Workflow - -Implement: - -- Accept observation -- Reject observation -- Mark needs review -- Link observation to entity - ---- - -## Phase 7: UI Preparation - -Stabilize APIs for future UI. - -Expose: - -- JSON reports -- Domain services -- Read-only dashboard endpoints if web UI is chosen - ---- - -# 22. Codex Work Instructions - -When using Codex, give it this instruction: - -```text -Read docs/bankbuddy-financial-intelligence-architecture.md completely. - -Do not implement immediately. - -First inspect the existing BankBuddy codebase and produce an architecture review. - -The goal is to evolve BankBuddy from a banking-focused app into the local-first financial intelligence platform described in the document. - -Backward compatibility is not required. - -Existing data can be discarded. - -You may propose schema redesign, package restructuring, domain model redesign, and CLI redesign. - -Produce the following first: - -1. Current-state assessment -2. Gap analysis -3. Proposed target architecture -4. Proposed database schema -5. Proposed package/module structure -6. Proposed CLI design -7. Refactoring plan -8. Phased implementation roadmap - -Wait for approval before changing code. -``` - ---- - -# 23. Open Design Questions - -These should be resolved during implementation planning. - -## 23.1 Document Storage - -Should imported documents be: - -- Referenced in place? -- Copied into managed storage? -- Both? - -Recommendation: - -Start with reference-in-place plus hash. -Add managed copy later. - ---- - -## 23.2 AI Model Strategy - -Should AI processing be: - -- Local only? -- Optional remote LLM? -- Configurable? - -Recommendation: - -Start with rule-based inference. -Define interfaces for AI later. - ---- - -## 23.3 UI Direction - -Should the first UI be: - -- CLI only? -- Local web? -- Native Swift? - -Recommendation: - -CLI first. -Then local web or Swift after domain model stabilizes. - ---- - -## 23.4 Tax Rules - -Should tax-specific rules be added? - -Recommendation: - -Only use tax documents as financial evidence initially. -Avoid tax advice and filing logic. - ---- - -# 24. Final Product Direction - -BankBuddy should become a trusted local financial intelligence system. - -It should help users: - -- Organize financial documents -- Understand their assets -- Understand their liabilities -- Track financial history -- Compare years -- Detect missing documents -- Detect new or missing accounts -- Understand multi-country financial relationships -- Preserve privacy - -The core thesis: - -> The user's financial life is a graph of documents, entities, observations, and relationships over time. - -BankBuddy should model that graph clearly, locally, and safely. diff --git a/docs/financial_intelligence_open_questions.md b/docs/financial_intelligence_open_questions.md new file mode 100644 index 0000000..b8ade4b --- /dev/null +++ b/docs/financial_intelligence_open_questions.md @@ -0,0 +1,137 @@ +# BankBuddy Financial Intelligence Open Questions + +**Status:** Living design queue +**Companion documents:** + +- `docs/financial_intelligence_vision.md` +- `docs/financial_intelligence_architecture.md` + +This document tracks unresolved design topics for the financial intelligence v2 +architecture. Resolved decisions belong in the architecture document. Stale +questions should be removed when the implementation or a later design decision +settles them. + +## User-Facing Infer + +Early v2 might not need a user-facing `bankbuddy infer` command. Deterministic +parsers can create observations during import, and inspect/report commands can +surface gaps and review items. + +Open question: + +- Should `infer` become a separate user command, or should inference remain an + internal service until import plus inspect proves insufficient? + +Current bias: + +- Build the v2 schema and observation model first. +- Keep source documents read-only for any future infer operation. +- Add user-facing infer only when there is a clear workflow that import, + inspect, and review do not cover. + +## Raw Extracted Text Storage + +Raw extracted text can make debugging, provenance, search, and future inference +much better. It also increases privacy risk because statements, tax forms, +account numbers, addresses, and identifiers may be stored outside the original +document. + +Open questions: + +- Should raw extracted text be stored by default? +- Should it be stored only for selected document types? +- Should it be an optional debug/audit feature? +- Should durable raw text storage wait for encryption/keychain support? + +Current bias: + +- Allow the architecture to support extracted text. +- Avoid requiring raw text persistence in the first v2 schema slice unless it is + needed for tests, provenance, or inspect commands. +- Keep debug logs and normal CLI output free of raw document content. + +## Managed Document Storage Layout + +BankBuddy already moves completed imports into managed storage. The v2 model +needs a generic document layout that works for banking, tax, investment, +property, insurance, failed imports, duplicates, and review-needed documents. + +Open questions: + +- What is the exact directory layout for successful documents? +- Should failed documents live under a shared failure area or under a + domain-specific path? +- Should exact duplicates continue to be preserved under a duplicates + directory, or eventually be deleted after recording the duplicate attempt? +- How should review-needed documents be named and grouped? + +Current bias: + +- Preserve source documents after every completed import attempt. +- Keep duplicate preservation for now because it is useful while the product is + still young. +- Store canonical paths in the database, but keep directory-shape assumptions + in path/layout services rather than spreading them through domain logic. + +## Type Dictionary Seed Values + +The v2 schema should use typed dictionaries instead of arbitrary JSON keys. +The first schema slice needs enough seed data to support accounts, people, +institutions, documents, observations, and relationships without pretending to +cover every future domain. + +Open questions: + +- What initial values belong in `BB_ENTITY_ATTRIBUTE_TYPE`? +- What initial values belong in `BB_RELATIONSHIP_TYPE`? +- What initial values belong in `BB_OBSERVATION_TYPE`? +- Which type values are system-managed versus user-created? + +Current bias: + +- Seed only the values needed by the first v2 implementation slice. +- Keep type dictionaries extensible. +- Treat addresses, account identifiers, jurisdictions, ownership, statement + periods, balances, and document identifiers as first-class typed values rather + than JSON blobs. + +## AI And Local Model Strategy + +The platform may eventually use AI to classify documents, extract fields, +summarize evidence, and suggest relationships. AI output must not become trusted +state without provenance and review semantics. + +Open questions: + +- Should the first AI-assisted workflows require local-only models? +- Should optional remote LLM calls be allowed with explicit user approval? +- How should prompts, extracted text, and outputs be logged or redacted? +- What confidence and review thresholds are required before AI output affects + reports? + +Current bias: + +- Start with deterministic parsers and rule-based inference. +- Design interfaces that can accept AI-generated observations later. +- Treat AI output as proposed observations, not facts. + +## Tax Readiness On V2 + +TaxBuddy issue #100 is paused until the v2 document/entity/observation +foundation exists. The tax readiness feature should become a projection over +documents, tax sources, expected document types, observations, and review state. + +Open questions: + +- Which tax-source and expected-document concepts belong in the first v2 tax + slice? +- How should manual overrides be modeled without weakening provenance? +- How much jurisdiction-specific tax semantics should BankBuddy store before it + risks looking like tax advice? + +Current bias: + +- Keep tax readiness focused on document completeness and evidence. +- Avoid tax filing, tax strategy, and legal advice. +- Build annual readiness summaries only after generic documents and + observations are in place. diff --git a/docs/financial_intelligence_vision.md b/docs/financial_intelligence_vision.md index 52c99df..6b46dbe 100644 --- a/docs/financial_intelligence_vision.md +++ b/docs/financial_intelligence_vision.md @@ -258,31 +258,3 @@ while remaining: - Privacy focused - User controlled - AI assisted - ---- - -# Codex Prompt - -Read this document completely before making any code changes. - -Your task is NOT to build a separate product. - -Your task is to evolve the existing BankBuddy architecture into the platform described in this document. - -Deliverables: - -1. Current-state assessment -2. Gap analysis -3. Proposed domain model -4. Entity relationship model -5. Database schema proposal -6. Service architecture proposal -7. Package/module structure -8. CLI evolution plan -9. Future UI architecture recommendations -10. Phased implementation roadmap - -Do not begin implementation immediately. - -Produce an architecture review and implementation proposal first. -Wait for approval before making code changes.