Replace Monica file import with API-crawl approach#18
Merged
bashar-qassis merged 4 commits intomainfrom Apr 3, 2026
Merged
Conversation
Replace the unreliable file-based Monica import supplement approach with a direct API-crawl that paginates through GET /api/contacts and imports all embedded data in a single pass. Resolves first_met_through_contact and relationships in a second pass after all contacts exist locally. - New MonicaApi source module with 4-phase crawl architecture - New MonicaApiCrawlWorker (Oban, 30-min timeout, queue: :imports) - Import wizard updated with "Monica CRM (API)" option and connection test - Batch photo import via GET /api/photos (no per-contact requests) - Extra notes fetched only for contacts with >3 notes - 40 tests covering pagination, cross-references, rate limiting, and more
The new API-crawl approach (monica_api) fully replaces the file-based Monica import. Remove all dead code paths: - Delete monica.ex (1400-line file-based import source) - Delete ApiSupplementWorker (per-contact API calls that hit rate limits) - Delete PhotoBatchSyncWorker (per-photo download, also rate-limit-prone) - Strip Monica-specific logic from ImportSourceWorker (now vCard-only) - Remove "Monica CRM (file)" option from import wizard - Remove api_supplement_options/fetch_supplement/list_photos callbacks - Remove :api_supplement Oban queue - Delete associated tests and fixtures (-3543 lines)
bashar-qassis
force-pushed
the
feature/monica-api-import
branch
from
1aeffea to
e474853
Compare
… how-we-met editor, and expanded data types - Add address and note deduplication during import to prevent duplicate sub-records - Add auto-merge toggle for definite duplicate contacts (same name + shared email/phone) - Add "How We Met" slide-over panel with grouped sections (When/Where/Who/Story) and contact picker - Add phone number normalization (E.164 storage) with account-wide display format setting - Import 8 new data types from Monica: pets, calls, activities, gifts, debts, tasks, reminders, conversations - Add async document import worker (MonicaDocumentImportWorker) for background file downloads - Add Playwright E2E tests: 5 spec files with 33 tests covering all new features - Add shared test helpers (contacts.ts) and vCard fixtures for import dedup testing
Owner
Author
|
Verified on dev. a lot of pollution from the monica data (duplicate contacts) but that's likely caused by a bug in their carddav. contacts merge feature handles this after import manually. automatic merge only happens if its 100% certain |
bashar-qassis
added a commit
that referenced
this pull request
Apr 3, 2026
Incorporate phone_format support from PR #18 (monica-api-import) into the redesigned contact page layout. Keep redesigned UI as the canonical version, add phone_format prop to ContactFieldsComponent in both desktop and mobile views.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GET /api/contacts?limit=100&with=contactfields, importing all embedded data in a single passWhat changed
New API-crawl import (
monica_api)first_met_through_contactresolved reliably via two-pass approach (all contacts imported before any cross-references resolved)GET /api/photos?limit=100instead of per-contact requestsOld code removed
monica.ex(1,407 lines) — file-based import with v2/v4 format normalizationApiSupplementWorker— per-contact API calls that hit rate limitsPhotoBatchSyncWorker— per-photo downloads, also rate-limit-prone:api_supplementOban queueCI improvements
feat:→ minor,fix:→ patch,!→ major)v0.1.0,v0.2.0, ...) and pushes to GHCR with semver + latestdev-build.ymlworkflow: manually trigger from any branch, optional tag suffix (e.g.rc1)Test plan
mix qualitypasses (compile + format + credo --strict + sobelow + dialyzer)🤖 Generated with Claude Code