diff --git a/.gitignore b/.gitignore index e871be6..2e45f92 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,4 @@ dist/ *.js.map .env .env.local +pnpm* diff --git a/CLAUDE.md b/CLAUDE.md index 91daf6e..41f2aa8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,6 +2,18 @@ This is the Field Theory CLI — a standalone tool for syncing and querying X/Twitter bookmarks locally. +## Running locally + +**Important:** If you have this installed globally (`npm install -g fieldtheory`), make sure to run the local version for development: + +```bash +pnpm start +# or +npm run start -- +``` + +The global `ft` command may be a different version than the local codebase. + ## Commands ```bash diff --git a/README.md b/README.md index e00827f..8683c52 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,18 @@ Sync and store locally all of your X/Twitter bookmarks. Search, classify, and ma Free and open source. Designed for Mac. +## Running locally + +**Important:** If you have this installed globally (`npm install -g fieldtheory`), make sure to run the local version for development: + +```bash +pnpm start +# or +npm run start -- +``` + +The global `ft` command may be a different version than the local codebase. + ## Install ```bash @@ -31,66 +43,174 @@ On first run, `ft sync` extracts your X session from Chrome and downloads your b ## Commands -### Sync - -| Command | Description | -|---------|-------------| -| `ft sync` | Download and sync bookmarks (no API required) | -| `ft sync --full` | Full history crawl (not just incremental) | -| `ft sync --gaps` | Backfill missing quoted tweets and expand truncated articles | -| `ft sync --classify` | Sync then classify new bookmarks with LLM | -| `ft sync --api` | Sync via OAuth API (cross-platform) | -| `ft auth` | Set up OAuth for API-based sync (optional) | - -### Search and browse - -| Command | Description | -|---------|-------------| -| `ft search ` | Full-text search with BM25 ranking | -| `ft list` | Filter by author, date, category, domain | -| `ft show ` | Show one bookmark in detail | -| `ft sample ` | Random sample from a category | -| `ft stats` | Top authors, languages, date range | -| `ft viz` | Terminal dashboard with sparklines, categories, and domains | -| `ft categories` | Show category distribution | -| `ft domains` | Subject domain distribution | - -### Classification - -| Command | Description | -|---------|-------------| -| `ft classify` | Classify by category and domain using LLM | -| `ft classify --regex` | Classify by category using simple regex | -| `ft classify-domains` | Classify by subject domain only (LLM) | -| `ft model` | View or change the default LLM engine | - -### Knowledge base - -| Command | Description | -|---------|-------------| -| `ft md` | Export bookmarks as individual markdown files | -| `ft wiki` | Compile a Karpathy-style interlinked knowledge base | -| `ft ask ` | Ask questions against the knowledge base | -| `ft ask --save` | Ask and save the answer as a concept page | -| `ft lint` | Health-check the wiki for broken links and missing pages | -| `ft lint --fix` | Auto-fix fixable wiki issues | - -### Agent integration - -| Command | Description | -|---------|-------------| -| `ft skill install` | Install `/fieldtheory` skill for Claude Code and Codex | -| `ft skill show` | Print skill content to stdout | -| `ft skill uninstall` | Remove installed skill files | - -### Utilities - -| Command | Description | -|---------|-------------| -| `ft index` | Rebuild search index from JSONL cache (preserves classifications) | -| `ft fetch-media` | Download media assets (static images only) | -| `ft status` | Show sync status and data location | -| `ft path` | Print data directory path | +### sync +Download and sync bookmarks from X into your local database. + +| Flag | Description | +|------|-------------| +| (default) | Incremental sync from your last bookmark | +| `--rebuild` | Full re-crawl of all bookmarks | +| `--gaps` | Backfill missing data (quoted tweets, truncated articles) | +| `--classify` | Classify new bookmarks with LLM after syncing | +| `--api` | Use OAuth v2 API instead of Chrome session | +| `--yes` | Skip confirmation prompts | +| `--max-pages ` | Max pages to fetch (default: 500) | +| `--target-adds ` | Stop after N new bookmarks | +| `--delay-ms ` | Delay between requests in ms (default: 600) | +| `--max-minutes ` | Max runtime in minutes (default: 30) | +| `--browser ` | Browser to read session from (chrome, chromium, brave, firefox) | +| `--cookies ` | Pass ct0 and auth_token directly (skips browser extraction) | +| `--chrome-user-data-dir ` | Chrome-family user-data directory | +| `--chrome-profile-directory ` | Chrome-family profile name | +| `--firefox-profile-dir ` | Firefox profile directory | + +### search +Full-text search across bookmarks with BM25 ranking. + +| Flag | Description | +|------|-------------| +| `` | Search query (supports FTS5 syntax: AND, OR, NOT, "exact phrase") | +| `--author ` | Filter by author handle | +| `--after ` | Bookmarks posted after this date (YYYY-MM-DD) | +| `--before ` | Bookmarks posted before this date (YYYY-MM-DD) | +| `--limit ` | Max results (default: 20) | + +### list +List bookmarks with filters. + +| Flag | Description | +|------|-------------| +| `--query ` | Text query (FTS5 syntax) | +| `--author ` | Filter by author handle | +| `--after ` | Posted after (YYYY-MM-DD) | +| `--before ` | Posted before (YYYY-MM-DD) | +| `--category ` | Filter by category | +| `--domain ` | Filter by domain | +| `--limit ` | Max results (default: 30) | +| `--offset ` | Offset into results (default: 0) | +| `--json` | JSON output | + +### show +Show one bookmark in detail. + +| Flag | Description | +|------|-------------| +| `` | Bookmark ID | +| `--json` | JSON output | + +### sample +Random sample from a category or domain. + +| Flag | Description | +|------|-------------| +| `` | Category or domain to sample from | +| `--limit ` | Max results (default: 10) | + +### classify +Classify bookmarks by category and domain using LLM. + +| Flag | Description | +|------|-------------| +| (default) | Classify categories and domains with LLM | +| `--regex` | Use simple regex classification instead of LLM | +| `--fail-fast` | Stop immediately on first classification failure | + +### classify-domains +Classify bookmarks by subject domain only (LLM). + +| Flag | Description | +|------|-------------| +| (default) | Classify only missing domains | +| `--all` | Re-classify all bookmarks, not just missing | +| `--fail-fast` | Stop immediately on first classification failure | + +### md +Export bookmarks as individual markdown files. + +| Flag | Description | +|------|-------------| +| `--force` | Re-export all bookmarks (overwrite existing files) | +| `--format ` | Filename format: `rev-iso` (default, e.g. 2024-01-15-id.md) or `legacy` (e.g. id-tweettext.md) | + +### wiki +Compile a Karpathy-style interlinked knowledge base. + +| Flag | Description | +|------|-------------| +| (default) | Incremental: only pages whose source bookmark count changed | +| `--full` | Recompile all pages (ignore incremental cache) | + +### ask +Ask questions against the knowledge base. + +| Flag | Description | +|------|-------------| +| `` | Question to ask | +| `--save` | Save the answer as a concept page | +| `--json` | Output JSON instead of text | + +### lint +Health-check the wiki for broken links and missing pages. + +| Flag | Description | +|------|-------------| +| (default) | Check and report issues | +| `--fix` | Auto-fix fixable issues with targeted recompile | +| `--json` | Output JSON instead of text | + +### index +Rebuild search index from JSONL cache. + +| Flag | Description | +|------|-------------| +| (default) | Preserve existing classifications | +| `--force` | Drop and rebuild from scratch (loses classifications) | + +### fetch-media +Download media assets (static images only). + +| Flag | Description | +|------|-------------| +| `--limit ` | Max bookmarks to process (default: 100) | +| `--max-bytes ` | Per-asset byte limit (default: 50MB) | + +### model +View or change the default LLM engine. + +| Flag | Description | +|------|-------------| +| (default) | Show current model and available options | +| `` | Set engine to `claude` or `codex` | + +### auth +Set up OAuth for API-based sync (optional). + +### status +Show sync status and data location. + +### path +Print data directory path. + +### categories +Show category distribution. + +### domains +Show subject domain distribution. + +### stats +Top authors, languages, date range. + +### viz +Terminal dashboard with sparklines, categories, and domains. + +### skill install +Install `/fieldtheory` skill for Claude Code and Codex. + +### skill show +Print skill content to stdout. + +### skill uninstall +Remove installed skill files. ## Agent integration diff --git a/src/bookmark-classify-llm.ts b/src/bookmark-classify-llm.ts index 82d7600..0aa394d 100644 --- a/src/bookmark-classify-llm.ts +++ b/src/bookmark-classify-llm.ts @@ -10,6 +10,7 @@ import { openDb, saveDb } from './db.js'; import { twitterBookmarksIndexPath } from './paths.js'; import type { ResolvedEngine } from './engine.js'; import { invokeEngine } from './engine.js'; +import { C, RESET } from './bookmarks-viz.js'; const BATCH_SIZE = 50; @@ -110,9 +111,9 @@ export interface LlmClassifyResult { } export async function classifyWithLlm( - options: { engine: ResolvedEngine; onBatch?: (done: number, total: number) => void }, + options: { engine: ResolvedEngine; onBatch?: (done: number, total: number) => void; failFast?: boolean }, ): Promise { - const { engine } = options; + const { engine, failFast = false } = options; const dbPath = twitterBookmarksIndexPath(); const db = await openDb(dbPath); @@ -170,7 +171,13 @@ export async function classifyWithLlm( saveDb(db, dbPath); } catch (err) { failed += batch.length; - process.stderr.write(` Batch ${batchCount} failed: ${(err as Error).message}\n`); + const errMsg = (err as Error).message; + const firstLine = errMsg.split('\n')[0]; + console.error(`\n${C.gold} Error: ${firstLine}${RESET}`); + if (failFast) { + console.error(`${C.gold} fail-fast enabled — stopping classification${RESET}`); + break; + } } } @@ -223,9 +230,9 @@ ${items}`; } export async function classifyDomainsWithLlm( - options: { engine: ResolvedEngine; all?: boolean; onBatch?: (done: number, total: number) => void }, + options: { engine: ResolvedEngine; all?: boolean; onBatch?: (done: number, total: number) => void; failFast?: boolean }, ): Promise { - const { engine } = options; + const { engine, failFast = false } = options; const dbPath = twitterBookmarksIndexPath(); const db = await openDb(dbPath); @@ -285,7 +292,13 @@ export async function classifyDomainsWithLlm( saveDb(db, dbPath); } catch (err) { failed += batch.length; - process.stderr.write(` Batch ${batchCount} failed: ${(err as Error).message}\n`); + const errMsg = (err as Error).message; + const firstLine = errMsg.split('\n')[0]; + console.error(`\n${C.gold} Error: ${firstLine}${RESET}`); + if (failFast) { + console.error(`${C.gold} fail-fast enabled — stopping classification${RESET}`); + break; + } } } diff --git a/src/bookmarks-viz.ts b/src/bookmarks-viz.ts index 78b6bb8..e326b9e 100644 --- a/src/bookmarks-viz.ts +++ b/src/bookmarks-viz.ts @@ -3,15 +3,15 @@ import { twitterBookmarksIndexPath } from './paths.js'; // ── ANSI helpers ───────────────────────────────────────────────────────────── -const ESC = '\x1b['; -const RESET = `${ESC}0m`; +export const ESC = '\x1b['; +export const RESET = `${ESC}0m`; const BOLD = `${ESC}1m`; const DIM = `${ESC}2m`; const rgb = (r: number, g: number, b: number) => `${ESC}38;2;${r};${g};${b}m`; // Palette — muted, tasteful -const C = { +export const C = { title: rgb(199, 146, 234), // soft lavender accent: rgb(130, 170, 255), // periwinkle warm: rgb(255, 180, 120), // peach @@ -93,7 +93,7 @@ function boxBottom(width: number): string { return C.dim + '╰' + '─'.repeat(width - 2) + '╯' + RESET; } function boxRow(content: string, width: number): string { - const stripped = content.replace(/\x1b\[[^m]*m/g, ''); + const stripped = content.replace(new RegExp(ESC + '\\[([^m]*m)', 'g'), ''); const pad = Math.max(0, width - 4 - stripped.length); return C.dim + '│ ' + RESET + content + ' '.repeat(pad) + C.dim + ' │' + RESET; } diff --git a/src/cli.ts b/src/cli.ts index 078493a..2aac584 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -1,5 +1,5 @@ #!/usr/bin/env node -import { Command } from 'commander'; +import { Command, Option } from 'commander'; import { syncTwitterBookmarks } from './bookmarks.js'; import { getBookmarkStatusView, formatBookmarkStatus } from './bookmarks-service.js'; import { runTwitterOAuthFlow } from './xauth.js'; @@ -25,7 +25,7 @@ import { loadPreferences, savePreferences } from './preferences.js'; import { compileMd } from './md.js'; import { askMd } from './md-ask.js'; import { lintMd, fixLintIssues } from './md-lint.js'; -import { exportBookmarks } from './md-export.js'; +import { exportBookmarks, detectFormatMigration, type FilenameFormat } from './md-export.js'; import { renderViz } from './bookmarks-viz.js'; import { listBrowserIds } from './browsers.js'; import { dataDir, ensureDataDir, isFirstRun, twitterBookmarksIndexPath, twitterBackfillStatePath, mdDir } from './paths.js'; @@ -767,12 +767,13 @@ export function buildCli() { .command('classify') .description('Classify bookmarks by category and domain using LLM (requires claude or codex CLI)') .option('--regex', 'Use simple regex classification instead of LLM') + .option('--fail-fast', 'Stop immediately on first classification failure') .action(safe(async (options) => { if (!requireData()) return; if (options.regex) { process.stderr.write('Classifying bookmarks (regex)...\n'); const result = await classifyAndRebuild(); - console.log(`Indexed ${result.recordCount} bookmarks \u2192 ${result.dbPath}`); + console.log(`Indexed ${result.recordCount} bookmarks → ${result.dbPath}`); console.log(formatClassificationSummary(result.summary)); } else { const engine = await resolveEngine(); @@ -781,10 +782,11 @@ export function buildCli() { process.stderr.write('Classifying categories with LLM (batches of 50, ~2 min per batch)...\n'); const catResult = await classifyWithLlm({ engine, + failFast: options.failFast ?? false, onBatch: (done: number, total: number) => { const pct = total > 0 ? Math.round((done / total) * 100) : 0; const elapsed = Math.round((Date.now() - catStart) / 1000); - process.stderr.write(` Categories: ${done}/${total} (${pct}%) \u2502 ${elapsed}s elapsed\n`); + process.stderr.write(` Categories: ${done}/${total} (${pct}%) │ ${elapsed}s elapsed\n`); }, }); console.log(`\nEngine: ${catResult.engine}`); @@ -795,10 +797,11 @@ export function buildCli() { const domResult = await classifyDomainsWithLlm({ engine, all: false, + failFast: options.failFast ?? false, onBatch: (done: number, total: number) => { const pct = total > 0 ? Math.round((done / total) * 100) : 0; const elapsed = Math.round((Date.now() - domStart) / 1000); - process.stderr.write(` Domains: ${done}/${total} (${pct}%) \u2502 ${elapsed}s elapsed\n`); + process.stderr.write(` Domains: ${done}/${total} (${pct}%) │ ${elapsed}s elapsed\n`); }, }); console.log(`\nDomains: ${domResult.classified}/${domResult.totalUnclassified} classified`); @@ -811,6 +814,7 @@ export function buildCli() { .command('classify-domains') .description('Classify bookmarks by subject domain using LLM (ai, finance, etc.)') .option('--all', 'Re-classify all bookmarks, not just missing') + .option('--fail-fast', 'Stop immediately on first classification failure') .action(safe(async (options) => { if (!requireData()) return; const engine = await resolveEngine(); @@ -819,10 +823,11 @@ export function buildCli() { const result = await classifyDomainsWithLlm({ engine, all: options.all ?? false, + failFast: options.failFast ?? false, onBatch: (done: number, total: number) => { const pct = total > 0 ? Math.round((done / total) * 100) : 0; const elapsed = Math.round((Date.now() - start) / 1000); - process.stderr.write(` Domains: ${done}/${total} (${pct}%) \u2502 ${elapsed}s elapsed\n`); + process.stderr.write(` Domains: ${done}/${total} (${pct}%) │ ${elapsed}s elapsed\n`); }, }); console.log(`\nDomains: ${result.classified}/${result.totalUnclassified} classified`); @@ -1015,12 +1020,64 @@ export function buildCli() { .command('md') .description('Export bookmarks as individual markdown files') .option('--force', 'Re-export all bookmarks (overwrite existing files)') + .addOption(new Option('--format ', 'Filename format').choices(['rev-iso', 'legacy']).default('rev-iso')) .action(safe(async (options) => { if (!requireIndex()) return; + + // ── Format migration guard ────────────────────────────────────── + const format = options.format as FilenameFormat; + const needsMigration = await detectFormatMigration(format); + + if (needsMigration === 'mismatch') { + console.log( + '\n ⚠ Existing markdown files use the legacy filename format,\n' + + ' but you requested the new rev-iso format.\n' + + ' Without action, every bookmark will be re-exported as duplicates.\n', + ); + console.log(' Options:'); + console.log(' [b] Back up old folder + force regenerate (recommended)'); + console.log(' [f] Force regenerate in-place (overwrite existing)'); + console.log(' [c] Continue as-is (will create duplicates)'); + console.log(' [x] Cancel\n'); + + const answer = await promptText(' Choice [b/f/c/x]: ', { output: process.stdout }); + if (answer.kind !== 'answer' || answer.value === 'x') { + console.log(' Cancelled.'); + return; + } + + const choice = answer.value.trim().toLowerCase(); + + if (choice === 'b' || choice === 'f') { + if (choice === 'b') { + const src = path.join(mdDir(), 'bookmarks'); + const dst = path.join(mdDir(), `bookmarks-backup-${Date.now()}`); + fs.renameSync(src, dst); + console.log(` Backed up to: ${dst}`); + } + // Force regen — pass force: true to overwrite / start fresh + let lastLine = ''; + const spinner = createSpinner(() => lastLine); + const result = await exportBookmarks({ + force: true, + filenameFormat: format, + onProgress: (s) => { lastLine = s; spinner.update(); }, + }); + spinner.stop(); + console.log(`Exported ${result.exported}/${result.total} bookmarks`); + console.log(` ${result.elapsed}s elapsed`); + console.log(`\n Open in your markdown viewer:\n ${mdDir()}`); + return; + } + + // choice === 'c' — fall through to normal export + } + let lastLine = ''; const spinner = createSpinner(() => lastLine); const result = await exportBookmarks({ force: options.force, + filenameFormat: format, onProgress: (s) => { lastLine = s; spinner.update(); diff --git a/src/engine.ts b/src/engine.ts index f8b35db..8ca9e6f 100644 --- a/src/engine.ts +++ b/src/engine.ts @@ -5,7 +5,7 @@ * Remembers the user's choice in ~/.ft-bookmarks/.preferences. */ -import { execFileSync, execFile } from 'node:child_process'; +import { execFileSync, execFile, execSync, type ExecSyncOptions } from 'node:child_process'; import fs from 'node:fs'; import path from 'node:path'; import { loadPreferences, savePreferences } from './preferences.js'; @@ -150,14 +150,34 @@ export interface InvokeOptions { maxBuffer?: number; } +export interface InvokeError extends Error { + stderr: string; + stdout: string; +} + export function invokeEngine(engine: ResolvedEngine, prompt: string, opts: InvokeOptions = {}): string { const { bin, args } = engine.config; - return execFileSync(bin, args(prompt), { + const fullArgs = args(prompt); + + const result = execSync(`${bin} ${fullArgs.map(a => `'${a.replace(/'/g, "'\\''")}'`).join(' ')}`, { encoding: 'utf-8', timeout: opts.timeout ?? 120_000, maxBuffer: opts.maxBuffer ?? 1024 * 1024, - stdio: ['pipe', 'pipe', 'ignore'], - }).trim(); + } as ExecSyncOptions) as unknown as { stdout: string; stderr: string }; + + const stdout = result.stdout.trim(); + const stderr = result.stderr ?? ''; + + const fullOutput = stdout + '\n' + stderr; + if (fullOutput.includes('ERROR:') || fullOutput.includes('error:')) { + const errorMsg = stderr.includes('ERROR:') || stderr.includes('error:') ? stderr.trim() : stdout.trim(); + const error = new Error(errorMsg) as InvokeError; + error.stderr = stderr; + error.stdout = stdout; + throw error; + } + + return stdout; } /** @@ -171,8 +191,13 @@ export function invokeEngineAsync(engine: ResolvedEngine, prompt: string, opts: encoding: 'utf-8', timeout: opts.timeout ?? 120_000, maxBuffer: opts.maxBuffer ?? 1024 * 1024, - }, (err, stdout) => { - if (err) return reject(err); + }, (err, stdout, stderr) => { + if (err) { + const fullErr = new Error(stderr?.trim() || err.message) as InvokeError; + fullErr.stderr = stderr ?? ''; + fullErr.stdout = stdout?.trim() ?? ''; + return reject(fullErr); + } resolve(stdout.trim()); }); }); diff --git a/src/md-export.ts b/src/md-export.ts index 50897f5..d5a41cf 100644 --- a/src/md-export.ts +++ b/src/md-export.ts @@ -7,19 +7,28 @@ * full tweet text, and [[wikilinks]] to wiki category/domain/entity pages. * No LLM required — fast, deterministic, portable. * - * Output: ~/.ft-bookmarks/md/bookmarks/--.md + * Output: ~/.ft-bookmarks/md/bookmarks/__--.md */ -import fs from 'node:fs'; -import path from 'node:path'; -import { ensureDir, writeMd } from './fs.js'; -import { mdDir } from './paths.js'; -import { listBookmarks, countBookmarks, type BookmarkTimelineItem } from './bookmarks-db.js'; -import { slug } from './md.js'; +import fs from "node:fs"; +import path from "node:path"; +import { ensureDir, writeMd } from "./fs.js"; +import { mdDir } from "./paths.js"; +import { + listBookmarks, + countBookmarks, + type BookmarkTimelineItem, +} from "./bookmarks-db.js"; +import { slug } from "./md.js"; + +export type FilenameFormat = "legacy" | "rev-iso"; + +export type FormatMigrationStatus = "none" | "mismatch" | "empty"; export interface ExportOptions { force?: boolean; onProgress?: (status: string) => void; + filenameFormat?: FilenameFormat; } export interface ExportResult { @@ -29,58 +38,166 @@ export interface ExportResult { elapsed: number; } +const DATE_STR_REGEX = RegExp( + /^(\w{3})\s+(\w{3})\s+(\d{1,2})\s+\d{2}:\d{2}:\d{2}\s+[+-]\d{4}\s+(\d{4})$/, +); + function bookmarksDir(): string { - return path.join(mdDir(), 'bookmarks'); + return path.join(mdDir(), "bookmarks"); +} + +/** + * Detect whether existing exported files use a different format than + * the one requested. Returns: + * "empty" — no .md files on disk (fresh export, no conflict) + * "none" — files exist and likely match the requested format + * "mismatch"— files exist but wouldn't match (format change detected) + */ +export async function detectFormatMigration( + format: FilenameFormat, +): Promise { + let files: string[]; + try { + files = fs.readdirSync(bookmarksDir()).filter((f) => f.endsWith(".md")); + } catch { + return "empty"; + } + if (files.length === 0) return "empty"; + if (format === "legacy") return "none"; // legacy was the old default; safe + + // format is "rev-iso" — sample the first bookmark and check if its + // new-style filename already exists on disk. + const sample = await listBookmarks({ limit: 1, sort: "desc" }); + if (sample.length === 0) return "none"; + + const newName = bookmarkFilename(sample[0], "rev-iso"); + const exists = files.includes(newName); + + return exists ? "none" : "mismatch"; +} + +function parsePostedAt(dateStr: string | null | undefined): { + year: string; + month: string; + day: string; + dow: string; +} | null { + if (!dateStr) return null; + + if (dateStr.length === 10 && dateStr.includes("-")) { + const year = parseInt(dateStr.slice(0, 4)); + const month = parseInt(dateStr.slice(5, 7)) - 1; + const day = parseInt(dateStr.slice(8, 10)); + const dowNum = new Date(year, month, day).getDay(); + const dow = ["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"][dowNum]; + return { + year: dateStr.slice(0, 4), + month: dateStr.slice(5, 7), + day: dateStr.slice(8, 10), + dow, + }; + } + + const match = dateStr.match(DATE_STR_REGEX); + if (match) { + const [, dow, , day, year] = match; + const monthNames = [ + "Jan", + "Feb", + "Mar", + "Apr", + "May", + "Jun", + "Jul", + "Aug", + "Sep", + "Oct", + "Nov", + "Dec", + ]; + const monthStr = match[2]; + const month = String(monthNames.indexOf(monthStr) + 1).padStart(2, "0"); + return { year, month, day: day.padStart(2, "0"), dow }; + } + + return null; } -function bookmarkFilename(b: BookmarkTimelineItem): string { - const date = (b.postedAt ?? b.bookmarkedAt ?? '').slice(0, 10) || 'undated'; - const author = b.authorHandle ? slug(b.authorHandle) : 'unknown'; +type Author = string | null | undefined; + +function bookmarkFilenameLegacy(b: BookmarkTimelineItem, a: Author): string { + if (!a) a = "unknown"; + const date = (b.postedAt ?? b.bookmarkedAt ?? "").slice(0, 10) || "undated"; + const author = a; const textSlug = slug(b.text.slice(0, 50)) || b.id; return `${date}-${author}-${textSlug}.md`; } +function bookmarkFilename( + b: BookmarkTimelineItem, + format: FilenameFormat = "rev-iso", +): string { + const author = b.authorHandle ? slug(b.authorHandle) : "unknown"; + if (format === "legacy") { + return bookmarkFilenameLegacy(b, author); + } + + const dateInfo = parsePostedAt(b.postedAt ?? b.bookmarkedAt ?? null); + const textSlug = slug(b.text.slice(0, 50)) || b.id; + + if (dateInfo) { + return `${dateInfo.year}_${dateInfo.month}_${dateInfo.day}-${dateInfo.dow}-${author}-${textSlug}.md`; + } + + return `undated-${author}-${textSlug}.md`; +} + function buildBookmarkMd(b: BookmarkTimelineItem): string { const lines: string[] = []; // ── Frontmatter ───────────────────────────────────────────────────── - lines.push('---'); + lines.push("---"); if (b.authorHandle) lines.push(`author: "@${b.authorHandle}"`); - if (b.authorName) lines.push(`author_name: "${b.authorName.replace(/\\/g, '\\\\').replace(/"/g, '\\"').replace(/\n/g, ' ')}"`); + if (b.authorName) + lines.push( + `author_name: "${b.authorName.replace(/\\/g, "\\\\").replace(/"/g, '\\"').replace(/\n/g, " ")}"`, + ); if (b.postedAt) lines.push(`posted_at: ${b.postedAt.slice(0, 10)}`); - if (b.bookmarkedAt) lines.push(`bookmarked_at: ${b.bookmarkedAt.slice(0, 10)}`); + if (b.bookmarkedAt) + lines.push(`bookmarked_at: ${b.bookmarkedAt.slice(0, 10)}`); if (b.primaryCategory) lines.push(`category: ${b.primaryCategory}`); if (b.primaryDomain) lines.push(`domain: ${b.primaryDomain}`); - if (b.categories.length > 0) lines.push(`categories: [${b.categories.join(', ')}]`); - if (b.domains.length > 0) lines.push(`domains: [${b.domains.join(', ')}]`); + if (b.categories.length > 0) + lines.push(`categories: [${b.categories.join(", ")}]`); + if (b.domains.length > 0) lines.push(`domains: [${b.domains.join(", ")}]`); lines.push(`source_url: ${b.url}`); lines.push(`tweet_id: "${b.tweetId}"`); if (b.likeCount) lines.push(`likes: ${b.likeCount}`); if (b.repostCount) lines.push(`reposts: ${b.repostCount}`); if (b.viewCount) lines.push(`views: ${b.viewCount}`); - lines.push('---'); - lines.push(''); + lines.push("---"); + lines.push(""); // ── Title ─────────────────────────────────────────────────────────── - const author = b.authorHandle ? `@${b.authorHandle}` : 'Unknown'; + const author = b.authorHandle ? `@${b.authorHandle}` : "Unknown"; lines.push(`# ${author}`); - lines.push(''); + lines.push(""); // ── Body ──────────────────────────────────────────────────────────── lines.push(b.text); - lines.push(''); + lines.push(""); // ── Links ─────────────────────────────────────────────────────────── if (b.links.length > 0) { - lines.push('## Links'); + lines.push("## Links"); for (const link of b.links) lines.push(`- ${link}`); - lines.push(''); + lines.push(""); } if (b.githubUrls.length > 0) { - lines.push('## GitHub'); + lines.push("## GitHub"); for (const url of b.githubUrls) lines.push(`- ${url}`); - lines.push(''); + lines.push(""); } // ── Wikilinks to wiki pages ───────────────────────────────────────── @@ -90,20 +207,23 @@ function buildBookmarkMd(b: BookmarkTimelineItem): string { if (b.authorHandle) refs.push(`[[entities/${slug(b.authorHandle)}]]`); if (refs.length > 0) { - lines.push('## Related'); + lines.push("## Related"); for (const ref of refs) lines.push(`- ${ref}`); - lines.push(''); + lines.push(""); } // ── Source ────────────────────────────────────────────────────────── lines.push(`[Original tweet](${b.url})`); - lines.push(''); + lines.push(""); - return lines.join('\n'); + return lines.join("\n"); } -export async function exportBookmarks(options: ExportOptions = {}): Promise { - const progress = options.onProgress ?? ((s: string) => fs.writeSync(2, s + '\n')); +export async function exportBookmarks( + options: ExportOptions = {}, +): Promise { + const progress = + options.onProgress ?? ((s: string) => fs.writeSync(2, s + "\n")); const startTime = Date.now(); await ensureDir(bookmarksDir()); @@ -117,9 +237,11 @@ export async function exportBookmarks(options: ExportOptions = {}): Promise