fix: upsert scoped Norgate metadata instead of replacing whole table

[mspinola]

Bug

cotdata-update --metadata --symbols DC ZO KE EMD NKD replaced the entire metadata/contract_specs.parquet with only those 5 rows, dropping the other 42 markets' contract specs (Norgate_Symbol, tick/point value, margin, …). This broke contract-spec lookups (position sizing, cost calc) across the deploy book. Confirmed: 42 rows → 5 after that run.

Root cause

Contract specs live in one Symbol-keyed table (unlike prices/COT, which are one parquet per symbol). norgate.update_metadata(symbols=…) built a specs frame for only the requested symbols and wrote it via store.write_metadata, which atomically replaces the whole file. A subset write therefore erased every unlisted market.

Fix

• store.upsert_metadata — read existing → replace/add requested Symbols → keep the rest → write the union (manifest reflects the union count).
• norgate.update_metadata — a scoped run (symbols given) now upserts by Symbol; a no-symbols run still regenerates the full registry and replaces the table.

Other writers checked

write_prices and the COT writers all write per-symbol/per-name files, so they have no replace-vs-merge exposure — contract_specs was the only single-table-for-all-symbols writer.

Tests (55 pass)

• scoped update preserves unlisted markets (the regression)
• scoped update_metadata upserts end-to-end through a tmp store
• no-symbols update still replaces the table
• upsert on an empty store behaves like a plain write

Operational note

Immediate recovery (already done): re-run cotdata-update --metadata with no --symbols on the Norgate/Windows box → regenerates all 47. This change prevents recurrence.

🤖 Generated with Claude Code