fix: offload bolt11_decode in list_transactions to worker thread#37
fix: offload bolt11_decode in list_transactions to worker thread#37BenGWeeks wants to merge 1 commit into
Conversation
Closes lnbits#35 `_on_list_transactions` decodes each payment's BOLT11 invoice with the synchronous `bolt11_decode()` inside the asyncio event loop. On wallets with non-trivial history this blocks the event loop long enough that NWC clients hit their reply timeout (~60s) and LNbits' other async work (relay keep-alives, publishes, concurrent NWC requests) stalls. Move the decode call to a worker thread via `asyncio.to_thread` so the event loop keeps running while each invoice is parsed. Sibling event-loop-starvation fixes: - lnbits/lnbits#3918 (IN_FLIGHT payment polling backoff) - lnbits/lnbits#3925 (non-blocking relay publishes in send_nostr_dm) Together these three changes restore NWC responsiveness on busy LNDRest-backed wallets. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Swap recovery service:
- Parse lockup tx hex to extract vout+amount (Boltz v2 /swap/{id} doesn't
return these fields; derive them from transaction.hex by matching the
lockup address script).
- initEccLib so bitcoinjs-lib can parse bech32m (taproot) addresses.
- Surface progress via react-native-toast-message: info toasts when
checking/claiming, success when recovered, error with full message when
recovery fails. Custom renderer lets body text wrap so long Electrum
errors (e.g. script-verify-flag-failed) are readable.
Boltz claim (work-in-progress, witness-hash mismatch debugging):
- Attempt MuSig2 key aggregation via @scure/btc-signer (MIT-licensed
alternative to AGPL boltz-core) for Taproot internal key.
- Diagnostic logging compares computed tweaked key vs actual lockup
output script. Currently doesn't match — internal-key derivation
still needs alignment with Boltz v2 (their docs say "Boltz key first"
but our sorted/claim_refund/refund_claim variants all diverge from
on-chain output key; needs boltz-core reference comparison).
LNbits container hot-fix patches (docker/patches/):
- README + apply.sh to re-apply patches after image upgrades.
- lnbits_wallets_lndrest.py — IN_FLIGHT payment polling backoff
(lnbits/lnbits#3918).
- lnbits_core_services_nostr.py — non-blocking relay publishes
(lnbits/lnbits#3925).
- lnbits_extensions_nwcprovider_tasks.py — bolt11_decode in worker
thread (lnbits/nwcprovider#37).
All three fixes target independent event-loop starvation paths that
together rendered NWC unusable on busy wallets. apply.sh requires
LNBITS_HOST to be set (no hard-coded hostname, safe for public repo).
Docs:
- TERMS.adoc: expand Taproot entry with bech32m/bc1p, add bech32m,
P2TR, swap-in/out, lockup/claim/refund tx, preimage, key-path vs
script-path, control block, tapleaf. Add @scure/btc-signer and
react-native-toast-message to dependent libraries table.
Ignore:
- .claude/ runtime state directory.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
please drop the AI slop description This is not a blocker operation, it runs in constant time. invoice_data = bolt11_decode(p.bolt11)Starting a new thread for each decode might actually degrade the overall performance. invoice_data = await asyncio.to_thread(bolt11_decode, p.bolt11)Is there a before/after performance test done? |
|
Feel free to fix how ever you guys see fit - you know your solution better than I do. The issue was that |
|
I see, the TL;DR is basically that you want to yield the event loop, not to make _on_list_transactions faster. Could you please change the PR to reflect that? Or if you prefer i can commit the fix to main branch. Thanks |
|
Hi guys, bit max'd out at the moment so will leave you to fix as you see fit. |
@riccardobl if you have free time please do 🙏🏼 |
done #46 |
Summary
Fixes #35
_on_list_transactionsintasks.pycalls the synchronousbolt11_decode()inside a Pythonforloop in an async handler. On wallets with non-trivial history this blocks the asyncio event loop long enough that:~60s) and seelist_transactionsfailOther NWC methods (
get_info,get_balance,make_invoice,lookup_invoice) are unaffected because they don't do per-record synchronous CPU work in a loop.Status
Fix
Offload each
bolt11_decodecall to a worker thread viaasyncio.to_thread, so the event loop can keep running while each invoice is parsed. This is the same pattern used in lnbits/lnbits#3925 for analogous synchronous websocket calls insend_nostr_dm.asynciois already imported; no new dependencies.Why this matters
Together with the other two fixes that are already in production testing, this closes the last of three event-loop-starvation paths that collectively rendered NWC unusable on busy LNDRest-backed wallets:
lnbits/core/services/payments.pylnbits/core/services/nostr.pynwcprovider/tasks.pyAlternatives considered
bolt11through and let the client decode. Smaller payload, no server CPU. But it changes the response schema (fieldsdescription,description_hash, and theinvoice_data.datefallback fortimestamp) and would break clients that rely on the current shape. Rejected for backward compatibility.asyncio.sleep(0)yields: works, but still runs all the CPU on the event-loop thread.asyncio.to_threadis cleaner and scales better.Related
Changes
tasks.py: wrapbolt11_decodeinasyncio.to_threadinside_on_list_transactions🤖 Generated with Claude Code