Skip to content

Implement idempotency key system for all blockchain transaction endpoints#16

Open
okeolaolatun23-glitch wants to merge 3 commits into
ChainLearnOfficial:mainfrom
okeolaolatun23-glitch:Implement-idempotency-key-system-for-all-blockchain-transaction-endpoints
Open

Implement idempotency key system for all blockchain transaction endpoints#16
okeolaolatun23-glitch wants to merge 3 commits into
ChainLearnOfficial:mainfrom
okeolaolatun23-glitch:Implement-idempotency-key-system-for-all-blockchain-transaction-endpoints

Conversation

@okeolaolatun23-glitch

Copy link
Copy Markdown

Closes #7

PR Documentation

Title

Implement idempotency keys for blockchain transaction endpoints

Summary

This change adds database-backed idempotency support for the two on-chain transaction flows in the API: reward claims and credential mints. It prevents duplicate Stellar submissions when a client retries after a timeout or network failure.

What Changed

  • Added a new idempotency_keys table to persist request state, cached responses, and transaction hashes.
  • Added an idempotency helper to reserve keys, detect reused keys with different payloads, and return cached responses on retries.
  • Updated /api/rewards/claim to accept an idempotencyKey and reuse the original result on retry.
  • Updated /api/credentials/mint to accept an idempotencyKey and reuse the original result on retry.
  • Added a cleanup job to remove expired idempotency keys after 24 hours.
  • Added unit coverage for request hashing, cached responses, conflict handling, and cleanup.

Why It Matters

Stellar transactions are irreversible. Without idempotency, a timeout after a successful on-chain submission could cause the client to retry and trigger duplicate work or a failure that is hard to recover from safely. This change makes retries deterministic and safer for users.

Verification

Suggested checks:

  1. Send the same idempotencyKey and same request body twice to /api/rewards/claim or /api/credentials/mint.
  2. Confirm the second request returns the cached response instead of re-submitting the transaction.
  3. Reuse the same key with a different request body and confirm the API returns 409 Conflict.
  4. Confirm expired keys are removed by the cleanup job.
  5. Run the unit tests for the idempotency helper.

@DeFiVC DeFiVC left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough implementation! The idempotency system is well-designed. Before merging, a few issues need attention:


Blocking Issues

1. Merge Conflicts

The branch is based on old main. PR #18 (now merged) added withLock() to reward.service.ts and introduced processRewardClaim(). Your diff doesn't account for these changes. You'll need to rebase on latest main:

git fetch origin
git rebase origin/main
# resolve conflicts in src/modules/rewards/reward.service.ts

2. CI Failure

The Lint & Typecheck step fails because the branch is missing package-lock.json. Rebasing on latest main will fix this.

3. Scope Creep — course.service.ts change

This PR includes the inArray fix from PR #14 in course.service.ts. That's a separate bug fix — please remove it from this PR to keep concerns separated. PR #14 should handle its own merge.


Code Issues

4. Schema vs Service mismatch — idempotencyKey required vs optional

The Zod schemas (claimRewardSchema, mintCredentialSchema) mark idempotencyKey as required (.string().min(16).max(64)), but the service methods accept it as optional (idempotencyKey?: string). This means:

  • The API will reject requests without an idempotency key
  • But the service code supports optional usage

Either make the schema .optional() to allow keyless requests, or update the service signatures to be required. Given that idempotency should be opt-in for backwards compatibility, .optional() is the right choice.

5. deriveNftAssetCode — deterministic NFT codes could collide

The old code used crypto.randomBytes(4) for unique NFT asset codes. The new deriveNftAssetCode() hashes the seed (e.g., userId:courseId) to produce 8 hex chars. If two users complete the same course, they'll get the same NFT asset code — which could cause conflicts on-chain or in the database. Consider keeping the random component or combining both.

6. reward.service.tsclaimResultFromState closure references submissionId from outer scope

The claimResultFromState async function (line ~82 in the diff) captures submissionId from the outer claimReward method, but also takes txHash as a parameter. This creates a closure that's hard to test in isolation. Consider extracting it as a standalone method on the class.

7. idempotency.ts — polling loop can hang

The reserveIdempotencyKey function has a while(true) loop with a MAX_WAIT_MS (30s) deadline when a key is in-progress. If the original request never completes (e.g., server crashed), this will block the retry request for 30s. Consider adding a shorter timeout or a way to detect orphaned reservations.


Minor Nits

  • idempotency.ts, cleanup-idempotency.ts, and idempotency.test.ts are missing trailing newlines (minor, but some linters flag this)
  • stableStringify is custom — consider using JSON.stringify with sorted keys, or a library like fast-json-stable-stringify to avoid maintaining a custom serializer

What's Good

  • The reserveIdempotencyKey race condition handling (insert with unique constraint + retry on conflict) is solid
  • Storing txHash for resume scenarios is a smart design choice
  • The polling mechanism for in-progress keys handles concurrent requests
  • Cleanup job with graceful shutdown integration is well done
  • Test coverage for the core idempotency helper is thorough

Please rebase on latest main, remove the course.service.ts change, fix the schema/service mismatch, and resolve the NFT code collision issue. Then I'll re-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Expert] Implement idempotency key system for all blockchain transaction endpoints

2 participants