Skip to content

PAGINATION.md

Latest commit

 

History

History
384 lines (291 loc) · 11.5 KB

PAGINATION.md

File metadata and controls

384 lines (291 loc) · 11.5 KB
Raw

API Pagination Conventions

This document describes the standardized pagination conventions used across all list endpoints in the YieldVault API.

Overview

All list endpoints follow consistent pagination patterns to make API consumption predictable and easy to use.

For runnable consumer examples, see:

Deterministic Paging Behavior

YieldVault list endpoints use stable sort keys and opaque cursors so API consumers can page through results without duplicates or gaps when the underlying dataset is unchanged.

Guarantees

Behavior What it means for consumers
Stable ordering Repeating the same query (same filters, limit, sortBy, sortOrder, and no cursor) returns the same item order.
Cursor advancement nextCursor always refers to the last item on the current page. The next request starts immediately after that anchor.
No page overlap IDs returned on page N never appear again on page N+1 when you only advance with nextCursor.
Opaque cursors Treat nextCursor as an opaque token. Do not parse or construct cursors manually.
Invalid cursors Malformed or unknown cursors return an empty page (data: [], hasNextPage: false) on public list routes, or 400 Bad Request on strict admin routes such as webhook delivery history.

Sort keys by endpoint

Endpoint Default sort Cursor anchor
GET /api/transactions timestamp desc Base64url-encoded transaction id
GET /api/portfolio/holdings valueUsd desc Base64url-encoded holding id
GET /api/vault/history date desc Base64url-encoded history point id
GET /admin/webhooks/deliveries createdAt desc, then id desc Base64url-encoded JSON { createdAt, id }

Worked example: three-page walkthrough

Assume seven transactions sorted by timestamp descending. Request limit=3:

Request 1 — first page

GET /api/transactions?limit=3&sortBy=timestamp&sortOrder=desc
{
  "data": [
    { "id": "tx-7", "timestamp": "2026-06-26T15:00:00.000Z" },
    { "id": "tx-6", "timestamp": "2026-06-26T14:00:00.000Z" },
    { "id": "tx-5", "timestamp": "2026-06-26T13:00:00.000Z" }
  ],
  "pagination": {
    "count": 3,
    "limit": 3,
    "total": 7,
    "nextCursor": "dHgtNQ",
    "prevCursor": null,
    "currentPage": null,
    "totalPages": null,
    "hasNextPage": true,
    "hasPrevPage": false
  },
  "timestamp": "2026-06-26T16:00:00.000Z"
}

nextCursor is the base64url encoding of the last row's id (tx-5 → dHgtNQ).

Request 2 — forward one page

GET /api/transactions?limit=3&sortBy=timestamp&sortOrder=desc&cursor=dHgtNQ
{
  "data": [
    { "id": "tx-4", "timestamp": "2026-06-26T12:00:00.000Z" },
    { "id": "tx-3", "timestamp": "2026-06-26T11:00:00.000Z" },
    { "id": "tx-2", "timestamp": "2026-06-26T10:00:00.000Z" }
  ],
  "pagination": {
    "count": 3,
    "limit": 3,
    "total": 7,
    "nextCursor": "dHgtMg",
    "prevCursor": null,
    "currentPage": null,
    "totalPages": null,
    "hasNextPage": true,
    "hasPrevPage": true
  },
  "timestamp": "2026-06-26T16:00:01.000Z"
}

Request 3 — final page

GET /api/transactions?limit=3&sortBy=timestamp&sortOrder=desc&cursor=dHgtMg
{
  "data": [
    { "id": "tx-1", "timestamp": "2026-06-26T09:00:00.000Z" }
  ],
  "pagination": {
    "count": 1,
    "limit": 3,
    "total": 7,
    "nextCursor": null,
    "prevCursor": null,
    "currentPage": null,
    "totalPages": null,
    "hasNextPage": false,
    "hasPrevPage": true
  },
  "timestamp": "2026-06-26T16:00:02.000Z"
}

Across all three pages the union of IDs is {tx-7, tx-6, tx-5, tx-4, tx-3, tx-2, tx-1} with no duplicates.

Sequence diagram

sequenceDiagram
  participant Client
  participant API as YieldVault API

  Client->>API: GET /api/transactions?limit=3
  API-->>Client: data=[tx-7,tx-6,tx-5], nextCursor=dHgtNQ

  Client->>API: GET /api/transactions?limit=3&cursor=dHgtNQ
  API-->>Client: data=[tx-4,tx-3,tx-2], nextCursor=dHgtMg

  Client->>API: GET /api/transactions?limit=3&cursor=dHgtMg
  API-->>Client: data=[tx-1], hasNextPage=false
Loading

Consumer loop (TypeScript)

let cursor: string | undefined;

while (true) {
  const url = new URL("/api/transactions", "http://localhost:3000");
  url.searchParams.set("limit", "20");
  if (cursor) url.searchParams.set("cursor", cursor);

  const page = await fetch(url).then((res) => res.json());

  for (const row of page.data) {
    processTransaction(row);
  }

  if (!page.pagination.hasNextPage || !page.pagination.nextCursor) {
    break;
  }

  cursor = page.pagination.nextCursor;
}

See the full runnable example in docs/examples/api_pagination_consumer.ts.

When data changes between requests

Cursor paging is stable for a fixed dataset, but new rows can appear while you page:

  • New rows inserted at the top (newer timestamp): already-fetched pages remain valid; you may see new rows only if you restart from the first page.
  • Do not mix page and cursor on the same traversal. Pick one strategy per export job.
  • Persist the last nextCursor if you need to resume a long export after a network failure.

Query Parameters

Standard Pagination Parameters

Parameter Type Default Description
limit number 20 Maximum number of items to return (1-100)
cursor string - Cursor for cursor-based pagination (opaque string)
page number - Page number for offset-based pagination (1-based)
sortBy string varies Field to sort by
sortOrder string 'desc' Sort direction: 'asc' or 'desc'

Endpoint-Specific Parameters

Transactions (GET /api/transactions)

Parameter Type Default Description
type string 'all' Filter by transaction type: 'deposit', 'withdrawal', or 'all'
walletAddress string - Filter by wallet address

Portfolio Holdings (GET /api/portfolio/holdings)

Parameter Type Default Description
status string 'all' Filter by status: 'active', 'pending', or 'all'
walletAddress string - Filter by wallet address

Vault History (GET /api/vault/history)

Parameter Type Default Description
from string - Start date (YYYY-MM-DD format)
to string - End date (YYYY-MM-DD format)

Response Format

All list endpoints return a standardized response structure:

{
  "data": [...],
  "pagination": {
    "count": 20,
    "total": 100,
    "nextCursor": "base64encodedcursor",
    "prevCursor": "base64encodedcursor",
    "currentPage": 1,
    "totalPages": 5,
    "hasNextPage": true,
    "hasPrevPage": false
  },
  "timestamp": "2026-03-28T18:00:00.000Z"
}

Pagination Metadata Fields

Field Type Description
count number Number of items returned in this response
total number Total number of items available (if known)
nextCursor string Cursor for the next page (if more items exist)
prevCursor string Cursor for the previous page (if applicable)
currentPage number Current page number (for offset-based pagination)
totalPages number Total number of pages (if total is known)
hasNextPage boolean Whether there are more items after this page
hasPrevPage boolean Whether there are items before this page

Pagination Strategies

Cursor-Based Pagination (Recommended)

Cursor-based pagination is recommended for most use cases as it provides stable ordering even when data changes between requests.

How it works:

  1. Make initial request without cursor parameter
  2. Use nextCursor from response for subsequent requests
  3. Continue until hasNextPage is false

Example:

# First page
GET /api/transactions?limit=20

# Next page (using cursor from previous response)
GET /api/transactions?limit=20&cursor=base64encodedcursor

Advantages:

  • Stable ordering even when new items are added
  • No duplicate or missing items when paginating
  • Efficient for large datasets

Offset-Based Pagination

Offset-based pagination is simpler but may have issues with changing data.

How it works:

  1. Use page parameter to specify page number (1-based)
  2. Use limit to specify items per page
  3. Calculate total pages from total and limit

Example:

# First page
GET /api/transactions?limit=20&page=1

# Second page
GET /api/transactions?limit=20&page=2

Advantages:

  • Simple to understand and implement
  • Easy to jump to specific pages

Disadvantages:

  • May show duplicate or missing items if data changes between requests
  • Less efficient for large datasets

Sorting

All list endpoints support sorting by multiple fields.

Example:

# Sort by timestamp descending (newest first)
GET /api/transactions?sortBy=timestamp&sortOrder=desc

# Sort by amount ascending (smallest first)
GET /api/transactions?sortBy=amount&sortOrder=asc

Filtering

Filtering is applied before pagination and sorting.

Example:

# Get only deposits
GET /api/transactions?type=deposit

# Get active holdings for a specific wallet
GET /api/portfolio/holdings?status=active&walletAddress=GABC...

Error Handling

Invalid pagination parameters are handled gracefully:

  • Invalid limit values are clamped to valid range (1-100)
  • Invalid page values default to page 1
  • Invalid sortOrder values default to 'desc'
  • Invalid cursor values return empty results

Best Practices

  1. Use cursor-based pagination for real-time data that may change frequently
  2. Use offset-based pagination for static or slowly-changing data
  3. Always check hasNextPage before requesting the next page
  4. Use reasonable limit values (20-50 is usually optimal)
  5. Cache responses when appropriate to reduce API calls
  6. Handle empty results gracefully (check count and data.length)

Examples

Get first 20 transactions

curl "http://localhost:3000/api/transactions?limit=20"

Get next page using cursor

curl "http://localhost:3000/api/transactions?limit=20&cursor=base64encodedcursor"

Get deposits only, sorted by amount

curl "http://localhost:3000/api/transactions?type=deposit&sortBy=amount&sortOrder=desc"

Get active portfolio holdings for a wallet

curl "http://localhost:3000/api/portfolio/holdings?status=active&walletAddress=GABC..."

Get vault history for a date range

curl "http://localhost:3000/api/vault/history?from=2026-01-01&to=2026-03-31&limit=100"

Rate Limiting

All list endpoints are subject to API rate limiting. See RATE_LIMITING.md for details.

Changelog

Version 1.1.0 (2026-06-26)

  • Add deterministic paging behavior walkthrough with concrete request/response examples
  • Add cursor usage sequence diagram and consumer loop patterns
  • Add runnable TypeScript and Python pagination consumer examples

Version 1.0.0 (2026-03-28)

  • Initial pagination conventions
  • Cursor-based and offset-based pagination support
  • Standardized response metadata
  • Consistent query parameter naming