Development

A Link Header and Cursor Contract Your SDK Can Generate

Antonio Evans9 min read

A technical diagram on a white background: on the left, a cursor shown as a short base64url string broken into labeled segments (sort key, filter hash, direction, expiry, signature).
Key points

Key points

  1. 1A cursor should carry the sort key values of the last row, plus a filter hash, sort order, and expiry, never an offset.
  2. 2Sign the encoded cursor with HMAC so a client cannot change the sort key or page size and skip or replay records.
  3. 3Emit RFC 8288 Link headers with rel next and prev so clients follow URLs instead of building them.
  4. 4Write the cursor and header contract once, then generate typed paging helpers and shared test vectors for every language you ship.

The problem with an offset in disguise

Offset pagination is easy to reason about and easy to get wrong. The server turns page=5000&limit=20 into LIMIT 20 OFFSET 99980, and the database reads and discards every row before the ones it returns. One benchmark on a ten million row PostgreSQL table puts page 1 near 8 ms and page 10,000 near 8,200 ms, a roughly thousandfold slowdown for the same page size the pagination guide from System Design Nuggets. The first pages look fine in development, and the failure only appears when a real dataset grows.

Offset also breaks under churn. If a row is inserted or deleted between two requests, the second page can repeat or skip an item, because the offset counts positions rather than values the Contentful engineering post on cursor pagination. The fix is keyset pagination: ask for rows after a known sort key value instead of after a count. A row being deleted does not break the traversal, because the query compares values, not positions Mohammad Faani's notes on cursor pagination.

The catch is that a keyset cursor is only as safe as its contents. If the cursor is a plain base64 blob of the sort keys, any client can decode it, change the timestamp or the page size, and re-encode it. That is where a signature and a written contract come in.

A cursor payload that survives contact with real data

Define the cursor as a signed, opaque token with a fixed schema. Five fields cover almost every collection endpoint:

FieldTypePurposeExample
vintegerContract version, so old cursors can be rejected after a breaking change1
karraySort key values of the last row, in sort order, including a unique tie-breaker["2025-03-01T10:00:00Z", 48213]
fstringShort hash of the normalized filter and sort parametersa91c3e
dstringDirection, next or prevnext
eintegerExpiry as a Unix timestamp in seconds1767225600

The tie-breaker matters. A timestamp alone is not unique, and without a second column in the ORDER BY the database has no deterministic order for rows that share a timestamp, so items can be missed or duplicated between pages the keyset guide on search pagination. Store both the sort column and the primary key in k, and use a tuple comparison in SQL, or a search_after-style value where the backend supports it.

The filter hash binds the cursor to the query that produced it. If a client changes status=open to status=closed and replays an old cursor, the hash will not match and the server rejects the request with a 400 rather than returning a plausible but wrong page. The expiry caps how long a stale cursor can circulate: fifteen minutes suits most UI scrolls, while exported batch jobs can ask for a longer server-side window that the issuer controls.

A worked example makes the cost clear. Suppose a collection has 50 million rows, you return 50 per page, and a client scrolls to page 5,000. With offset, the query reads about 250,000 rows to return 50. With keyset, the database seeks directly to the cursor value through the index and reads 50 rows plus index overhead. The page cost stops growing with depth. The design trade-off is that clients lose random access and total counts, which is exactly what Contentful removed when it moved its content APIs to cursor pagination Contentful's pagination post.

Sign it so clients cannot forge offsets

Sign the encoded payload with HMAC-SHA256 and keep the key on the server. The token on the wire is base64url(payload) + "." + base64url(mac). Sign the base64url payload bytes themselves, not the JSON, so the exact string a client sends back is the exact string you verify. Use a constant-time comparison; a naive == comparison on a MAC is a timing side channel.

import base64, hmac, hashlib, json, time

def b64e(raw: bytes) -> str:
    return base64.urlsafe_b64encode(raw).rstrip(b"=").decode()

def b64d(text: str) -> bytes:
    pad = "=" * (-len(text) % 4)
    return base64.urlsafe_b64decode(text + pad)

def encode_cursor(keys, filter_hash, direction, ttl, key_id, secret):
    payload = {"v": 1, "k": keys, "f": filter_hash, "d": direction,
               "e": int(time.time()) + ttl, "kid": key_id}
    body = json.dumps(payload, separators=(",", ":")).encode()
    token = b64e(body)
    mac = hmac.new(secret, token.encode(), hashlib.sha256).digest()
    return f"{token}.{b64e(mac)}"

def decode_cursor(token, secrets_by_id):
    token_b64, mac_b64 = token.split(".")
    payload = json.loads(b64d(token_b64))
    secret = secrets_by_id.get(payload["kid"])
    if secret is None:
        raise ValueError("unknown key id")
    expected = hmac.new(secret, token_b64.encode(), hashlib.sha256).digest()
    if not hmac.compare_digest(expected, b64d(mac_b64)):
        raise ValueError("bad signature")
    if payload["e"] < time.time():
        raise ValueError("cursor expired")
    return payload

Two decisions are worth writing down. First, include the key id in the signed payload so rotation can run without a flag day: old cursors verify with the old key, new cursors are issued with the new one. Second, reject a filter hash mismatch before you touch the database. This makes cursors idempotent under retries and safe to treat as bears of state.

A forged cursor in the offset world lets a client request an arbitrary page number. A forged cursor here lets a client request arbitrary sort key values, which can be worse: it can skip ahead to records the user should not see if the sort key aligns with a cheaper query path, or it can pass a filter hash that no longer matches the row-level policy. That is why cursors are opaque. Clients must not parse, decode, or rebuild them, and your SDK should not encourage it.

For cursor errors, return a standard problem detail body rather than an ad hoc error shape. RFC 9457 defines a JSON problem detail with an application/problem+json media type that machines can parse RFC 9457, and the Swagger guide explains why inventing a custom error format costs integration time and money the Swagger problem details guide. A cursor that fails verification should come back with a stable type URI such as https://api.example.com/problems/invalid-cursor, so every SDK can map it to the same retry decision.

The response should publish navigation, not hide it. RFC 8288 defines the Link header and its relation types RFC 8288, and the practical guide to Link pagination explains that clients read the URL labeled next and request it rather than constructing a new URL Vincent Schmalbach on Link header pagination. GitHub documents prev and next; Shopify uses previous and next and requires its page_info value to be reused exactly, so the client must not assume a relation name or parameter shape Schmalbach's Link header guide.

A page response then looks like this:

HTTP/1.1 200 OK
Content-Type: application/json
Link: <https://api.example.com/v1/tickets?cursor=eyJ2IjoxLC...&limit=50>; rel="next",
      <https://api.example.com/v1/tickets?cursor=eyJ2IjoxLC...&limit=50&dir=prev>; rel="prev"

{"items":[...],"has_more":true}

Three rules keep it honest. Omit the next link entirely on the last page, because the absence of the relation is the documented end signal. Do not emit prev when the current request has no cursor. And never express page size or filters in the link if they were not part of the signed contract: the link should be the exact URL the client can replay, with the cursor as the only state that changes.

The Link header also lets the API describe itself to search and other clients. Google's own guidance confirms that rel="next" and rel="prev" are supported in the HTTP header as well as in the document head the Stack Overflow answer quoting Google. That is a free win for crawlable listings.

One spec, generated clients in four languages

The real leverage is to write the contract once and generate everything else. A minimal spec needs four parts: the cursor payload schema, the signing algorithm and test vectors, the Link header format, and the endpoint shape. Once those are in a machine-readable file, each language helper is mechanical and reviewable.

Spec elementWhat it definesWhat the generators produce
Cursor schemaField names, types, order, versionFrozen dataclass, TypedDict, struct, or record
Signing rulesHMAC algorithm, key id, expirySign, verify, and rotate functions per language
Test vectorsInput payload, key, expected tokenUnit tests that must pass in all languages
Link formatRelation names, URL shapeResponse parser that returns next and prev URLs
Page envelopeitems, has_more, errorsTyped page result and error mapping

The test vectors are what keep languages from drifting. Generate a handful of known payloads with a known key and record the exact tokens. Every SDK runs the same vectors in its test suite; the day a Python helper and a Go helper disagree about JSON key order inside the payload, the vectors fail before a customer sees a mismatched cursor. Since the signed bytes are the base64url payload, the payload must be serialized canonically: fixed field order, no whitespace, no integer formatting differences. The test vectors catch all of it.

The generated helpers should expose a small surface: a Page object with items, next, prev, and a client method that follows next until it is absent. Keep the raw cursor out of the public type. If a user needs to resume a traversal later, give them an opaque checkpoint string and a resume(checkpoint) call.

For the TypeScript helper, a paginate function that takes a URL and a fetch-like function is the whole contract:

type Page<T> = { items: T[]; next: string | null; prev: string | null };

async function* paginate<T>(url: string, get: (u: string) => Promise<{ body: { items: T[] }; links: { next?: string } }>) {
  let current: string | undefined = url;
  while (current) {
    const res = await get(current);
    yield* res.body.items;
    current = res.links.next;
  }
}

Because the helper only follows links, a server change that adds a page-size cap or a new parameter does not require an SDK release. That is the payoff of publishing navigation over rebuilding URLs.

If you are generating clients while also calling models for classification, embedding, or extraction inside the same service, keeping the credential and retry logic in one place helps. Quanta Inference offers text, image, video, audio, and embedding models through one OpenAI-compatible API key Quanta Inference, which is a separate contract from the paging one but the same principle: one interface, many models.

Versioning, rotation, and a deprecation path

Cursors are a wire format, and wire formats change. Version them. API versioning exists precisely to let a producer make breaking changes without breaking existing clients the Postman guide to API versioning. The Speakeasy guide draws the practical line: additive changes like new response fields are usually safe, while renaming or removing fields is a breaking change the Speakeasy versioning guide. The v field in the cursor is your escape hatch. Increment it when the payload schema or signing bytes change, then reject old cursors with a problem detail that tells the client to restart from page one.

Key rotation needs its own plan because cursors outlive deploys. Run two keys live at all times: a current key for signing and a previous key for verifying. Rotate on a schedule that is shorter than your maximum cursor lifetime, publish the new key id to each server instance, and drop the oldest key only after every cursor signed with it has expired. If you skip this, a rotation will reject every in-flight scroll on the platform at once.

The deprecation path has three stages. First, accept both the old and new cursor versions during a migration window and log which one arrives. Second, return a Deprecation and Sunset style signal in the problem detail for old cursors, pointing to the changelog. Third, reject old cursors only after your published window closes. Backward compatibility tests belong in CI: assert that an old-version cursor still verifies for the whole window and fails cleanly on the last day.

Then keep the contract where clients can find it. A changelog that documents each cursor version and any field additions is one of the most important reference documents an API consumer has the xMatters API versioning guide. If several teams own servers and SDKs, generating the helpers, the test vectors, and the docs from one file turns coordination into a build step.

Checklist before you ship

  • The cursor contains sort keys with a unique tie-breaker, a filter hash, a direction, an expiry, and a version.
  • The signature covers the encoded payload exactly, with constant-time comparison and a key id.
  • Cursor verification failures return an RFC 9457 problem detail with a stable type URI RFC 9457.
  • Responses emit Link headers with rel="next" and rel="prev", and omit next on the last page RFC 8288.
  • Every target language passes the same signing and parsing test vectors.
  • Two signing keys are live, and rotation is tied to cursor lifetime.
  • A version field and a documented deprecation window exist before the second consumer integrates.

If the team that consumes these APIs is building with coding agents, the generated helpers pair well with skills the agent can install. Quanta Skills is a library you can drop into a coding agent or assign to an expert Quanta Skills, which is useful when the same team wants the paging client and the model calls built to one contract.

FAQ

FAQ

Why not just base64 encode the sort keys without a signature?

Because base64 is not security. Any client can decode it, edit the timestamp or page size, and send it back, and your server will honor a position the user never legitimately reached. The signature turns the cursor into a server-issued capability, which is also why clients should never parse or rebuild one.

Can I use a cursor with a server that returns a total count?

You can return the count for display, but computing it on large tables is expensive, which is one reason Contentful removed total counts in its cursor migration Contentful's pagination post. Keep the count optional and cached, and never derive paging positions from it.

What happens if the row a cursor points to is deleted?

Nothing breaks. The cursor stores sort key values, not a row identifier, so the next query compares against those values and starts at the next available record Mohammad Faani's pagination notes. This is also why you need a deterministic ORDER BY with a tie-breaker.

How long should a cursor live?

Long enough for a normal scroll and short enough to bound replay risk. Fifteen minutes covers most interactive use, while batch exports can request a longer server-controlled window. Tie your rotation schedule to the maximum lifetime so no cursor outlives its signing key.

Do I need Link headers if my JSON body includes next and prev URLs?

It depends on your clients, but the header is the interoperable choice. RFC 8288 is a standard that generic HTTP tooling and crawlers can read RFC 8288, and Google documents support for rel="next" and rel="prev" in the header the Stack Overflow answer quoting Google. If you keep both, generate them from the same source so they cannot disagree.

Sources

Sources

  1. API Pagination Guide: Cursor vs Offset vs Keyset for High ...
  2. Why Contentful is moving to cursor-based pagination
  3. Systems Design - Pagination | TIL with Mohammad
  4. Keyset + filters + sort orders without losing consistency
  5. RFC 9457: Problem Details for HTTP APIs
  6. Problem Details (RFC 9457): Doing API Errors Well - Swagger
  7. RFC 8288: Web Linking
  8. How Link-Header Pagination Works in REST APIs - Vincent Schmalbach
  9. SEO of Pagination using rel="next" rel="prev" HTTP Headers
  10. What is API versioning? Benefits, types & best practices | Postman
  11. Versioning Best Practices in REST API Design - Speakeasy
  12. API Versioning: Strategies & Best Practices - xMatters
About the author

About the author

Antonio Evans
Antonio Evans

Antonio Evans is the founder of Quanta Intelligence and a fractional CTO who builds HIPAA-ready AI systems for health and life-science companies. Before that he was technical co-founder of FriendsEAT for twenty years, CEO of Socialdraft, and a data engineer at IBM. Antonio lives with his dog Santino in Lisbon, Portugal.

All articles