Pagination and Versioning

Two problems every production API eventually faces: returning large datasets without choking the client, and evolving without breaking existing integrations. Both have well-understood solutions and common failure modes.

Analogy

Pagination and versioning are like running a busy coat check. Offset pagination is telling the attendant "skip the first 200 coats and give me the next 25" — fine until someone grabs a coat off the rack while you're waiting, now every coat behind them shifts one hook left and you either see the same coat twice or miss one entirely. Cursor pagination is instead handing the attendant the tag of the last coat you saw: "give me the 25 hanging after tag C-4291" — insertions and removals further up the rack don't shuffle your position. API versioning is the coat check running a second rack with a different hook numbering system while the old one winds down — old tickets still match the old rack, new tickets use the new rack, and the deprecation notice says "the old rack closes in six months, please redeem any remaining tickets before then".

Offset pagination

The approach taught in most tutorials:

GET /posts?page=2&per_page=25
GET /posts?offset=50&limit=25

The server skips offset rows, then returns limit rows. Simple to implement, simple to explain.

The problem: offset is computed at query time. If a row is inserted before your offset while you are paginating, all subsequent pages shift by one. You will see a row twice (if it was inserted before your current position) or miss a row entirely (if inserted after). On a busy dataset, long-running paginations produce inconsistent results.

// Page 1 read: rows 1–25
// Row 5 deleted between requests
// Page 2 read: rows 27–51 (row 26 was skipped — it's now at position 25)

Cursor pagination

Instead of a position, the client sends an opaque cursor that encodes the last seen record:

GET /posts?limit=25
→ returns rows, plus "next_cursor": "eyJpZCI6MjV9"

GET /posts?cursor=eyJpZCI6MjV9&limit=25
→ returns the next 25 rows

The cursor is typically the Base64-encoded primary key of the last record on the page. The server issues: WHERE id > :last_seen_id LIMIT 25.

Insertions and deletions before the cursor do not affect the page — the query anchors to the last-seen ID, not a count. Cursor pagination gives stable, consistent results under concurrent writes.

Trade-offs:

	Offset	Cursor
Jump to page N	Yes	No
Stable under inserts/deletes	No	Yes
Total count	Easy	Expensive
Implementation	Simple	Slightly more complex
Client cacheability	Easy	Harder

Use cursor pagination for feeds, activity streams, and any dataset that changes while clients are paginating. Use offset pagination only when you need random access by page number and the dataset is small enough that instability does not matter.

Filtering and sorting conventions

Consistent query parameter names reduce client friction:

GET /posts?status=published&author_id=42
GET /posts?sort=created_at&order=desc
GET /posts?sort=-created_at             ← minus prefix for descending (common shorthand)
GET /posts?fields=id,title,author       ← sparse fieldsets

Document which fields are filterable and which are sortable. Attempting to filter on a non-indexed field on a large table is a DoS waiting to happen.

Bulk and batch endpoints

Sometimes a client needs to create or update many resources at once. Two patterns:

Bulk create:

POST /posts/bulk
Content-Type: application/json

{ "items": [ { "title": "…" }, { "title": "…" } ] }

→ 207 Multi-Status
{ "results": [ { "status": 201, "id": 1 }, { "status": 422, "error": "…" } ] }

Batch operations process multiple resources in a single request. Use 207 Multi-Status when items can succeed or fail independently. Clients must inspect per-item status codes.

Versioning strategies

An API version defines a contract. Breaking changes require a new version.

What counts as a breaking change:

• Removing or renaming a field
• Changing a field's type
• Removing an endpoint
• Changing required parameters
• Changing error response shapes

What is not a breaking change:

• Adding new optional fields
• Adding new endpoints
• Adding new query parameters

URL path versioning

/v1/users
/v2/users

Explicit, easy to route, easy to cache, visible in logs. Most commonly used. The downside: clients must update their base URL.

Header versioning

Accept: application/vnd.example.v2+json
Api-Version: 2024-01-15

Keeps URLs clean. GitHub uses date-based header versioning. The downside: harder to debug in a browser address bar, harder to cache with a CDN.

Content negotiation

Accept: application/vnd.example+json;version=2

Theoretically purist REST, rarely used in practice.

URL versioning is the pragmatic choice for most APIs. It is obvious to users, cache-friendly, and simple to route.

Deprecation and sunsetting

Breaking a client's integration is a trust violation. Give clients time to migrate.

1. Announce deprecation: add Deprecation and Sunset headers to deprecated endpoints.
2. Set a sunset date: typically 6–12 months for public APIs, 3 months for internal.
3. Log usage: track which clients still call deprecated endpoints; reach out before sunset.
4. Hard sunset: after the date, return 410 Gone with a message pointing to the migration guide.

HTTP/1.1 200 OK
Deprecation: true
Sunset: Sat, 31 Dec 2025 23:59:59 GMT
Link: <https://docs.example.com/migration/v2>; rel="successor-version"

RFC 8594 defines the Sunset header. Most API gateways can inject it automatically once you set a deprecation policy.

Why pagination strategy matters at scale

Consider 10,000 posts. An offset of 9,975 with limit 25 requires the database to read and discard 9,975 rows before returning 25. At PostgreSQL's typical index scan speed this is fast, but at 100M rows it becomes a full table scan. Cursor-based queries anchor on the indexed key and stay O(log n) regardless of depth.

The playground demonstrates this: insert a row while offset pagination is mid-flight and watch a record disappear from the results. The cursor-based paginator is unaffected.