# Pagination

Every list endpoint paginates. You **must** follow pagination to retrieve all records — a single response only ever returns a page.

Celigo uses two pagination styles depending on the endpoint:

1. **`Link` header** (RFC 5988) — the default, used on almost every resource listing (integrations, flows, connections, exports, imports, jobs, users, and so on).
2. **`nextPageURL` in the body** — used on a handful of stats / diagnostic endpoints like flow errors.

Both are transparent if you use the Celigo CLI or the JavaScript SDK; if you're calling the API directly, you just need to know which style an endpoint uses.

## Link header pagination

The server returns up to `N` records (default 1000) and, if more exist, a `Link` header pointing at the next page:

```http
HTTP/1.1 200 OK
Link: <https://api.integrator.io/v1/integrations?skip=100&limit=1000>; rel="next",
      <https://api.integrator.io/v1/integrations?skip=0&limit=1000>;   rel="prev"
Content-Type: application/json

[ ...1000 records... ]
```

Iterate until `rel="next"` is missing. In Node:

```javascript
import parseLinkHeader from "parse-link-header";

async function *listAll(path) {
  let url = `${BASE}${path}?limit=100`;
  while (url) {
    const res = await fetch(url, { headers: { Authorization: `Bearer ${TOKEN}` } });
    const records = await res.json();
    for (const r of records) yield r;
    url = parseLinkHeader(res.headers.get("link"))?.next?.url ?? null;
  }
}

for await (const integration of listAll("/v1/integrations")) {
  console.log(integration._id, integration.name);
}
```

Or let the CLI handle pagination for you:

```bash
celigo integrations list --format json
# auto-paginates up to 50 pages and emits one flat JSON array
```

### Supported query params

| Param   | Meaning                                                             |
| ------- | ------------------------------------------------------------------- |
| `limit` | Max records per page. Default 100. Most endpoints cap at 1000.      |
| `skip`  | Offset-based cursor. The server sets this on the `next` link.       |
| `sort`  | See [Sorting & filtering](/api/using-the-api/sorting-filtering.md). |

**Tip:** never hand-craft `skip`/`limit`. Follow the `Link` header the server returns — that's the only contract that survives when endpoints migrate to cursor-based paging internally.

## Body-based pagination

A few endpoints return the next URL **inside** the response body because the result set is a nested structure rather than a top-level array:

```http
GET /v1/flows/{flowId}/errors?pageSize=100

{
  "errors": [ /* 100 error records */ ],
  "nextPageURL": "/v1/flows/.../errors?after=abc123&pageSize=100",
  "totalErrors": 437
}
```

Iterate by following `nextPageURL` until it's absent or empty. In Node:

```javascript
async function *listErrors(flowId) {
  let next = `/v1/flows/${flowId}/errors?pageSize=100`;
  while (next) {
    const res = await fetch(`${BASE}${next}`, {
      headers: { Authorization: `Bearer ${TOKEN}` }
    });
    const body = await res.json();
    for (const e of body.errors) yield e;
    next = body.nextPageURL || null;
  }
}
```

Or with the CLI:

```bash
celigo flows errors $FLOW_ID --format json
```

## Pagination limits

Clients should set a sane upper bound so a runaway loop can't hammer the API. The Celigo CLI caps at **50 pages** by default and errors out if more are needed — which is enough for \~50,000 records at the default page size. If you expect more, raise the limit or filter the listing so each run returns fewer pages.

## Warnings

* **Never cache the full `Link` URL across a page.** Each `next` link is tied to the specific request that produced it. Always follow the fresh header on the latest response.
* **Don't parallelize a single paginated walk.** Records can be reordered between pages as underlying data changes; follow `next` sequentially.
* **`skip`-based pagination drifts.** If records are being created or deleted while you paginate, you may see the same record twice or miss one. For snapshot-accurate exports, restrict by a time window (`createdAfter`, `createdBefore`) and sort by `_id`.

## Related links

* [Sorting & filtering](/api/using-the-api/sorting-filtering.md)
* [Rate limits](/api/using-the-api/rate-limits.md) — tight pagination loops are the #1 cause of self-throttling.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://developer.celigo.com/api/using-the-api/pagination.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.