> For the complete documentation index, see [llms.txt](https://developer.celigo.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.celigo.com/api/using-the-api/pagination.md). # Pagination Every list endpoint paginates. You **must** follow pagination to retrieve all records — a single response only ever returns a page. Celigo uses two pagination styles depending on the endpoint: 1. **`Link` header** (RFC 5988) — the default, used on almost every resource listing (integrations, flows, connections, exports, imports, jobs, users, and so on). 2. **`nextPageURL` in the body** — used on a handful of stats / diagnostic endpoints like flow errors. Both are transparent if you use the Celigo CLI or the JavaScript SDK; if you're calling the API directly, you need to know which style an endpoint uses. ## Link header pagination The server returns one page of records (page size set by `limit`) and, if more exist, a `Link` header pointing at the next page. The `next` URL carries an opaque `after` cursor — there is no `rel="prev"`: ```http HTTP/1.1 200 OK Link: ; rel="next" Content-Type: application/json [ ...100 records... ] ``` Iterate until `rel="next"` is missing. In Node: ```javascript import parseLinkHeader from "parse-link-header"; async function *listAll(path) { let url = `${BASE}${path}?limit=100`; while (url) { const res = await fetch(url, { headers: { Authorization: `Bearer ${TOKEN}` } }); const records = await res.json(); for (const r of records) yield r; url = parseLinkHeader(res.headers.get("link"))?.next?.url ?? null; } } for await (const integration of listAll("/v1/integrations")) { console.log(integration._id, integration.name); } ``` Or let the CLI handle pagination for you: ```bash celigo integrations list --format json # auto-paginates up to 50 pages and emits one flat JSON array ``` ### Supported query params | Param | Meaning | | ------- | ---------------------------------------------------------------------------------------------------------------------------- | | `limit` | Max records per page. The CLI defaults to 100; server caps vary by endpoint (commonly 1000). | | `after` | Opaque forward cursor. The server bakes it into the `next` link — pass the value from the `Link` header, don't construct it. | | `sort` | Endpoint-specific; not universal. See [Sort & filter](/api/using-the-api/sorting-filtering.md). | **Tip:** never hand-craft `after`/`limit`. Follow the `Link` header the server returns — that's the only contract that survives when endpoints change their cursor scheme internally. ## Body-based pagination A few endpoints return the next URL **inside** the response body because the result set is a nested structure rather than a top-level array: ```http GET /v1/flows/{flowId}/errors?pageSize=100 { "errors": [ /* 100 error records */ ], "nextPageURL": "/v1/flows/.../errors?after=abc123&pageSize=100", "totalErrors": 437 } ``` Iterate by following `nextPageURL` until it's absent or empty. In Node: ```javascript async function *listErrors(flowId) { let next = `/v1/flows/${flowId}/errors?pageSize=100`; while (next) { const res = await fetch(`${BASE}${next}`, { headers: { Authorization: `Bearer ${TOKEN}` } }); const body = await res.json(); for (const e of body.errors) yield e; next = body.nextPageURL || null; } } ``` Or with the CLI: ```bash celigo flows errors $FLOW_ID --format json ``` ## Pagination limits Clients should set a sane upper bound so a runaway loop can't overload the API. The Celigo CLI caps at **50 pages** by default and errors out if more are needed — which is enough for \~50,000 records at the default page size. If you expect more, raise the limit or filter the listing so each run returns fewer pages. ## Warnings * **Never cache the full `Link` URL across a page.** Each `next` link is tied to the specific request that produced it. Always follow the fresh header on the latest response. * **Don't parallelize a single paginated walk.** Records can be reordered between pages as underlying data changes; follow `next` sequentially. * **Pagination drifts under concurrent writes.** If records are being created or deleted while you walk pages, you may see the same record twice or miss one. For snapshot-accurate exports, restrict the listing to a closed time window where the endpoint supports one (e.g. jobs' `createdAt_gte`/`createdAt_lte`, audit's `from`/`to`). ## Related links * [Sorting & filtering](/api/using-the-api/sorting-filtering.md) * [Rate limits](/api/using-the-api/rate-limits.md) — tight pagination loops are the most common cause of self-throttling. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://developer.celigo.com/api/using-the-api/pagination.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.