Using Pagination in the API to Efficiently Retrieve Large Datasets
When accessing large datasets through the API, it’s essential to manage data retrieval efficiently. Pagination is a technique that divides a large dataset into smaller, manageable chunks or “pages.” This approach enhances performance and prevents overwhelming both the client and server with excessive data in a single request.
Each request retrieves a maximum of 100 records. Use the offset parameter to specify the number of records to skip before starting the return.
| Query | Type | Description |
|---|---|---|
| offset | number | Number of records to skip. Each endpoint has a maximum (see Maximum Offset below). |
Example Query Params:
curl --request GET \
--url 'https://api-prod-us-west-2.solinkcloud.com/v2/reports?offset=100' \
--header 'Authorization: Bearer {ACCESS_TOKEN}' \
--header 'accept: application/json' \
--header 'x-api-key: {API_KEY}'Here offset is set to 100, so retrieval starts at the 101st record. Omit offset (or set it to 0) to start from the first record.
Pagination in Action
To retrieve the entire dataset, loop through the data by incrementing the offset by 100 until all records are fetched.
First Request: offset=0 - Retrieves records 1 to 100.
Second Request: offset=100 - Retrieves records 101 to 200.
Subsequent Requests: Keep adding 100 to the offset until a page returns fewer than 100 records.
Maximum Offset
Each list endpoint has a maximum offset. A request with a higher offset is rejected with 422 OFFSET_TOO_LARGE.
| Endpoint | Maximum offset |
|---|---|
Events (/events) | 10,000 |
Sites (/sites), Devices (/devices), Cameras (/cameras), Reports (/reports) | 100,000 |
Example error response:
{
"code": "OFFSET_TOO_LARGE",
"message": "Offset must be <= 100,000. Reduce the offset.",
"requestId": "..."
}For events, the message is Offset must be <= 10,000. Reduce the offset or narrow the time range.
Retrieving More Than 10,000 EventsThe events endpoint stops at an offset of 10,000. To retrieve more, split your queries into smaller time ranges:
- When the offset gets close to the limit (for example, 9,900 or more) or a page returns fewer than 100 items, move the time window forward.
- Set the next request's start time to the last item's timestamp plus a small increment (for example, +1 ms). Keep the same end time and reset the offset to 0.
- Repeat until you reach your overall end time.
This avoids the offset limit and covers the full range without overlaps or gaps.