A request that returns one hundred issues may still omit most of a large repository. Pagination makes the remaining pages discoverable, but the client has to follow that contract and explain what happens when it stops early. This lesson builds a bounded collector with deterministic fixture responses, so you can run it without credentials or changing any repository. It also describes the real request headers, status handling and rate-limit policy needed when replacing the fixture transport. A local output match validates the collector's behavior, not the availability of GitHub's live service.
Follow the response contract instead of guessing pages
Many GitHub REST endpoints paginate collection results and return Link headers describing relations such as next and last. Follow the next relation when present rather than assuming the response's array length proves completion. A short page can still have an endpoint-specific meaning, and not every endpoint exposes identical pagination parameters. Read the chosen endpoint's documentation and preserve its query parameters when following supplied links.
Choose an appropriate per_page value within that endpoint's documented limit to reduce unnecessary calls. Bigger pages still need memory and response-size limits, and a collector may not need every historical result. Define whether the task requires a complete list, a bounded recent sample or a resumable cursor. If you stop at a configured limit, report that explicitly rather than presenting a partial collection as the complete repository inventory. An honest incomplete result is better than silent truncation.
Validate links and bound traversal
The next link is still input to a network client. Validate its scheme and origin before sending authentication headers, disable or deliberately validate redirects and guard against a repeated URL. The example requires https://api.github.com and a maximum number of pages. A cyclic link fails clearly instead of looping forever. For GitHub Enterprise Server, use the intended instance origin and documented API base rather than weakening the check to any HTTPS host.
The compact parser in the example handles the standard next-link form used by its fixtures; a production generic Link-header parser should cover the grammar your transport requires. Page limits bound the number of requests, but also bound response body bytes and total records for large inputs. A timeout should cover network and body consumption, not merely obtaining headers. Make limits part of the client's contract so a caller can distinguish a successfully complete result from a resource-limit failure.
Treat statuses and rate limits as decisions
A real adapter should request the appropriate representation, set an intentional supported X-GitHub-Api-Version and provide authentication through a protected credential source when needed. The 2026-03-10 API version is documented at the time of this lesson. Check the response status before consuming a success-shaped body. A 404 may reflect access or resource visibility, and a 403 or 429 needs interpretation using documented rate-limit and retry information rather than an unconditional immediate retry.
Respect Retry-After when present and the relevant rate-limit reset information for exhausted primary limits. Secondary limits require their own backoff behavior. Bound retries and total time, and avoid running many collectors concurrently just to accelerate a single report. Do not retry writes blindly, because a timed-out request may already have changed state. This lesson's collector is read-only, but the surrounding client still needs a deliberate retry and cancellation policy for the operation it performs.
Interpret results and consistency honestly
The repository issues endpoint can include pull requests, recognizable through their pull_request field. If you want issues only, filter intentionally and say so. Do not assume every row is an issue because the endpoint's path contains issues. Choose a stable identifier for deduplication, and preserve the fields necessary for the report rather than logging entire payloads unnecessarily.
A multi-page traversal is not automatically a database snapshot. Results can change while requests are in flight, depending on endpoint sorting and concurrent activity. For an audit, record the query, collection time and completeness limits, and decide whether deduplication or a repeat pass is required. For ongoing synchronization, follow the documented conditional-request or event-driven patterns appropriate to the task. The fixture below stays constant so its output is deterministic; real repository activity needs an explicit consistency expectation.
Sources and prerequisites
Start with CLI usage, issues and pull requests. Read the official pagination, best practices, rate limits, issues endpoint and API versions references.
GitHub REST API Pagination with Bounded Clients Summary
- Follow documented next links and state whether collection is complete.
- Bound traversal, response consumption and retries separately.
- Interpret issue records, API versions and rate-limit statuses deliberately.
GitHub REST API Pagination with Bounded Clients Examples
Each example shows the code, the output it produces and what each part does.
Collect two fixture pages and exclude pull requests
Save this complete program as pagination-demo.cjs and run it with Node.js18+. fetchPage is an offline transport whose records imitate the issues endpoint. The collector follows Link relations, validates destinations, rejects cycles and reports only issue records. No GitHub request or token is used.
Record2 is a pull request and is intentionally excluded. The second page has no next link, so the traversal completes. When adapting this to a real transport, preserve the status and Link information and add byte limits, cancellation, protected authentication and a bounded rate-limit policy.
const origin = 'https://api.github.com';
const first = origin + '/repos/demo/project/issues?per_page=2&page=1';
const second = origin + '/repos/demo/project/issues?per_page=2&page=2';
const pages = new Map([
[first, { status: 200, link: '<' + second + '>; rel="next"', rows: [
{ id: 1, title: 'Bug' }, { id: 2, title: 'Patch', pull_request: {} }
] }],
[second, { status: 200, link: '', rows: [{ id: 3, title: 'Question' }] }]
]);
const fetchPage = async url => {
if (!pages.has(url)) throw new Error('Unknown fixture page');
return pages.get(url);
};
const nextLink = header => {
const match = header.match(/<([^>]+)>;\s*rel="next"/);
return match ? match[1] : null;
};
async function collect(start, request, maxPages = 10) {
const seen = new Set();
const issues = new Map();
let url = start;
while (url) {
const parsed = new URL(url);
if (parsed.origin !== origin || parsed.protocol !== 'https:') throw new Error('Unexpected origin');
if (seen.has(url)) throw new Error('Pagination cycle');
if (seen.size >= maxPages) throw new Error('Page limit reached');
seen.add(url);
const page = await request(url);
if (page.status !== 200) throw new Error('Request failed: ' + page.status);
if (!Array.isArray(page.rows)) throw new Error('Expected collection');
for (const row of page.rows) {
if (!Number.isSafeInteger(row.id)) throw new Error('Invalid record identity');
if (!row.pull_request) issues.set(row.id, row);
}
url = nextLink(page.link || '');
}
return { issues: [...issues.values()], pages: seen.size };
}
collect(first, fetchPage).then(result => {
console.log('issue IDs:', result.issues.map(row => row.id).join(','));
console.log('pages:', result.pages);
}).catch(error => { console.error(error.message); process.exitCode = 1; });issue IDs: 1,3
pages: 2GitHub REST API Pagination with Bounded Clients in Real Projects
Validate next-page links before following them
Run this separate complete Node.js exercise. It checks destination validation and repeated-link detection with fabricated URLs. The goal is to identify failure states before a credential-bearing adapter sends another request.
const expectedOrigin = 'https://api.github.com';
function validateNext(url, seen) {
const parsed = new URL(url);
if (parsed.origin !== expectedOrigin) return 'reject origin';
if (seen.has(url)) return 'reject cycle';
return 'accept';
}
const current = expectedOrigin + '/repos/demo/project/issues?page=1';
const seen = new Set([current]);
console.log(validateNext(expectedOrigin + '/repos/demo/project/issues?page=2', seen));
console.log(validateNext(current, seen));
console.log(validateNext('https://other.invalid/issues?page=2', seen));accept
reject cycle
reject originA next link is not an instruction to send a token to any destination. The full collector also has a page limit; a production adapter should apply response-size and timeout limits as separate boundaries. Preserve a clear error when a traversal cannot satisfy its completeness contract.
GitHub REST API Pagination with Bounded Clients FAQ
Can I stop when a page has fewer than per_page rows?
Follow the endpoint's documented pagination contract, particularly its next link, instead of using row count as a universal completion rule. Different endpoints can expose different pagination behavior. If you intentionally stop for a sample or resource limit, label the result incomplete or bounded. Keep the original query parameters and record the collection conditions so a reader knows whether the report claims a complete inventory.
Why does the issues endpoint return pull requests?
GitHub's repository issues collection includes pull-request records, which carry a pull_request field. Filter that field when your report needs issues only, or retain it when your task wants both kinds of discussion. Do not classify solely from the endpoint name or title text. The fixture includes one such record deliberately so the worked collector demonstrates the distinction with a deterministic expected result.
Should I immediately retry every 403 response?
No. Interpret the status and relevant headers under GitHub's documented API behavior. Rate-limit responses need bounded waiting and backoff, while access or policy problems may require a different correction. Respect Retry-After and applicable reset information, limit attempts and total duration, and avoid creating more concurrent requests during a limit event. A retry loop should explain why another request is likely to succeed rather than merely repeat indefinitely.
Is a paginated report a consistent snapshot?
Not automatically. Repository state may change between requests, and endpoint ordering affects which rows move across pages. Decide what consistency the report requires, deduplicate by stable identity where appropriate and record timing and completeness conditions. A constant offline fixture is useful for validating traversal logic but does not establish live snapshot behavior. For synchronization or audits, choose the endpoint and collection strategy that match the actual task.
GitHub REST API Pagination with Bounded Clients Exercises
Answer each one and check it instantly. If you get stuck, open the hint before the answer.
Why is record2 absent from issue IDs in the main fixture?
GitHub REST API Pagination with Bounded Clients Quiz
2 questions to check what you learned.
Which signal does this collector use to continue traversal?
- The Link header's next relation
- The last issue title
- The repository's display color
Why validate the next-link origin before an authenticated request?
- To ensure every repository has exactly two pages
- To avoid sending credentials to an unexpected destination
- To grant the client write access