Home / Blog / Performance

Scaling to 1,000+ Files with the Git Tree API

Deep dive: How we switched to a recursive tree fetching strategy to make Pageel lightning fast for large repositories.

P
Pageel Team
June 5, 2025
8 min read
Scaling Git Tree API

In earlier versions of Pageel, we listed files by fetching the directory contents. This works fine for 50 blog posts.

The API Rate Limit

GitHub’s standard API has a rate limit of 5,000 requests per hour. For a repository with 1,000 files spread across 50 directories, fetching the full tree could consume 50+ API calls.

We needed a way to get the entire file structure in a single request.

The Recursive Tree Strategy

We switched to using the Git Database API with the recursive=1 flag:

GET /repos/{owner}/{repo}/git/trees/{tree_sha}?recursive=1

This returns the entire repository tree in a single API call, no matter how many files or directories exist.

Client-Side Caching

We cache this tree in memory and use IndexedDB for persistence. Benefits:

  • Near-zero latency navigation between folders
  • Instant search across all file names
  • Offline-capable file browsing

The result? Pageel can now handle repositories with 10,000+ files without breaking a sweat.

#Engineering #Git #API