Three times faster under load: gzip, and the two settings that matter

Hello, and thank you for being here. This one starts with a developer in Seoul who did something we love: they measured us. They fired 200 getRecords() calls at once from one browser tab, at Skapi and at two other backends, and sent us the numbers. Skapi did not look good. A single call took 84 ms, but with 200 at once the median call took 2 seconds and the slowest 5% took 3.5 seconds. We dug in, found the cause, shipped a fix to every region, and the same test now runs three times faster. This article shows the before and after, explains what the fix did, and then covers the two settings you control, limit and requestBatchSize, with the values we recommend for common cases.

Before and after

The test sent 1, 10, 50, 100 and 200 getRecords() calls at the same time, three rounds each, from a laptop on office Wi-Fi in Seoul to a project in the Seoul region. Each call asked for 90 to 100 records of about 2 KB each from one table. The same developer ran it before the fix and after it, on the same network and the same project.

Median time of a getRecords request by how many were sent at once. Before the fix: 84 ms, 284 ms, 678 ms, 1.11 s, 2.01 s at 1, 10, 50, 100 and 200. After: 121 ms, 209 ms, 320 ms, 446 ms, 681 ms.0500 ms1.00 s1.50 s2.00 s11050100200requests sent at the same timemedian time per requestBefore, 1 at once: median 84 msBefore, 10 at once: median 284 msBefore, 50 at once: median 678 msBefore, 100 at once: median 1.11 sBefore, 200 at once: median 2.01 sBefore2.01 sAfter, 1 at once: median 121 msAfter, 10 at once: median 209 msAfter, 50 at once: median 320 msAfter, 100 at once: median 446 msAfter, 200 at once: median 681 msAfter681 ms

The median time per request. Before, it grew almost in a straight line with the number of requests. After, 200 at once take about as long as 50 used to.

Requests at once Before, median After, median Before, slowest 5% After, slowest 5%
1 84 ms 121 ms 99 ms 1.05 s
10 284 ms 209 ms 346 ms 1.13 s
50 678 ms 320 ms 1.00 s 1.14 s
100 1.11 s 446 ms 1.86 s 658 ms
200 2.01 s 681 ms 3.54 s 994 ms

At low counts the two runs are within network noise of each other: a single request is dominated by the trip itself, and the slowest 5% of a handful of requests is one unlucky request. The fix shows where it was meant to, under load.

200 requests at once. Median: 2.01 s before, 0.68 s after. Slowest 5%: 3.54 s before, 0.99 s after.01.00 s2.00 s3.00 s4.00 sMedian (p50)Before, Median (p50): 2.01 sBefore 2.01 sAfter, Median (p50): 681 msAfter 681 msSlowest 5% (p95)Before, Slowest 5% (p95): 3.54 sBefore 3.54 sAfter, Slowest 5% (p95): 994 msAfter 994 ms

200 requests at once. The median is three times faster, the slowest 5% three and a half times.

Where the time went

Our first question was the obvious one: was the server slow? It was not. During the bursts, the Skapi function behind getRecords() took 16 ms at the median, it scaled up to about 120 copies to meet each burst, with no errors and no throttling, and even the slowest answer left the API gateway in under 1.2 seconds. Nothing on Skapi limits how many requests a project, a plan or an IP address can make; there was no queue to wait in.

The request path: browser, CDN edge, API gateway, Skapi function. The function took 16 ms at the median and the gateway under 1.2 s for the slowest; most of the time went carrying 43 MB of uncompressed answers to the browser.Your browserCDN edgeAPI gatewaySkapi function16 ms medianto find the recordsunder 1.2 sfor the slowestmost of the 3.4 s went here200 answers of about 214 KB each, uncompressed:about 43 MB through one Wi-Fi in about 3.4 sOne getRecords request, and the answer on its way backThe server was quick. The answers were big, and they had to squeeze through the network.

The server side of a request was quick. The trip back to the browser was not.

The time was on the way back. Each answer was about 214 KB, and it travelled uncompressed: our API gateways were set to compress only answers larger than 5 MB, so practically no answer was ever compressed. 200 answers of 214 KB is about 43 MB per burst. Moving 43 MB in 3.4 seconds is about 100 megabits a second, which is what a good office Wi-Fi delivers. The requests were not waiting for the server; they were waiting for each other in the same pipe. It also explained two puzzles in the report: the test was twice as fast on a different Wi-Fi network, and two users on the same network slowed each other down. Both are what a full pipe does.

What the fix did

We lowered the compression threshold on every Skapi API from 5 MB to 1 KB, in every region. Every answer larger than 1 KB, including every getRecords() page of more than a record or two, now leaves the server gzipped, and your browser unzips it by itself. There is nothing to change in your code and no new SDK version to install.

Record data is JSON, and JSON compresses very well. We measured it on 100 records of the same size as the test's:

200 answers of 100 records: 44.6 MB before, 7.1 MB after gzip. One answer: 223 KB before, 35 KB after.0 MB10 MB20 MB30 MB40 MB50 MBBeforeBefore: 44.6 MB for 200 answers (223 KB each)44.6 MB223 KB eachAfter, gzipAfter, gzip: 7.1 MB for 200 answers (35 KB each)7.1 MB35 KB eachBytes that cross the network for 200 answers of 100 records (2.2 KB each)

The same 200 answers, before and after gzip. One answer went from 223 KB to 35 KB, six times smaller.

Six times fewer bytes is why 200 requests at once now take about as long as 50 did before. Compressing an answer takes the server a moment and saves far more than that on the network, on every connection, and the most on slow ones: a phone on mobile data gets the same six-fold saving.

The two settings that matter

Gzip made every answer smaller. Two settings in your own code decide how many bytes you ask for and how they are sent.

limit: ask for what you show

getRecords() returns 50 records per call by default, and up to 1000 when you set fetchOptions.limit. The size of an answer grows with the limit, so the cheapest record is the one you did not fetch. If a page shows 20 items, ask for 20 and fetch the next page with fetchMore when the user scrolls:

let page = await skapi.getRecords(
    { table: { name: 'posts', access_group: 'public' } },
    { limit: 20 }
);
// later, when the user scrolls to the end
let next = await skapi.getRecords(
    { table: { name: 'posts', access_group: 'public' } },
    { limit: 20, fetchMore: true }
);

A smaller page also helps the database. All queries on one table read from the same place, and that place serves a fixed amount of data per second. A page of 100 records this size costs about 24 read units, and one table serves about 3,000 read units a second, so roughly 125 such pages a second from that table, across all of your users. Twenty records per page is five times as many pages.

requestBatchSize: how many calls leave together

The SDK does not send every call the moment you make it. It queues them and sends them in batches of requestBatchSize, 50 by default. A batch goes out together, and the next batch starts when every call in the current one has finished.

With the default requestBatchSize of 50, 200 calls go out in 4 waves and each wave waits for its slowest request. Raised to 200, they go out in one wave.Default 504 wavesWave 1: 50 requests; the next wave starts when the slowest of these is donewave 1: 50Wave 2: 50 requests; the next wave starts when the slowest of these is donewave 2: 50Wave 3: 50 requests; the next wave starts when the slowest of these is donewave 3: 50Wave 4: 50 requests; the next wave starts when the slowest of these is donewave 4: 50each wave waits for its slowest requestRaised to 2001 waveOne wave: all 200 requests leave togetherone wave: 200time200 getRecords calls made at once, as the SDK sends themFewer waves, less waiting. But a whole wave reaches the server at once.

With the default, 200 calls made at once go out in four waves, and each wave waits for its slowest call.

If your page makes many calls at once on purpose, raise it when you create the Skapi instance. The SDK reads it from the first new Skapi() on the page:

const skapi = new Skapi('<Project ID>', { requestBatchSize: 200 });

The developer tested exactly this after the fix, with 50 against 5000:

Slowest 5% of requests by how many were sent at once, requestBatchSize 50 against 5000. Default 50: 88 ms, 239 ms, 592 ms, 800 ms, 1.16 s. 5000: 71 ms, 191 ms, 358 ms, 604 ms, 934 ms.0400 ms800 ms1.20 s11050100200requests sent at the same timeslowest 5% (p95)Default 50, 1 at once: slowest 5% 88 msDefault 50, 10 at once: slowest 5% 239 msDefault 50, 50 at once: slowest 5% 592 msDefault 50, 100 at once: slowest 5% 800 msDefault 50, 200 at once: slowest 5% 1.16 sDefault 501.16 s5000, 1 at once: slowest 5% 71 ms5000, 10 at once: slowest 5% 191 ms5000, 50 at once: slowest 5% 358 ms5000, 100 at once: slowest 5% 604 ms5000, 200 at once: slowest 5% 934 ms5000934 ms

The slowest 5% of requests with the default batch size and with 5000, after the fix. Raising it helps once there are 50 or more calls at once.

At 50 calls or more, one wave instead of several cut the slowest 5% by 20 to 40 percent and the median by 10 to 15 percent. Below 50, there is only one wave either way, so the setting changes nothing.

Settings for common cases

What your page does limit requestBatchSize
A list or a feed What fits on the screen, 20 to 50 Default, 50
Infinite scroll 20 to 50 per page, with fetchMore: true Default, 50
A dashboard that loads 5 to 20 queries at once What each widget shows Default, 50
Exporting or syncing a whole table, one page after another 500 to 1000 Default, 50
Many calls at once on purpose, like prefetching hundreds of items Small, 20 to 100 The number of calls you make at once, for example 200
Users on phones or slow networks 20 to 30 Default, 50

A word on the test

The test fired 200 calls at once from one tab. That is an extreme setting, and a good one for finding a bottleneck, but it is not how an app normally behaves: a typical page makes a handful of calls when it loads and one more when the user does something. For almost every app, keep requestBatchSize at its default. A larger batch only helps a page that makes dozens of calls at the same moment, and it sends each of them to the server at that same moment, which is exactly the burst a smaller batch smooths out. Two more things the developer did right, if you ever measure Skapi yourself: they gave each call a slightly different limit, because the SDK merges identical calls that are in flight into one request; and they repeated each round three times, because a single burst on Wi-Fi is mostly noise.

Thank you

To the developer who measured us and sent the numbers: thank you. A report like yours, with the setup, the numbers and the questions, is the fastest way for us to make Skapi better for everyone, and the fix it led to now speeds up every Skapi API in every region. If you measure something and it looks wrong, tell us. We love to support anyone who builds with passion, and we are always happy to dig in with you.

← Back to all articles