yuris/spacetimedb-defold

Fork 0

feat: spread a large apply across engine frames behind frame_budget #76

Merged

yuris merged 4 commits from feat/67-frame-budget into main

2026-08-20 18:45:24 +00:00

yuris commented

2026-08-20 18:04:37 +00:00

Owner

Closes #67.

connect({ frame_budget = 0.008 }) spends at most that long per engine frame decoding rows and resumes on the next, so a large first subscription costs several ordinary frames instead of one hitch.

Opt-in, nil by default. Slicing is visible in timing — callbacks that fired inside the receive frame now land one or more frames later — and the 1.0 surface is frozen, so a timing-visible change cannot be the default. Unset, the code path is what it was: the EVENT_MESSAGE branch still calls conn:handle_frame inline and cache.slice_hook stays nil.

Shape

stdb/cache.lua gains one module-level hook called every 256 decoded rows, inside decode_rows' two loops and nowhere else. That is the only place in an apply where nothing has been written yet — the cache decodes a whole batch before touching the store so a malformed row cannot half-apply — so it is the only place a yield cannot leave a half-populated table. Store mutation and event emission stay atomic, which is what makes "callbacks only ever see a complete batch" fall out for free rather than needing bookkeeping.

stdb/init.lua gains one FIFO queue of raw frame strings and at most one live coroutine, pumped by timer.delay(0, true, ...) (once per engine frame). One coroutine at a time is what keeps ordering intact: a TransactionUpdate arriving mid-InitialSubscription is appended behind it and applied strictly after. The deadline is per pump invocation and shared across every frame drained in it, so N queued frames cannot spend N budgets in one engine frame. A dropped socket clears queue and coroutine — the cache is cleared and every subscription replayed on reconnect, so finishing a dead socket's half-decoded batch is work against state about to be discarded, and it is safe to abandon precisely because nothing was written. close() cancels the pump and clears the hook.

Two bugs the measurement caught

Neither was visible from reading the code, and the first made the feature almost useless:

1. The hook closure was being cached across pumps. decode_rows hoisted local hook = M.slice_hook at entry. Each pump installs a new closure carrying that frame's deadline, so a loop holding the first one kept asking a deadline that had expired frames ago — yielding after every single 256-row batch. Measured doing exactly that: 25000 rows took 97 frames at 1.5ms each instead of 4 at 8.5ms. Fixed by re-reading M.slice_hook at each slice point; the cost is one table lookup per 256 rows.

2. PHASE6 hung when the decode fitted in one frame. The completion gate required slice_max, which is nil when the first frame absorbs the whole decode (that frame is billed separately). Gated on store_ms instead, which always lands, so the "nothing was sliced" verdict reports it rather than the phase waiting 900s for a slice that was never coming.

What it does and does not bound

Measured at a 8ms budget, native, medians over six samples:

unsliced  BULK SUMMARY   rows=25000 runs=3 frame med=53.7ms decode=35.3 quote=0.0 build=16.9
unsliced  BULK SUMMARY   rows=25000 runs=3 frame med=57.3ms decode=37.4 quote=0.0 build=18.4

sliced    SLICED SWEEP   rows=10000 runs=3 budget=8.0ms worst decode min=8.3 med=8.4 max=8.7ms store med=4.5ms span med=21.2ms frames med=2
sliced    SLICED SWEEP   rows=25000 runs=3 budget=8.0ms worst decode min=8.4 med=8.5 max=8.5ms store med=8.5ms span med=53.2ms frames med=4

The row-decoding frames sit at the budget: 8.5ms against an unsliced 55.5ms at 25000 rows, and the whole apply costs about the same wall clock (53.2ms vs 55.5ms), so this is redistribution rather than overhead.

But two parts of an apply cannot be sliced, and after this they are what is left. Both are O(rows) and both run whole:

decode_server's single json.decode of the entire frame text, which happens before any row loop exists to yield from — 12.8–23.3ms at 25000 rows, i.e. now the worst frame in the apply.
The store mutation, atomic on purpose — 7.3–13.7ms at 25000 rows.

So the honest headline is the worst frame drops from ~55ms to ~20ms at 25000 rows, not that it drops to the budget. The issue's sketched acceptance ("no frame over budget + idle + 5ms") is not achievable and I did not make the harness pretend otherwise: PHASE6 asserts the ceiling on the row-decoding frames and reports first frame= and store frame= beside them, with the reasoning in a comment at the assertion. Widening the slack until the numbers fit would have hidden exactly the thing worth knowing.

On wasm the mechanism works and the arithmetic is worse for reasons that are the tier's: a browser frame has a ~17ms rAF floor, so an 8ms budget is already below the frame quantum, and socket delivery is inside the first frame. Measured at 10000 rows: decode frames land at 16–17ms (the floor), first frame ~75ms. Documented as such rather than quoted as a win.

Tests

New dev/test/test_slicing.lua, driven through connect with the websocket/timer globals stubbed and opts.clock hand-advanced so "out of budget" is a decision the test makes rather than a race. Covers: budget nil applies synchronously inside the socket callback; a large subscription spans several pumps and emits nothing until it completes; a mid-slice TransactionUpdate applies strictly after (asserted on the cache count at the moment its callback fires, and on it being the last event); disconnect mid-slice abandons the batch and leaves the store empty; an error surfaces out of the pump without wedging the queue behind it; close() cancels the pump and clears the hook; and frame_budget of 0, negative, non-numeric, or with no clock available all raise at connect.

Mutation-tested, since "8 passed" means nothing on its own — each of these fails a distinct assertion:

mutation	assertion that fires
hook never yields	`an apply larger than one budget spans more than one pump`
queue removed, apply synchronously	`a budgeted apply does nothing inside the socket callback`
reconnect abandonment removed	`and no later pump resurrects it`

Verification

8 passed, 0 failed; stylua --check clean; shellcheck exit 0. Native conformance passes unsliced (all six phases) and sliced at 10000 and 25000 rows; HTML5 conformance passes both ways. html5-conformance.mjs needed STDB_BULK_BUDGET added to the query-string forwarding list — without it the budget silently never reached the browser and the wasm run measured the unsliced path while claiming otherwise.

README gains a Spreading a large apply across frames subsection with the table above and both unsliceable pieces named, plus frame_budget in the connect options table and cache.slice_hook in the stdb.cache reference.

Closes #67. `connect({ frame_budget = 0.008 })` spends at most that long per engine frame decoding rows and resumes on the next, so a large first subscription costs several ordinary frames instead of one hitch. **Opt-in, nil by default.** Slicing is visible in timing — callbacks that fired inside the receive frame now land one or more frames later — and the 1.0 surface is frozen, so a timing-visible change cannot be the default. Unset, the code path is what it was: the `EVENT_MESSAGE` branch still calls `conn:handle_frame` inline and `cache.slice_hook` stays nil. ## Shape `stdb/cache.lua` gains one module-level hook called every 256 decoded rows, inside `decode_rows`' two loops and **nowhere else**. That is the only place in an apply where nothing has been written yet — the cache decodes a whole batch before touching the store so a malformed row cannot half-apply — so it is the only place a yield cannot leave a half-populated table. Store mutation and event emission stay atomic, which is what makes "callbacks only ever see a complete batch" fall out for free rather than needing bookkeeping. `stdb/init.lua` gains one FIFO queue of raw frame strings and at most one live coroutine, pumped by `timer.delay(0, true, ...)` (once per engine frame). One coroutine at a time is what keeps ordering intact: a `TransactionUpdate` arriving mid-`InitialSubscription` is appended behind it and applied strictly after. The deadline is per *pump invocation* and shared across every frame drained in it, so N queued frames cannot spend N budgets in one engine frame. A dropped socket clears queue and coroutine — the cache is cleared and every subscription replayed on reconnect, so finishing a dead socket's half-decoded batch is work against state about to be discarded, and it is safe to abandon precisely because nothing was written. `close()` cancels the pump and clears the hook. ## Two bugs the measurement caught Neither was visible from reading the code, and the first made the feature almost useless: **1. The hook closure was being cached across pumps.** `decode_rows` hoisted `local hook = M.slice_hook` at entry. Each pump installs a *new* closure carrying that frame's deadline, so a loop holding the first one kept asking a deadline that had expired frames ago — yielding after every single 256-row batch. Measured doing exactly that: 25000 rows took **97 frames at 1.5ms each** instead of 4 at 8.5ms. Fixed by re-reading `M.slice_hook` at each slice point; the cost is one table lookup per 256 rows. **2. PHASE6 hung when the decode fitted in one frame.** The completion gate required `slice_max`, which is nil when the first frame absorbs the whole decode (that frame is billed separately). Gated on `store_ms` instead, which always lands, so the "nothing was sliced" verdict reports it rather than the phase waiting 900s for a slice that was never coming. ## What it does and does not bound Measured at a 8ms budget, native, medians over six samples: ``` unsliced BULK SUMMARY rows=25000 runs=3 frame med=53.7ms decode=35.3 quote=0.0 build=16.9 unsliced BULK SUMMARY rows=25000 runs=3 frame med=57.3ms decode=37.4 quote=0.0 build=18.4 sliced SLICED SWEEP rows=10000 runs=3 budget=8.0ms worst decode min=8.3 med=8.4 max=8.7ms store med=4.5ms span med=21.2ms frames med=2 sliced SLICED SWEEP rows=25000 runs=3 budget=8.0ms worst decode min=8.4 med=8.5 max=8.5ms store med=8.5ms span med=53.2ms frames med=4 ``` The row-decoding frames sit at the budget: **8.5ms against an unsliced 55.5ms** at 25000 rows, and the whole apply costs about the same wall clock (53.2ms vs 55.5ms), so this is redistribution rather than overhead. **But two parts of an apply cannot be sliced, and after this they are what is left.** Both are O(rows) and both run whole: 1. `decode_server`'s single `json.decode` of the entire frame text, which happens before any row loop exists to yield from — 12.8–23.3ms at 25000 rows, i.e. **now the worst frame in the apply**. 2. The store mutation, atomic on purpose — 7.3–13.7ms at 25000 rows. So the honest headline is **the worst frame drops from ~55ms to ~20ms at 25000 rows**, not that it drops to the budget. The issue's sketched acceptance ("no frame over budget + idle + 5ms") is not achievable and I did not make the harness pretend otherwise: PHASE6 asserts the ceiling on the *row-decoding* frames and reports `first frame=` and `store frame=` beside them, with the reasoning in a comment at the assertion. Widening the slack until the numbers fit would have hidden exactly the thing worth knowing. On wasm the mechanism works and the arithmetic is worse for reasons that are the tier's: a browser frame has a ~17ms rAF floor, so an 8ms budget is already below the frame quantum, and socket delivery is inside the first frame. Measured at 10000 rows: decode frames land at 16–17ms (the floor), first frame ~75ms. Documented as such rather than quoted as a win. ## Tests New `dev/test/test_slicing.lua`, driven through `connect` with the `websocket`/`timer` globals stubbed and `opts.clock` hand-advanced so "out of budget" is a decision the test makes rather than a race. Covers: budget nil applies synchronously inside the socket callback; a large subscription spans several pumps and emits **nothing** until it completes; a mid-slice `TransactionUpdate` applies strictly after (asserted on the cache count at the moment its callback fires, and on it being the last event); disconnect mid-slice abandons the batch and leaves the store empty; an error surfaces out of the pump without wedging the queue behind it; `close()` cancels the pump and clears the hook; and `frame_budget` of 0, negative, non-numeric, or with no clock available all raise at `connect`. Mutation-tested, since "8 passed" means nothing on its own — each of these fails a distinct assertion: | mutation | assertion that fires | |---|---| | hook never yields | `an apply larger than one budget spans more than one pump` | | queue removed, apply synchronously | `a budgeted apply does nothing inside the socket callback` | | reconnect abandonment removed | `and no later pump resurrects it` | ## Verification `8 passed, 0 failed`; `stylua --check` clean; `shellcheck` exit 0. Native conformance passes unsliced (all six phases) and sliced at 10000 and 25000 rows; HTML5 conformance passes both ways. `html5-conformance.mjs` needed `STDB_BULK_BUDGET` added to the query-string forwarding list — without it the budget silently never reached the browser and the wasm run measured the unsliced path while claiming otherwise. README gains a `Spreading a large apply across frames` subsection with the table above and both unsliceable pieces named, plus `frame_budget` in the `connect` options table and `cache.slice_hook` in the `stdb.cache` reference.

yuris added 1 commit

2026-08-20 18:04:37 +00:00

feat: spread a large apply across engine frames behind frame_budget (#67 )

ci / ci (pull_request) Successful in 20s

Details

68a7c540e0

yuris added 1 commit

2026-08-20 18:06:25 +00:00

fix: survive close() called from inside a sliced apply (#67 )

ci / ci (pull_request) Successful in 1m28s

Details

fdeb2f988f

yuris commented

2026-08-20 18:27:52 +00:00

Author

Owner

The tear was real; `apply_database_update` is now two passes

apply_database_update interleaved decode and store per updates entry, so a yield in the second table's decode suspended with the first table's rows already written and unannounced. Reproduced before fixing, exactly as reported.

Restructured as suggested: pass one walks tables/updates doing only decode_rows and pair_by_key (both pure), accumulating { name, def, deleted, inserted, paired }; pass two walks that list doing the release/retain loops and event accumulation, with no path back into decode_rows. The store now moves from "this frame absent" to "this frame present" in one uninterruptible step, and the malformed-row guarantee strengthens from per-updates-entry to per-frame at the cost of one intermediate list.

Two tests added, and both fail against the interleaved version (verified by git checkout HEAD -- stdb/cache.lua, which reproduces your two-frame observation exactly):

FAIL a two-table frame is never observable with one table populated and the other empty: expected 0, got 2

a two-table frame is never observable with one table populated and the other empty — polls both counts every pump across a 600+600 row two-table subscription.
an abandoned two-table frame strands no rows — drops the socket mid-decode of the same frame and asserts both tables are empty, which is the permanent-tear half.

The connect fixture gained a second table (other) so the single-table blind spot cannot come back.

Four surviving mutations now caught

Each fails a distinct assertion; all four ran against the current branch:

mutation	assertion that fires
`table.remove(queue, 1)` → `table.remove(queue)`	`the frame that arrived first is applied first: expected insert:1, got insert:101`
delete the end-of-loop deadline `break`	`one pump applies one small frame, not the whole queue: expected 2, got 10`
delete `M.cache.slice_hook = nil` at the end of `pump`	`and clears the hook on the way out: expected nil, got function`
hoist `local hoisted = M.slice_hook` in `decode_rows`	`and take a bounded number of pumps, not one per slice point`

On the FIFO one you were right about why the old test could not see it: it delivers its second frame when the queue holds exactly one element, so the two table.remove forms are the same call. What it actually pinned was "an in-progress coroutine is not preempted". Both tests are kept — they pin different things.

The deadline test uses frames below SLICE_ROWS, per your note that the gap is widest there because the hook never fires at all and the end-of-loop check is the only thing that can end the drain.

The pump-bound test replaces nothing; it sits beside pumps > 1, which as you say gets more true as the hoisted-hook bug gets worse.

The harness could report PASS having sliced nothing

if self.slice_frames and self.slice_frames < 2 → if (self.slice_frames or 0) < 2. nil is exactly what a budget larger than the whole decode produces, so the nil check skipped the verdict in the one case it existed for. Confirmed fixed:

SLICED SUMMARY rows=25000 budget=50.0ms worst decode frame=0.0ms ... frames=0 ... cached=25000
FAIL 25000 row(s) applied inside a single frame with a 50.0ms budget, so nothing was sliced

Previously that run printed PASS PHASE6.

README and comments

Both bullets are now true as written rather than narrowed, since the two-pass restructure landed — the first says explicitly that a frame spanning several tables is all-or-nothing too, and points at apply_database_update for why. The abandonment bullet now says it holds because of the first bullet rather than independently.

"byte-identical to before" is gone, in the README and in cache.lua's module comment. Both now say what holds: no hook is called and no behaviour changes when the budget is unset, at a per-row modulo cost that does not show above run-to-run variance — which matches your measurement (branch medians at or below main's, within a 14–18% spread).

Verification after the changes

8 passed, 0 failed; stylua --check clean. Native conformance passes unsliced (all six phases, 25000 rows at frame med=54.1/55.0ms) and sliced at 10000 and 25000:

SLICED SWEEP rows=10000 runs=3 budget=8.0ms worst decode min=8.3 med=8.3 max=8.6ms store med=4.2ms span med=20.9ms frames med=2
SLICED SWEEP rows=25000 runs=3 budget=8.0ms worst decode min=8.4 med=8.5 max=8.5ms store med=9.2ms span med=53.3ms frames med=4

HTML5 conformance passes both ways; sliced at 10000 rows gives worst decode frame 16–17ms (the rAF floor, as documented) over 5 frames with the cache complete.

Thank you for attacking the invariant rather than the code — that finding was worth the whole review, and it would have shipped.

All seven findings actioned. The first one was correct and it was the important one — the invariant the whole design rests on was false, and the tests passed because every fixture happened to be single-table. ## The tear was real; `apply_database_update` is now two passes `apply_database_update` interleaved decode and store per `updates` entry, so a yield in the second table's decode suspended with the first table's rows already written and unannounced. Reproduced before fixing, exactly as reported. Restructured as suggested: **pass one** walks `tables`/`updates` doing only `decode_rows` and `pair_by_key` (both pure), accumulating `{ name, def, deleted, inserted, paired }`; **pass two** walks that list doing the `release`/`retain` loops and event accumulation, with no path back into `decode_rows`. The store now moves from "this frame absent" to "this frame present" in one uninterruptible step, and the malformed-row guarantee strengthens from per-`updates`-entry to per-frame at the cost of one intermediate list. Two tests added, and both fail against the interleaved version (verified by `git checkout HEAD -- stdb/cache.lua`, which reproduces your two-frame observation exactly): ``` FAIL a two-table frame is never observable with one table populated and the other empty: expected 0, got 2 ``` - `a two-table frame is never observable with one table populated and the other empty` — polls both counts every pump across a 600+600 row two-table subscription. - `an abandoned two-table frame strands no rows` — drops the socket mid-decode of the same frame and asserts both tables are empty, which is the permanent-tear half. The `connect` fixture gained a second table (`other`) so the single-table blind spot cannot come back. ## Four surviving mutations now caught Each fails a distinct assertion; all four ran against the current branch: | mutation | assertion that fires | |---|---| | `table.remove(queue, 1)` → `table.remove(queue)` | `the frame that arrived first is applied first: expected insert:1, got insert:101` | | delete the end-of-loop deadline `break` | `one pump applies one small frame, not the whole queue: expected 2, got 10` | | delete `M.cache.slice_hook = nil` at the end of `pump` | `and clears the hook on the way out: expected nil, got function` | | hoist `local hoisted = M.slice_hook` in `decode_rows` | `and take a bounded number of pumps, not one per slice point` | On the FIFO one you were right about *why* the old test could not see it: it delivers its second frame when the queue holds exactly one element, so the two `table.remove` forms are the same call. What it actually pinned was "an in-progress coroutine is not preempted". Both tests are kept — they pin different things. The deadline test uses frames *below* `SLICE_ROWS`, per your note that the gap is widest there because the hook never fires at all and the end-of-loop check is the only thing that can end the drain. The pump-bound test replaces nothing; it sits beside `pumps > 1`, which as you say gets *more* true as the hoisted-hook bug gets worse. ## The harness could report PASS having sliced nothing `if self.slice_frames and self.slice_frames < 2` → `if (self.slice_frames or 0) < 2`. nil is exactly what a budget larger than the whole decode produces, so the nil check skipped the verdict in the one case it existed for. Confirmed fixed: ``` SLICED SUMMARY rows=25000 budget=50.0ms worst decode frame=0.0ms ... frames=0 ... cached=25000 FAIL 25000 row(s) applied inside a single frame with a 50.0ms budget, so nothing was sliced ``` Previously that run printed `PASS PHASE6`. ## README and comments Both bullets are now true as written rather than narrowed, since the two-pass restructure landed — the first says explicitly that a frame spanning several tables is all-or-nothing too, and points at `apply_database_update` for why. The abandonment bullet now says it holds *because of* the first bullet rather than independently. "byte-identical to before" is gone, in the README and in `cache.lua`'s module comment. Both now say what holds: no hook is called and no behaviour changes when the budget is unset, at a per-row modulo cost that does not show above run-to-run variance — which matches your measurement (branch medians at or below `main`'s, within a 14–18% spread). ## Verification after the changes `8 passed, 0 failed`; `stylua --check` clean. Native conformance passes unsliced (all six phases, 25000 rows at `frame med=54.1/55.0ms`) and sliced at 10000 and 25000: ``` SLICED SWEEP rows=10000 runs=3 budget=8.0ms worst decode min=8.3 med=8.3 max=8.6ms store med=4.2ms span med=20.9ms frames med=2 SLICED SWEEP rows=25000 runs=3 budget=8.0ms worst decode min=8.4 med=8.5 max=8.5ms store med=9.2ms span med=53.3ms frames med=4 ``` HTML5 conformance passes both ways; sliced at 10000 rows gives worst decode frame 16–17ms (the rAF floor, as documented) over 5 frames with the cache complete. Thank you for attacking the invariant rather than the code — that finding was worth the whole review, and it would have shipped.

yuris added 1 commit

2026-08-20 18:27:52 +00:00

fix: apply a frame's decodes before any of its writes (#67 )

ci / ci (pull_request) Successful in 1m27s

Details

92cd50d6cf

yuris added 1 commit

2026-08-20 18:45:16 +00:00

test: make the two-table slicing check self-guarding (#67 )

ci / ci (pull_request) Successful in 20s

Details

fa17b49d64