yuris/spacetimedb-defold
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 9 Packages Wiki Activity Actions

Frame budget: parsing a large subscription must not hitch the game #30

New issue
Closed
opened 2026-08-19 15:42:14 +00:00 by yuris · 1 comment
yuris commented 2026-08-19 15:42:14 +00:00
Owner
Copy link

Everything the SDK does with an incoming frame happens on Defold's main thread inside one update(). A large InitialSubscription is decoded start to finish in a single frame: json.decode per row, then a cache write per row, with no yield point anywhere.

Ten thousand rows on a fresh connect is one very long frame — a visible hitch at precisely the worst moment, when the player has just joined and the world is populating.

What is already threaded, and what is not

An earlier draft of this issue said extension-websocket delivers frames on the main thread. That is only half right, and the other half narrows the problem usefully. From websocket/src/websocket.cpp (4.2.3, the pinned version; master reads the same):

  • CreateConnectionWslay spawns one OS thread per connection via dmThread::New(ConnectionWorker, 0x80000, conn, "WSConnect").
  • That worker owns the socket for the connection's whole life: dial, TLS, handshake write, handshake read, then a WSL_Poll(conn->m_Ctx) loop on a 10ms sleep. Socket reads, unmasking and frame reassembly all happen off the main thread.
  • PushMessage appends each completed message to conn->m_Messages under DM_MUTEX_SCOPED_LOCK(conn->m_Mutex).
  • The extension's OnUpdate — main thread — takes the same mutex and drains the queue, calling HandleCallback → dmScript::PCall once per message.

So the producer/consumer split already exists, and everything up to "here is one complete text frame" is already parallel. What is serialized on the main thread begins exactly at our Lua callback: json.decode, row decode, cache write.

HTML5 is a separate path — emscripten_websocket_* with browser callbacks, no wslay, no worker thread.

Consequence this issue previously missed: OnUpdate drains the entire queue in one pass. A burst arrives as N back-to-back Lua callbacks inside a single frame. A budget scoped to one callback therefore does not bound the frame; the budget must be a deadline shared across every callback within a frame.

Why threading is not the fix

dmThread is available to native extensions (defold/defold#5079, closing defold/defold#4835), and as above we already benefit from it. It cannot take any more of this work:

  1. Lua is single-lua_State and not reentrant. dmScript::SetupCallback/PCall are main-thread only. Anything that builds a Lua table runs on the main thread by definition — including the per-row lua_createtable/lua_setfield traffic, which is plausibly the larger half of the cost at 10k rows. Unmeasured; see below.
  2. A real win means moving the row cache into C++, with Lua holding views into it — plus forking extension-websocket to intercept frames before they become Lua strings. That is a different project, not a change to this SDK.
  3. HTML5 would lose it anyway. Emscripten threads require pthreads, which requires SharedArrayBuffer, which requires COOP/COEP headers from whoever serves the game. Defold's HTML5 build does not arrange this, and the web target is load-bearing here (patches/extension-websocket-html5-subprotocol.patch).

For reference, the Godot SDK does move BSATN deserialization to a worker thread — packet queue in, parsed results out, applied on the main thread in _process — and its own docs put the cost at "1-2 frames delay". Note that even it auto-disables on non-threaded builds and keeps a single-threaded path.

Approach

Time-slicing, not parallelism. Spread the application of one large update across frames:

  • A budget per frame — rows, or a socket.gettime deadline — after which the remainder defers to the next frame. The deadline is per frame, shared across every callback OnUpdate delivers in that frame, not per callback.
  • The natural Lua mechanism is a coroutine per in-flight batch, resumed each frame until it finishes.
  • Ordering must hold absolutely. A partially applied InitialSubscription followed by a TransactionUpdate touching the same rows, applied out of order, corrupts the cache. Later frames must queue behind an in-progress batch rather than interleave with it.
  • Callbacks (#21, per-table API) fire only when their batch completes, or a consumer observes a half-populated table and draws half a world.

Measure before choosing a number. The honest first step is finding where the cliff actually is:

  • Add a conformance scenario subscribing to a table with 10k+ rows, reporting the wall time of the applying frame
  • Split that measurement between json.decode and cache/table construction — it decides whether a C++ decoder could ever have helped, and it is cheap to collect while we are already measuring
  • Decide from that measurement whether time-slicing is needed at all, and at what row count it starts to matter
  • Only then implement, with the budget configurable and defaulting to whatever the measurement justifies

Acceptance

A measurement first, posted here, before any implementation. If a 10k-row subscription applies in 3ms, this closes as unnecessary and says so — a perfectly good outcome, and far cheaper than a speculative scheduler.

If it is slow: a test proving a large batch spans multiple frames, that the resulting cache is identical to the single-frame result, and that a transaction arriving mid-batch is applied after it rather than interleaved.

Priority

Unknown until measured, which is the point. Worth resolving before 1.0 because the fix, if needed, changes when callbacks fire — exactly the kind of observable timing an API freeze should not have to break later.

Everything the SDK does with an incoming frame happens on Defold's main thread inside one `update()`. A large `InitialSubscription` is decoded start to finish in a single frame: `json.decode` per row, then a cache write per row, with no yield point anywhere. Ten thousand rows on a fresh connect is one very long frame — a visible hitch at precisely the worst moment, when the player has just joined and the world is populating. ## What is already threaded, and what is not An earlier draft of this issue said `extension-websocket` delivers frames on the main thread. That is only half right, and the other half narrows the problem usefully. From `websocket/src/websocket.cpp` (4.2.3, the pinned version; `master` reads the same): - `CreateConnectionWslay` spawns one OS thread per connection via `dmThread::New(ConnectionWorker, 0x80000, conn, "WSConnect")`. - That worker owns the socket for the connection's whole life: dial, TLS, handshake write, handshake read, then a `WSL_Poll(conn->m_Ctx)` loop on a 10ms sleep. Socket reads, unmasking and frame reassembly all happen off the main thread. - `PushMessage` appends each completed message to `conn->m_Messages` under `DM_MUTEX_SCOPED_LOCK(conn->m_Mutex)`. - The extension's `OnUpdate` — main thread — takes the same mutex and drains the queue, calling `HandleCallback` → `dmScript::PCall` once per message. So the producer/consumer split already exists, and everything up to "here is one complete text frame" is already parallel. What is serialized on the main thread begins exactly at our Lua callback: `json.decode`, row decode, cache write. HTML5 is a separate path — `emscripten_websocket_*` with browser callbacks, no wslay, no worker thread. **Consequence this issue previously missed:** `OnUpdate` drains the *entire* queue in one pass. A burst arrives as N back-to-back Lua callbacks inside a single frame. A budget scoped to one callback therefore does not bound the frame; the budget must be a deadline shared across every callback within a frame. ## Why threading is not the fix `dmThread` is available to native extensions (defold/defold#5079, closing defold/defold#4835), and as above we already benefit from it. It cannot take any more of this work: 1. **Lua is single-`lua_State` and not reentrant.** `dmScript::SetupCallback`/`PCall` are main-thread only. Anything that builds a Lua table runs on the main thread by definition — including the per-row `lua_createtable`/`lua_setfield` traffic, which is plausibly the larger half of the cost at 10k rows. Unmeasured; see below. 2. **A real win means moving the row cache into C++**, with Lua holding views into it — plus forking `extension-websocket` to intercept frames before they become Lua strings. That is a different project, not a change to this SDK. 3. **HTML5 would lose it anyway.** Emscripten threads require pthreads, which requires `SharedArrayBuffer`, which requires COOP/COEP headers from whoever serves the game. Defold's HTML5 build does not arrange this, and the web target is load-bearing here (`patches/extension-websocket-html5-subprotocol.patch`). For reference, the Godot SDK does move BSATN deserialization to a worker thread — packet queue in, parsed results out, applied on the main thread in `_process` — and its own docs put the cost at "1-2 frames delay". Note that even it auto-disables on non-threaded builds and keeps a single-threaded path. ## Approach Time-slicing, not parallelism. Spread the application of one large update across frames: - A budget per frame — rows, or a `socket.gettime` deadline — after which the remainder defers to the next frame. The deadline is per *frame*, shared across every callback `OnUpdate` delivers in that frame, not per callback. - The natural Lua mechanism is a coroutine per in-flight batch, resumed each frame until it finishes. - **Ordering must hold absolutely.** A partially applied `InitialSubscription` followed by a `TransactionUpdate` touching the same rows, applied out of order, corrupts the cache. Later frames must queue behind an in-progress batch rather than interleave with it. - Callbacks (#21, per-table API) fire only when their batch completes, or a consumer observes a half-populated table and draws half a world. Measure before choosing a number. The honest first step is finding where the cliff actually is: - [ ] Add a conformance scenario subscribing to a table with 10k+ rows, reporting the wall time of the applying frame - [ ] Split that measurement between `json.decode` and cache/table construction — it decides whether a C++ decoder could ever have helped, and it is cheap to collect while we are already measuring - [ ] Decide from that measurement whether time-slicing is needed at all, and at what row count it starts to matter - [ ] Only then implement, with the budget configurable and defaulting to whatever the measurement justifies ## Acceptance A measurement first, posted here, before any implementation. If a 10k-row subscription applies in 3ms, this closes as unnecessary and says so — a perfectly good outcome, and far cheaper than a speculative scheduler. If it is slow: a test proving a large batch spans multiple frames, that the resulting cache is identical to the single-frame result, and that a transaction arriving mid-batch is applied after it rather than interleaved. ## Priority Unknown until measured, which is the point. Worth resolving before 1.0 because the fix, if needed, changes *when callbacks fire* — exactly the kind of observable timing an API freeze should not have to break later.
yuris commented 2026-08-20 04:21:07 +00:00
Author
Owner
Copy link

Research: the native-extension alternatives, before anyone measures

No measurement yet — this is the "would a C/C++ rewrite have been the answer anyway?" question, resolved on paper so it does not get re-litigated after the numbers land. Three findings change the shape of this issue; two of them correct claims in the body above.

1. json.decode is already native C — the parse is not a Lua cost

Defold's json.decode binds to lua_cjson_decode; openresty/lua-cjson is vendored into the engine at engine/script/src/luacjson/ and has been the decoder since 1.4.0 (jsmn before that — see Defold's 2022-11-18 blog post "Improving our json performance"). On HTML5 that is C compiled to wasm.

So the parse cannot be sped up by moving it to a native extension: it is already there. Two consequences:

  • The measurement must use Defold's json, not the test one. dev/test/runner.lua:11-16 injects vendored pure-Lua rxi/json so tests can run outside the engine. The ~5.0 us/row vs ~5.6 us/row note at stdb/protocol.lua:785 does not record which codec it timed. If it was rxi, the in-engine number is likely far better and this issue may already be closed by measurement alone. Verify that first.
  • The only genuinely-Lua per-row costs are ours: quote_wide_ints / quote_big_ints (character-by-character scans over raw row text) and decode_value's per-column coercion walk. Those, not the parse, are the native-extension candidates if one is ever justified.

2. Defold HTML5 does have threads now — and it still does not help

Correcting the "Why the peer solution does not port" section above, which is right about Lua but was written against stale platform facts.

build_tools/waf_dynamo.py (L694-760): with_pthread = False; if build_util.get_target_platform() == 'wasm_pthread-web': with_pthread = True … else: DEFINES += ['DM_NO_THREAD_SUPPORT']. So:

target threads
js-web, wasm-web (default) no — DM_NO_THREAD_SUPPORT
wasm_pthread-web yes — -pthread compile + link, -sPTHREAD_POOL_SIZE=1

Added by defold/defold#10527 + #10575, shipped in 1.10.2 (2025-05-28). We build with 1.13.0, so it is available. Before this, threads on web genuinely did not exist (Mathias Westerdahl, Feb 2024: "we currently use Emscripten 2.x which doesn't have pthread support") — that is the statement the section above was echoing.

It changes nothing here, for three independent reasons:

  1. Lua is still single-lua_State, main-thread only. A worker cannot build a Lua table. The row materialization this issue is about does not move.
  2. wasm_pthread-web needs cross-origin isolation (Cross-Origin-Opener-Policy: same-origin + Cross-Origin-Embedder-Policy: require-corp), and an iframe cannot isolate itself unless every ancestor does. That costs Poki, CrazyGames and GitHub Pages (no custom headers at all — open request since 2022, staff "no ETA"). itch.io supports it behind an opt-in checkbox, at the price of a domain change that discards players' existing localStorage saves.
  3. A plain Web Worker needs none of that — structured-clone postMessage works on every host with zero headers — but it could only take the parse, which is already native C per finding 1.

Also worth recording, since it is the obvious next idea: calling the browser's JSON.parse is a dead end. It is reachable (EM_JS, synchronous, strings via stringToNewUTF8), but wasm linear memory holds raw bytes only, so a parsed JS object can enter wasm just two ways — re-serialize with JSON.stringify and parse a second time, or walk it per-field through emscripten::val at embind's measured ~200ns per call. Both are worse than the in-wasm C parse we already get.

3. Lazy row proxies do not work here

The deeper alternative — keep rows in C memory, hand Lua a userdata whose __index fetches a column on demand, so we never pay for columns the game does not read. Ruled out:

  • cache.lua reads key columns on every row regardless. store() writes t[row[pk]] = row; reindex() reads row[col] for every unique index, per insert. find and pair_by_key compare those values. Laziness could only ever apply to non-key columns of never-touched rows.
  • pairs(row) is a hard error on both our runtimes. Lua 5.1 has no __pairs (added in 5.2); LuaJIT honors it only under -DLUAJIT_ENABLE_LUA52COMPAT, which Defold does not set. Same for rawget(row, …), type(row) == "table", json.encode(row), and pprint (Defold's pprint only recurses LUA_TTABLE). All legal today.
  • Storing a row across frames becomes a use-after-free. A proxy handed to an on_insert callback points into C memory we free on delete, on replace, and on every reconnect (cache.drop_all, which fires no delete callbacks). Making it safe means either a refcounted per-row snapshot — re-spending the allocation laziness saved — or a (table_id, key, generation) handle revalidated on every column read.
  • No JIT on the platforms that need it most. Native desktop is LuaJIT 2.1 with JIT on; iOS and Android build with -DLUAJIT_DISABLE_JIT; HTML5 is vanilla Lua 5.1.4 (no LuaJIT package for web in packages/). SO 61890009 — benchmarked by Tarantool's Mons Anderson, explained by LuaJIT's Igor Munkin — with JIT on, proxy ≈ table; with JIT off, "tables are 20 times faster", because every read is a metamethod lookup plus a call.
  • Nobody does this for a client cache. Both official SpacetimeDB SDKs decode eagerly and cache decoded objects before any user callback: C# sdks/csharp/src/Table.cs (DecodeValue per row into POCOs, MultiDictionary<object, Row> Entries), TS bindings-typescript/src/sdk/table_cache.ts (Map<key, [RowType, version]>). Same for LuaSQL, luadbi, lua-resty-redis. Lazy userdata is proven only in adjacent niches — Tarantool's box.tuple, where the tuple is the engine's storage format, and net.box's opt-in per-response return_raw. The designers who own the BSATN codec and could trivially keep bytes chose to materialize, twice.

Unrelated correction while confirming the above: Defold does not disable the LuaJIT FFI (LUAJIT_DISABLE_FFI appears nowhere; package.preload.ffi() works, since script_module.cpp replaces package.loaders so plain require("ffi") misses). Irrelevant to this issue — HTML5 has no FFI at all — but worth not re-deriving.

Revised plan

Unchanged in shape, sharper in detail:

  • Conformance scenario: subscribe to a table with 10k+ rows, report wall time of the applying frame — in-engine, with Defold's native json
  • Split that time three ways: scan / decode / table-build. Decode being already-native means only the first and third are actionable, and the split says which
  • Decide whether time-slicing is needed at all, and at what row count
  • Only then implement, budget configurable, defaulting to what the measurement justifies

If it is slow, the ranked responses are: (1) time-slice with a per-frame deadline (cheap, pure Lua, works everywhere); (2) port quote_wide_ints + decode_value to an optional C extension, Lua path retained as fallback (helps every platform, no threads, no header requirements); (3) threads — helps least, costs most.

Filed separately, not blocking this

BSATN is an access question, not a speed one: the v2 message schema (InitialConnection, QuerySetUpdate, ReducerResult) is reachable only over v2.bsatn.spacetimedb, and nothing in the tracker needs it today. Two facts found while researching, recorded here so they are cheap to find later:

  • Option is Some = tag 0, None = tag 1 — inverted from idiomatic Rust ordering. Confirmed in crates/sats/src/ser/impls.rs and de/impls.rs. Note the C++ module SDK's own bsatn/reader.h read_optional gets this backwards relative to both the Rust decoder and its own writer.h, so it is not usable as a reference implementation (it also aborts on error via std::abort(), is statically typed rather than typespace-directed, and is BSL 1.1).
  • v2.bsatn compresses by default — brotli above 1 KiB, gzip optional, behind a one-byte tag. ?compression=none on the WebSocket URL avoids the dependency entirely at the cost of bandwidth.

If BSATN is ever built, it is a decoder, not a rewrite: extension-websocket hands binary frames to Lua as 8-bit-clean strings (HandleCallback → lua_pushlstring(L, conn->m_Buffer + msg_offset, msg_length)), so the seam is one pure bytes -> Lua table call and connection.lua / cache.lua / tables.lua are untouched. Try it in Lua first and measure before reaching for C++.

## Research: the native-extension alternatives, before anyone measures No measurement yet — this is the "would a C/C++ rewrite have been the answer anyway?" question, resolved on paper so it does not get re-litigated after the numbers land. Three findings change the shape of this issue; two of them correct claims in the body above. ### 1. `json.decode` is already native C — the parse is not a Lua cost Defold's `json.decode` binds to `lua_cjson_decode`; openresty/lua-cjson is vendored into the engine at `engine/script/src/luacjson/` and has been the decoder since 1.4.0 (jsmn before that — see Defold's 2022-11-18 blog post "Improving our json performance"). On HTML5 that is C compiled to wasm. So the parse cannot be sped up by moving it to a native extension: it is already there. Two consequences: - **The measurement must use Defold's `json`, not the test one.** `dev/test/runner.lua:11-16` injects vendored pure-Lua rxi/json so tests can run outside the engine. The `~5.0 us/row vs ~5.6 us/row` note at `stdb/protocol.lua:785` does not record which codec it timed. If it was rxi, the in-engine number is likely far better and this issue may already be closed by measurement alone. **Verify that first.** - **The only genuinely-Lua per-row costs are ours**: `quote_wide_ints` / `quote_big_ints` (character-by-character scans over raw row text) and `decode_value`'s per-column coercion walk. Those, not the parse, are the native-extension candidates if one is ever justified. ### 2. Defold HTML5 *does* have threads now — and it still does not help Correcting the "Why the peer solution does not port" section above, which is right about Lua but was written against stale platform facts. `build_tools/waf_dynamo.py` (L694-760): `with_pthread = False; if build_util.get_target_platform() == 'wasm_pthread-web': with_pthread = True … else: DEFINES += ['DM_NO_THREAD_SUPPORT']`. So: | target | threads | |---|---| | `js-web`, `wasm-web` (default) | no — `DM_NO_THREAD_SUPPORT` | | `wasm_pthread-web` | **yes** — `-pthread` compile + link, `-sPTHREAD_POOL_SIZE=1` | Added by defold/defold#10527 + #10575, shipped in **1.10.2** (2025-05-28). We build with 1.13.0, so it is available. Before this, threads on web genuinely did not exist (Mathias Westerdahl, Feb 2024: "we currently use Emscripten 2.x which doesn't have pthread support") — that is the statement the section above was echoing. It changes nothing here, for three independent reasons: 1. **Lua is still single-`lua_State`, main-thread only.** A worker cannot build a Lua table. The row materialization this issue is about does not move. 2. **`wasm_pthread-web` needs cross-origin isolation** (`Cross-Origin-Opener-Policy: same-origin` + `Cross-Origin-Embedder-Policy: require-corp`), and an iframe cannot isolate itself unless every ancestor does. That costs Poki, CrazyGames and GitHub Pages (no custom headers at all — open request since 2022, staff "no ETA"). itch.io supports it behind an opt-in checkbox, at the price of a domain change that discards players' existing localStorage saves. 3. **A plain Web Worker needs none of that** — structured-clone `postMessage` works on every host with zero headers — but it could only take the parse, which is already native C per finding 1. Also worth recording, since it is the obvious next idea: **calling the browser's `JSON.parse` is a dead end.** It is reachable (`EM_JS`, synchronous, strings via `stringToNewUTF8`), but wasm linear memory holds raw bytes only, so a parsed JS object can enter wasm just two ways — re-serialize with `JSON.stringify` and parse a second time, or walk it per-field through `emscripten::val` at embind's measured ~200ns per call. Both are worse than the in-wasm C parse we already get. ### 3. Lazy row proxies do not work here The deeper alternative — keep rows in C memory, hand Lua a userdata whose `__index` fetches a column on demand, so we never pay for columns the game does not read. Ruled out: - **`cache.lua` reads key columns on every row regardless.** `store()` writes `t[row[pk]] = row`; `reindex()` reads `row[col]` for every unique index, per insert. `find` and `pair_by_key` compare those values. Laziness could only ever apply to non-key columns of never-touched rows. - **`pairs(row)` is a hard error on both our runtimes.** Lua 5.1 has no `__pairs` (added in 5.2); LuaJIT honors it only under `-DLUAJIT_ENABLE_LUA52COMPAT`, which Defold does not set. Same for `rawget(row, …)`, `type(row) == "table"`, `json.encode(row)`, and `pprint` (Defold's `pprint` only recurses `LUA_TTABLE`). All legal today. - **Storing a row across frames becomes a use-after-free.** A proxy handed to an `on_insert` callback points into C memory we free on delete, on replace, and on every reconnect (`cache.drop_all`, which fires no delete callbacks). Making it safe means either a refcounted per-row snapshot — re-spending the allocation laziness saved — or a `(table_id, key, generation)` handle revalidated on every column read. - **No JIT on the platforms that need it most.** Native desktop is LuaJIT 2.1 with JIT on; **iOS and Android build with `-DLUAJIT_DISABLE_JIT`; HTML5 is vanilla Lua 5.1.4** (no LuaJIT package for web in `packages/`). SO 61890009 — benchmarked by Tarantool's Mons Anderson, explained by LuaJIT's Igor Munkin — with JIT on, proxy ≈ table; with JIT off, "tables are 20 times faster", because every read is a metamethod lookup plus a call. - **Nobody does this for a client cache.** Both official SpacetimeDB SDKs decode eagerly and cache decoded objects before any user callback: C# `sdks/csharp/src/Table.cs` (`DecodeValue` per row into POCOs, `MultiDictionary<object, Row> Entries`), TS `bindings-typescript/src/sdk/table_cache.ts` (`Map<key, [RowType, version]>`). Same for LuaSQL, luadbi, lua-resty-redis. Lazy userdata is proven only in adjacent niches — Tarantool's `box.tuple`, where the tuple *is* the engine's storage format, and net.box's opt-in per-response `return_raw`. The designers who own the BSATN codec and could trivially keep bytes chose to materialize, twice. Unrelated correction while confirming the above: Defold does **not** disable the LuaJIT FFI (`LUAJIT_DISABLE_FFI` appears nowhere; `package.preload.ffi()` works, since `script_module.cpp` replaces `package.loaders` so plain `require("ffi")` misses). Irrelevant to this issue — HTML5 has no FFI at all — but worth not re-deriving. ### Revised plan Unchanged in shape, sharper in detail: - [ ] Conformance scenario: subscribe to a table with 10k+ rows, report wall time of the applying frame — **in-engine, with Defold's native `json`** - [ ] Split that time **three ways: scan / decode / table-build**. Decode being already-native means only the first and third are actionable, and the split says which - [ ] Decide whether time-slicing is needed at all, and at what row count - [ ] Only then implement, budget configurable, defaulting to what the measurement justifies If it is slow, the ranked responses are: **(1)** time-slice with a per-frame deadline (cheap, pure Lua, works everywhere); **(2)** port `quote_wide_ints` + `decode_value` to an optional C extension, Lua path retained as fallback (helps every platform, no threads, no header requirements); **(3)** threads — helps least, costs most. ### Filed separately, not blocking this BSATN is an access question, not a speed one: the v2 message schema (`InitialConnection`, `QuerySetUpdate`, `ReducerResult`) is reachable only over `v2.bsatn.spacetimedb`, and nothing in the tracker needs it today. Two facts found while researching, recorded here so they are cheap to find later: - **`Option` is Some = tag 0, None = tag 1** — inverted from idiomatic Rust ordering. Confirmed in `crates/sats/src/ser/impls.rs` and `de/impls.rs`. Note the C++ module SDK's own `bsatn/reader.h` `read_optional` gets this backwards relative to both the Rust decoder and its own `writer.h`, so it is not usable as a reference implementation (it also aborts on error via `std::abort()`, is statically typed rather than typespace-directed, and is BSL 1.1). - **`v2.bsatn` compresses by default** — brotli above 1 KiB, gzip optional, behind a one-byte tag. `?compression=none` on the WebSocket URL avoids the dependency entirely at the cost of bandwidth. If BSATN is ever built, it is a **decoder, not a rewrite**: `extension-websocket` hands binary frames to Lua as 8-bit-clean strings (`HandleCallback` → `lua_pushlstring(L, conn->m_Buffer + msg_offset, msg_length)`), so the seam is one pure `bytes -> Lua table` call and `connection.lua` / `cache.lua` / `tables.lua` are untouched. Try it in Lua first and measure before reaching for C++.
yuris closed this issue 2026-08-20 16:32:05 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
release/v4.0.1
fix/hold-pre-open-frames
v4.0.2
v4.0.1
v4.0.0
v3.1.0
v3.0.0
v2.0.0
v1.1.0
v1.0.0
v0.1.0
Labels
Clear labels   
area:defold
Defold engine integration

  
area:docs
Documentation and examples

  
area:infra
Flake, CI, secretspec

  
area:protocol
Wire protocol conformance

  
area:testing
Test harness and fixtures

  
kind:bug
Something broken

  
kind:chore
Maintenance

  
phase:breadth
UnsubscribeMulti, procedures, BSATN

  
phase:core
Core SDK layers (protocol/cache/connection)

  
phase:hardening
Reconnect, token persistence, errors

  
phase:slice
Vertical slice: first playable end-to-end

No labels
area:defold
area:docs
area:infra
area:protocol
area:testing
kind:bug
kind:chore
phase:breadth
phase:core
phase:hardening
phase:slice
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
yuris/spacetimedb-defold#30
No description provided.