18 Roadmap
yuris edited this page 2026-08-22 17:21:29 +07:00

Roadmap

Where the SDK is going after v4.0.0, and, more usefully, what is deliberately not planned.

One fact about the wire format decides most of what follows, and it is not a fact about this SDK at all.

The wire format is the ceiling

v1.json.spacetimedb was the only JSON protocol SpacetimeDB has, and it is frozen. Every later protocol version is binary, which is what the BSATN codec below went and built, and since 4.0.0 it is the only wire this SDK speaks.

subprotocol encoding message schema
v1.json.spacetimedb JSON v1
v1.bsatn.spacetimedb BSATN v1
v2.bsatn.spacetimedb BSATN v2, the only one here
v3.bsatn.spacetimedb BSATN v2, several messages per payload

That is read off crates/client-api-messages/src/websocket/{v1,v2,v3}.rs in the SpacetimeDB repository. v1.rs declares both a TEXT_PROTOCOL and a BIN_PROTOCOL; v2.rs and v3.rs declare only BIN_PROTOCOL. The server offers exactly those four and negotiates them newest-first, and crates/client-api/src/routes/subscribe.rs lists V3, then V2, then v1 binary, then v1 text.

v3.rs describes no new message schema. It is permission to put one or more consecutive BSATN v2::ServerMessage values in a single websocket payload. Either side may coalesce or not, independently, so long as logical order holds. So v3 is a framing change, and supporting it is a loop over the reader rather than a second codec.

Where the official clients actually sit, read off their source rather than their changelogs.

client subprotocol source
Rust SDK v2, hardcoded sdks/rust/src/websocket.rs:250,371
C# SDK v2, hardcoded sdks/csharp/src/SpacetimeDBClient.cs:226
Unreal SDK v2, hardcoded sdks/unreal/.../Websocket.cpp:52
TypeScript SDK v1 BSATN packages/sdk/src/db_connection_impl.ts:233
spacetime subscribe v3, strict crates/cli/src/subcommands/subscribe.rs:254

None of the SDKs negotiates. Each sends one subprotocol, and its v2 types are welded into its signatures. The CLI is the only first-party v3 client, and it verifies that the server echoed v3 rather than accepting a downgrade. The TypeScript SDK is a protocol version behind this one.

So the JSON protocol still works against a 2.x server, but it will not grow, and as of 4.0.0 this SDK no longer offers it. The clearest example of the ceiling is event tables, the 2.x feature aimed most squarely at games. They are transient rows for damage numbers, chat, hit indicators and sound cues, broadcast to subscribers and never stored. The documentation is explicit that they need v2, and that a v1 client which subscribes to one gets an error telling it to upgrade. There was no JSON path to them, which is why the v2 path was built, and event tables are the first thing it delivered.

That is why the roadmap below is shaped the way it is. There is a stretch of work that makes the SDK pleasant to use inside the ceiling it already has, and one large piece of work that raised the ceiling. That piece has landed, and since 4.0.0 it is the only thing under the SDK.

Now, making it usable by someone who did not write it

The SDK has been more capable than it is approachable, and closing that gap is the work in front of it. None of it needs a new wire format. The tracked checklist, with what is ticked, lives on Developer experience and documentation; this section is the reasoning behind those items, which is the part that goes stale slowest.

Getting a schema is no longer the blocker it was. A consumer installs stdb/ and nothing else. dev/ is export-ignored, so dev/tools/gen-schema.sh was never in the archive they receive, and the Quickstart told them to run it anyway. Typed decoding is the SDK's headline behaviour (Identity as a hex string, Timestamp as micros, exact 128-bit integers, and the only description a view can have) and it depends on a generated schema, so that path was documented and undeliverable. Hand-writing the schema instead is a footgun the generator exists to remove. Column order pairs with type tags by position, sum variants are identified by ordinal, and some row encodings are positional. 2.0.0 raised the stakes rather than changing them. On the default wire a hand-written schema cannot decode a row at all, so the gap stopped being between typed rows and untyped ones and became one between rows and none.

stdb.schema.fetch reads the schema off a running server with no external tool, stdb.schema.to_lua writes the pinned module a release wants, and Project ▸ SpacetimeDB: Regenerate schema does the second in one click. Defold reloads a library's editor scripts on Fetch Libraries, and editor.execute() can run a shell command and capture its output, which is what lets a library ship a command at all. Both paths walk one Lua type-tag transform, so a fetched schema and a written one cannot disagree, and gen-schema.sh stays in the repo as the contributor drift check it always was.

What is still missing is smaller, and each piece has its own reason.

A Defold tutorial. dev/example-chat is now the openable sample this list used to ask for, with its own game.project, a GUI script and a headless variant CI runs on every conformance job, and the wiki's Quickstart walks a module through to a connected game. What is still absent is the thing upstream ships for Unity, Godot and Unreal, a staged tutorial hosted where a newcomer looks for one rather than in this repository's wiki.

.script_api is decided rather than pending. No .script_api file is planned. extensions-script-api scopes the format to native extensions. It is how a native extension ships auto-complete for the C/C++ API it exposes to Lua, "located together with the extension files." writing-code states separately that the editor already provides auto-complete for Lua functions, including modules loaded with require, which is exactly what this SDK is, a pure-Lua library reached through require("stdb.init") with no native extension of its own. There is no API surface .script_api would add here, so this item closes with evidence rather than code.

A coroutine mode beside the callbacks has shipped as stdb.await and stdb.await_event, see Done.

Saying what the connection is doing has shipped as db:state(), db:is_connected(), db:latency() and db:ping(cb), see Done.

The HTML5 subprotocol fix landed upstream (#15) and is done. extension-websocket 4.2.4, released 2026-08-22, honours the protocol option on HTML5, so the handshake carries Sec-WebSocket-Protocol and the server no longer answers 400 no valid protocol selected. HTML5 is a first-class target now. A consumer adds two plain dependency URLs and is done on every platform, with no patch, no fork and no vendoring step anywhere. Reported upstream as defold/extension-websocket#67, with the two handshakes but not the cause, so a maintainer had room to reach the ordering independently, and did. PR #68 separates connection allocation from platform startup so the Emscripten constructor never sees a half-initialised connection, verified on wasm-web, wasm_pthread-web and both macOS targets. The patches/ directory that carried the fix locally is gone with it.

One thing the reproduction settled is that custom headers are a separate problem. Emscripten's WebSocket API cannot set arbitrary request headers at all, so the fixed build sends the subprotocol and still no Authorization. Passing the token as ?token= is permanent on HTML5 rather than a stopgap waiting on upstream.

Next, making the documentation true

This is mostly about not overclaiming, and it is largely done. The restructure this wiki has just been through is that work. The README describes the SDK and the repo instead of project status, the Usage Guide leads with the wire a bare connect actually negotiates rather than correcting itself eleven hundred lines in, and the archaeology (upstream citations, captures, benchmark narratives) sits on Protocol Notes and Development rather than in a reader's way.

light has its two-client test now (#104). This page used to say the claim was broader than the evidence, and it was. Only your own updates had been exercised, so "changes nothing observable" rested on a hand-run measurement. Phase 8 of the conformance run settled it, on the v1 wire, before 4.0.0 deleted the light option along with that wire. An ordinary observer, with no flag anywhere, watches a commit caused by the main socket, and the asymmetry is asserted on that one transaction. The observer receives TransactionUpdateLight and the caller a full TransactionUpdate. The test asserts both halves together, because either alone is consistent with a server that ignores the flag. The documentation was right and now has a test behind it, which was the outcome this item asked for either way.

Writing it turned up a defect worth more than the item. stdb.protocol was pure and kept its JSON codec in a module local, while connection.new injected one per connection, so a second connection carrying a different implementation re-pointed how the first one decoded its rows, with nothing to notice it by. It surfaced as a wrong number rather than an error. The observer's uninstrumented codec left the measurement on the other socket reporting zero decode calls for ten thousand rows. Two sockets in one process are a supported thing to want, which is what made it reachable. A guard in connection.new fixed it, and 4.0.0 retired both the guard and the hazard. connection.new no longer takes a json at all, so there is no process-global codec two connections could disagree about.

HTML5 conformance in CI (#16) is done. Chromium runs in the job container and the wasm bundle passes the full smoke on both loads, with the identity intact across the reload. It was never blocked on #15 the way this page once claimed. While the fix was still local the harness unpacked the extension bob resolved and applied it itself, so the run carried it without a fork or an upstream release. Since 4.2.4 the harness builds against the released extension like any consumer. The weekly cron is gone too, since nothing here drifts on its own with the server, CLI, chromium and bob all pinned through flake.lock; the tier runs on the commits that touch it.

#81 is what came out of that. The HTML5 tier fails roughly one run in five and then passes on the same commit. It gates anyway. HTML5 is the only place the wasm build of the binary wire runs at all, so it had to gate before v2 could be the default, and the bet is that a flake blocking a merge gets diagnosed where one printing a footnote never does. It still posts its verdict to that issue as well, because this Forgejo serves no job logs over the API. Suspects are IDBFS flush timing at the reload boundary, the phase-2 server restart, and Chromium startup under load; none confirmed. A named cause retires the reporting step; a second flake on one commit puts continue-on-error back.

#52 stays open on purpose. One conformance run once observed a subscription payload missing a row, it has never reproduced, and the mechanism originally proposed for it was disproved. It is harness reliability rather than shipped behaviour. If it recurs, capture the raw frames.

Done, and where the ceiling now is

The DX pass landed (#111). It fixed five silent failures and deleted three required arguments. connect{ schema = gen } now takes the whole generated module and reads typespace, reducers and procedures off it, so the old four-option spelling is still valid but no longer what the docs teach. json defaults to the engine's json global on stdb.schema.fetch and stdb.auth.exchange, and is only passed off-engine, such as in tests. stdb.query is re-exported from stdb.init, so the typed builder is stdb.query.tables(gen) with no second require. db:ping(cb) calls back (err, seconds), matching every other async entry point instead of inverting against them. stdb.await wires the handle of the call it awaits, so an awaited reducer no longer parks forever when the socket drops and an awaited subscribe survives a reconnect replay, and stdb.await_event(db, "connected") resolves immediately when the connection is already up. What used to fail quietly now raises and names the mistake. That covers an unknown db.tables.<name>, an empty reducer or procedure section, a base URL with no ws:// or wss:// scheme, a row with bytes left over after decoding (regenerate the schema), and ctx.db.tables, which does not exist because db.tables, db.db and ctx.db are one object under three names. stdb.builder(opts) takes a seed table for the options the fluent setters do not cover. The editor command requires database under [spacetimedb] in game.project rather than defaulting to chat; base_url still defaults to http://127.0.0.1:3000.

The v1 JSON wire is gone, as of 4.0.0. It was deleted rather than deprecated, along with every option that selected it. What went is the protocol option, light, json on connect, Builder:with_protocol, the legacy whole-set db:subscribe (and with it subscription_replaced), Call:on_out_of_energy, SUBPROTOCOL_V2, and the untyped cache decode path. Every one of those was already unreachable on the default wire. The v2 ClientMessage has no legacy Subscribe, ReducerOutcome has no OutOfEnergy variant to set, and light raised at connect. stdb/ lost 1710 lines, 20% of it.

The decision was upstream's before it was ours. No official SDK keeps v1 selectable, as the table above shows, and this SDK was the only client carrying both wires, paying for it with a second implementation of every send path, a parallel handler table, and a JSON codec that shipped to every consumer inside include_dirs = stdb.

One piece survived the deletion, moved rather than removed. The v1 JSON decoder now lives at dev/test/crosscheck_v1.lua, test-only and not shipped. It stays because BSATN is opaque, since a wrong belief about a column's type returns a plausible number instead of raising, so test_protocol_v2.lua decodes the same captured frame both ways and asserts the two Lua values are equal. Mutating the BSATN identity byte order or its u64 reader fails that check. The v1 encoder was deleted outright, because an encoder has no second implementation to disagree with.

connection.dispatch(c, msg) was split out of handle_frame, which is now decode plus dispatch. Decoding bytes and acting on a message are two jobs, and the split is what lets the connection tests drive the whole server-to-client surface from a plain Lua table, since there is no server-side BSATN encoder in the repo to build frames with.

Gzip frames are decoded, as of 3.1.0. The server compresses any message over 1KiB (decide_compression, websocket_building.rs) and its default scheme is brotli, which is why connect never omits the parameter. An omitted one is a wire no Defold build can read. It asks for gzip when a gunzip is in reach and for none when it is not, and inflates with the engine's builtin zlib.inflate, which only learned to read a gzip wrapper in Defold 1.12.4, now the floor. The prize is bandwidth on the one message that reliably crosses the threshold. A 33KiB subscription readback measured 14KiB compressed, and a 479KiB one 204KiB. Below 1KiB nothing changes, so the ordinary per-transaction update is untouched. Brotli stays unreadable and raises by name; a pure-Lua inflate was never on the table, and does not need to be now that the engine ships one.

A BSATN codec (#9) is implemented. stdb/bsatn.lua is the primitive codec (integers to 256 bits, IEEE-754 floats, length-prefixed strings, all little-endian, all in arithmetic, because Lua 5.1 has no bitwise operators), and stdb/protocol_v2.lua is the v2 message schema in both directions plus a typed row decoder. connection.lua maps v2 messages onto the event and cache surface the SDK already had, so existing on_event consumers did not fork.

It unlocked event tables, query-set-scoped subscriptions, and reducer results that arrive separately from the transaction they caused, addressed to the caller rather than broadcast to every subscriber. Event-table rows get their own event type and never enter the cache, which is the point of them. The Usage Guide covers using them, and v2 over BSATN documents the wire.

A typed query builder, coroutine mode, and connection state and latency (#77) are implemented. stdb/query.lua compiles tables.message:where(...) and :left_semijoin(...) to the same SQL string a hand-assembled subscribe_query call takes, with the official SDKs' own operator names (eq/ne/lt/lte/ gt/gte) and structural not_ support despite the grammar having no NOT token. stdb/await.lua adds stdb.await/stdb.await_event, a coroutine wrapper beside every existing callback, mirroring the nakama-defold precedent this page named. db:state()/db:is_connected()/db:latency()/ db:ping(cb) on stdb/connection.lua answer what this page called "no exposed connection state and no latency figure". State comes from the same five call sites that already learn of a transition, and latency is a smoothed EMA fed by every correlated request/response. See the Usage Guide and API Reference for all three.

Context-first callbacks, stdb.builder(), reducer and procedure accessors, and db:subscription_builder() (#77) are implemented, matching the official SDKs' shape (contract §1-§5 in the 3.0.0 parity work). Every row callback, every stdb.builder() lifecycle callback, and every db:subscription_builder() callback now receives ctx first, one table built once per connection (build_ctx in stdb/connection.lua), holding ctx.db/ctx.reducers/ctx.procedures (the same objects as db.db/db.reducers/db.procedures) and the mutable ctx.event, plus ctx:is_active()/ctx:disconnect()/ctx:subscription_builder(). stdb.builder() is sugar over stdb.connect. with_url/with_database/with_token/ with_schema/with_protocol collect options build() passes through in one call, and it derives on_connect/on_connect_error/on_disconnect from the existing on_event stream rather than a separate wire signal. db.reducers.<name>/db.procedures.<name> are built eagerly, one closure per name in the generated schema, over the unchanged db:call_reducer/db:call_procedure; an unknown name raises listing what the schema does have, and a wrong argument count raises before anything is sent. Only the count is checked; an argument's type still is not, despite the contract asking for both. db:subscription_builder() wraps db:subscribe_query (which stays exactly as it was) and its handle raises on a second unsubscribe/unsubscribe_then once the set has already ended, where the older Subscription:unsubscribe stays a no-op. db.db (the official name for db.tables) and db:disconnect() (alias of db:close()) round out the naming parity, and Table:remove_on_insert/ remove_on_update/remove_on_delete are the official spellings for off_insert/off_update/ off_delete.

One deviation is worth naming on its own. ctx.event.reducer was { name, args } on v1 only. A ReducerResult carries no reducer name or arguments on the wire, so ctx.event.reducer is nil for the "reducer" tag. That is shipped behaviour rather than a gap to close, and since 4.0.0 there is no other wire it could differ on.

See the Usage Guide and API Reference for the full surface and worked examples.

Conformance was green on all four suites at the time, meaning the full smoke over v1 and over v2, each of them natively and in the browser, as a matrix over protocol: [v1, v2] with fail-fast off, so a broken wire could not delete the other one's verdict. Running v1 beside v2 was the half that mattered most, because it is what showed a second protocol cost the first nothing, and the HTML5 pair is what proved binary frames survive emscripten. 4.0.0 collapsed that matrix to a single leg. There is one wire to hold to the gate now, and the cross-client phase was rewritten to prove the v2-native claim (a broadcast TransactionUpdate carries no request_id at all) rather than v1's light-frame asymmetry.

Shipped as 3.0.0. It is worth recording how close that came to being a lie. The docs were bumped to archive/v3.0.0.zip as part of the parity work and the tag was never cut, so for a day every install instruction in the README and this wiki pointed at a 404 while the one URL that did resolve, v2.0.0.zip, shipped the 2.x callback signature the docs no longer described. A version in game.project is not a release, and the docs are what makes the difference visible.

The default became v2 in 2.0.0, and the only wire in 4.0.0. connect negotiates v2.bsatn.spacetimedb; between those releases connect({ protocol = "v1" }) still selected the JSON wire, run against a live server on every conformance trigger. The same rows decoded to the same Lua values over either protocol for every type but f32, which is what the differential test exists to prove. f32 cannot match. BSATN carries the stored single's four bytes, so 0.1 reads back as 0.10000000149011612, that single's exact value, where v1's decimal text parses to the double 0.1. Only values that do not fit 24 mantissa bits diverge, and v2 is the more faithful side, since it returns what the server stored. That was the substance of the decision rather than a footnote to it. The flip makes one column type read more accurately and stop comparing equal to a literal written against v1, and it is the only value change an upgrading consumer meets quietly. The rest of the flip is loud by construction, with one exception named below. on_out_of_energy simply stops firing.

Constraints the binary wire carries, all of them permanent rather than pending. Nothing on it names a column, so a table can only be decoded from a generated schema that carries its types, and tables without one are reported undecodable and skipped, the same way the SDK already handles a table it cannot type. Calling a reducer or procedure needs reducers/procedures from the generated module, since BSATN carries neither argument names nor widths. And there is no out-of-energy signal. The outcome has no OutOfEnergy variant and no substitute, so that handling belongs in on_error, which is why 4.0.0 removed the handle for it rather than leaving a callback that could never fire. light and the legacy whole-set subscribe had no v2 frame either; 2.0.0 raised on both, and 4.0.0 removed them.

The benchmark that once gated this work became a deliverable of it. The byte figure is settled. A v2 subscription readback is 46% of v1's bytes, and byte counts do not care what else the machine is doing. The latency figure is not settled to the digit. v2 is consistently faster at every row count, but a re-run of the same commit on a loaded box measured every absolute 2-3x higher and put the 10k ratio at 0.59-0.65 rather than the 0.53 first published. A clean re-measure on a quiesced machine is an open follow-up on #9. The full tables, per row count, both wires, with the bucket split, the browser tier and the measurement conditions, are in Protocol Notes, and the numbers are in #9.

v3 stays out of scope. It is the same message schema with several messages coalesced into one frame; v2 sends exactly one, which is what keeps frame handling unchanged.

Not planned

Recorded with reasons, so the questions do not have to be re-answered.

  • Client-side prediction and rollback. SpacetimeDB's model is server-authoritative, so reducers mutate state on the server and clients receive the committed result. Predicting locally means keeping a speculative copy of the cache and reconciling it, which is a game-architecture decision with no single right answer. It belongs in game code, on top of the cache this SDK gives you, rather than inside the SDK.
  • A write queue for offline play. Reducer calls made while the socket is down are settled with an error rather than held. Silently queueing them would mean replaying writes against a database that moved on, and after a reconnect this SDK clears the cache and replays every subscription precisely because the old state cannot be trusted. An application that wants offline writes needs its own intent log with its own conflict rules.
  • Server-side surfaces. HTTP handlers, the PostgreSQL wire protocol, submodules and schedule tables are module-authoring and operations features. They are worth knowing about when writing a module; none of them is a client SDK's job.

How this page is maintained

Every item here is a candidate rather than a commitment, and this page is not the tracker. #77 carries the developer-experience and documentation checklist, and anything larger gets an issue of its own when it starts. This page holds the reasoning, meaning why an item is worth doing and what a reader would otherwise have to re-derive. If an item lands, the narrative here says so and the change shows up in the release notes. If an item is abandoned, it moves to Not planned with the reason, and the reasons are the most useful part of this page.