Developer experience and documentation #77

New issue

Closed

opened 2026-08-21 02:17:41 +00:00 by yuris · 0 comments

yuris commented

2026-08-21 02:17:41 +00:00

Owner

The checklist for making the SDK approachable and its documentation true. The narrative version,
with the reasoning and the evidence, is on the wiki:
Roadmap. Each item gets its own
issue when it starts; tick it here when that lands.

Done

BSATN codec and the v2 default, shipped as 2.0.0 (#9, #94). v2 is what connect speaks
unless a caller opts out, which brings event tables, query-set-scoped subscriptions, reducer
results separated from transaction updates, and 46% of v1's bytes.
HTML5 conformance in CI (#16). Chromium runs the full six-phase smoke inside the job
container, both loads, on path-filtered push and PR rather than a weekly clock.
The HTML5 flake diagnosed and fixed (#81, #93, #98). Two causes under one number. Chromium
outlived its own teardown, and killing the process group ended that. What was left was not a
race: bob resolve re-downloaded extension-websocket on every invocation, fatal when GitHub
refused even with the archive already cached. Both tiers now build offline from a warm
checkout. 21 v2 and 5 v1 runs green locally, then green on CI's own runners.
confirmed_reads documented and pinned per protocol version (#90). The option stays
tri-state, so a defaulted caller gets whichever default the server applies — false on v1, true
on v2 — and a test holds that resolution to peer-SDK parity.
Runtime schema fetch (#97). stdb.schema.fetch reads the schema endpoint and builds the type
tags in Lua, so a consumer can connect and decode with nothing installed beyond the SDK.
stdb.schema.to_lua writes the pinned module a release build wants, through the same walker,
so the two paths cannot drift.
An editor-script regenerate command (#97). stdb/spacetimedb.editor_script puts
Project ▸ SpacetimeDB: Regenerate schema one click away, reading [spacetimedb] base_url
and database from the consumer's game.project. It carries its own JSON decoder because the
editor runtime has none, and it shells out to the consumer's curl because a library's files
are zip entries, not files a shell can execute.
An example project a consumer can open and run (#102). dev/example-chat/ is a real
game.project with a GUI, plus a headless entry point that proves the wiring: it sends a
message and prints PASS only when that row comes back through on_insert.
A coroutine mode beside the callbacks (#102). stdb.await(fn, ...) and
stdb.await_event(db, type), following nakama-defold's opt-in wrapper rather than
auto-detecting a mode. A truthy err raises inside the coroutine instead of being returned
for the caller to forget, and the state guard is what stops a resume on a coroutine that
never yielded.
Expose connection state and latency (#102). db:state() is connecting/connected/
reconnecting/closed, assigned only at the five points the SDK already learns of a
transition. db:latency() is an EMA at weight 1/8 — TCP's own SRTT smoothing — over
request/response pairs the SDK already exchanges, so it costs no traffic; db:ping(cb) forces
a fresh one.
Restructure the README and the wiki for their two audiences (#97, #99, #100, #102). The README
describes the SDK and carries a working example; the Usage Guide's spine is the v2 default,
with v1 on its own V1-Opt-Out page, wire archaeology on Protocol Notes, and harness and CI on
Development.
.script_api — answered, and needs no code. The Defold manual scopes .script_api to
native extensions ("located together with the extension files") and separately says the editor
generates completions for Lua modules a script requires. A pure-Lua library already gets
completion; there is nothing for a .script_api to add. Recorded on the wiki roadmap.
The cross-client light claim is a test, not a measurement (#104). Phase 8 of the v1
conformance run opens a second, ordinary socket and asserts the asymmetry on one commit: the
observer receives TransactionUpdateLight, the caller a full TransactionUpdate. Writing it
found a real defect - stdb.protocol holds its JSON codec in a module local, so a second
connection with a different implementation re-pointed the first one's decoding in silence.
Guarded in connection.new now, and mutation-proven.
The extension-websocket subprotocol fix is upstream, as far as it can be. Reported as
defold/extension-websocket#67 and
answered by PR #68 from a Defold
maintainer, verified across wasm-web, wasm_pthread-web and both macOS targets. Merging is
theirs; #15 stays open until it lands and patches/ can go.
Released as 3.0.0. The
docs had been bumped to archive/v3.0.0.zip before the tag existed, so every install
instruction 404'd; the tag makes them true.

Parity with the official SDKs (#102, 3.0.0)

Shipped together, because they are one decision: the SDK should be recognisable to someone who
read the official docs.

Typed query builder. tables.message:where(function(r) return r.sender:eq("yuris") end)
compiles to SELECT * FROM "message" WHERE "message"."sender" = 'yuris'. This was listed as
impossible in the comparison table and is not: the official builder also compiles to SQL text
in the client (lib/query.ts's toSql), and the subscription grammar is small and fully
specified. Operators eq ne lt lte gt gte, combinators and_ or_ not_,
left_semijoin/right_semijoin. An unknown column, a value whose type cannot be the column's,
a u64 above 2^53 given as a number, a non-hex value for an Identity, an unindexed join column —
each raises at the callsite, naming what was wrong. not_ works despite the grammar having no
NOT, by inverting comparisons and applying De Morgan.
A context first in every callback. on_insert(function(ctx, row) end), carrying db,
reducers, procedures, event, and is_active/disconnect/subscription_builder. This is
the 3.0.0 break, and every caller in the repo moved with it.
Connection builder. stdb.builder():with_url(...):with_database(...):with_schema(gen) :on_connect(...):build(), sugar over stdb.connect, which stays.
Reducer and procedure accessors. db.reducers.send_message({ ... }, cb), generated from
the schema, raising on a typo with the real names listed rather than failing later inside the
encoder.
Subscription builder and handle. db:subscription_builder():on_applied(...):on_error(...) :subscribe(...), with is_active, is_ended, unsubscribe, unsubscribe_then.
The example runs in CI (#103). The v2 leg builds it headless and greps for the PASS it
prints only once the row it sent comes back through on_insert, so the example cannot rot
into a quickstart nobody executes.

Open

Nothing here. What is left of the upstream patch is not this repo's to do, and
#15 tracks it.

Not planned

Reasons recorded so they need not be re-derived: frame compression (a pure-Lua gzip/brotli inflate
is large and slow, and BSATN already halves the bytes); client-side prediction and rollback (a
game-architecture decision belonging above the cache, not inside the SDK); an offline write queue
(replaying writes against a database that moved on, when reconnect deliberately clears the cache and
replays subscriptions); and server-side surfaces such as HTTP handlers, PGWire, submodules and
schedule tables (module-authoring and operations, not a client's job). The wiki page carries the
longer form.

The checklist for making the SDK approachable and its documentation true. The narrative version, with the reasoning and the evidence, is on the wiki: [Roadmap](https://git.nokopia.com/yuris/spacetimedb-defold/wiki/Roadmap). Each item gets its own issue when it starts; tick it here when that lands. ## Done - [x] BSATN codec and the v2 default, shipped as 2.0.0 (#9, #94). v2 is what `connect` speaks unless a caller opts out, which brings event tables, query-set-scoped subscriptions, reducer results separated from transaction updates, and 46% of v1's bytes. - [x] HTML5 conformance in CI (#16). Chromium runs the full six-phase smoke inside the job container, both loads, on path-filtered push and PR rather than a weekly clock. - [x] The HTML5 flake diagnosed and fixed (#81, #93, #98). Two causes under one number. Chromium outlived its own teardown, and killing the process group ended that. What was left was not a race: `bob resolve` re-downloaded extension-websocket on every invocation, fatal when GitHub refused even with the archive already cached. Both tiers now build offline from a warm checkout. 21 v2 and 5 v1 runs green locally, then green on CI's own runners. - [x] `confirmed_reads` documented and pinned per protocol version (#90). The option stays tri-state, so a defaulted caller gets whichever default the server applies — false on v1, true on v2 — and a test holds that resolution to peer-SDK parity. - [x] Runtime schema fetch (#97). `stdb.schema.fetch` reads the schema endpoint and builds the type tags in Lua, so a consumer can connect and decode with nothing installed beyond the SDK. `stdb.schema.to_lua` writes the pinned module a release build wants, through the same walker, so the two paths cannot drift. - [x] An editor-script regenerate command (#97). `stdb/spacetimedb.editor_script` puts *Project ▸ SpacetimeDB: Regenerate schema* one click away, reading `[spacetimedb] base_url` and `database` from the consumer's `game.project`. It carries its own JSON decoder because the editor runtime has none, and it shells out to the consumer's `curl` because a library's files are zip entries, not files a shell can execute. - [x] An example project a consumer can open and run (#102). `dev/example-chat/` is a real `game.project` with a GUI, plus a headless entry point that proves the wiring: it sends a message and prints PASS only when that row comes back through `on_insert`. - [x] A coroutine mode beside the callbacks (#102). `stdb.await(fn, ...)` and `stdb.await_event(db, type)`, following nakama-defold's opt-in wrapper rather than auto-detecting a mode. A truthy `err` raises inside the coroutine instead of being returned for the caller to forget, and the `state` guard is what stops a `resume` on a coroutine that never yielded. - [x] Expose connection state and latency (#102). `db:state()` is `connecting`/`connected`/ `reconnecting`/`closed`, assigned only at the five points the SDK already learns of a transition. `db:latency()` is an EMA at weight 1/8 — TCP's own SRTT smoothing — over request/response pairs the SDK already exchanges, so it costs no traffic; `db:ping(cb)` forces a fresh one. - [x] Restructure the README and the wiki for their two audiences (#97, #99, #100, #102). The README describes the SDK and carries a working example; the Usage Guide's spine is the v2 default, with v1 on its own V1-Opt-Out page, wire archaeology on Protocol Notes, and harness and CI on Development. - [x] `.script_api` — **answered, and needs no code.** The Defold manual scopes `.script_api` to native extensions ("located together with the extension files") and separately says the editor generates completions for Lua modules a script requires. A pure-Lua library already gets completion; there is nothing for a `.script_api` to add. Recorded on the wiki roadmap. - [x] The cross-client `light` claim is a test, not a measurement (#104). Phase 8 of the v1 conformance run opens a second, ordinary socket and asserts the asymmetry on one commit: the observer receives `TransactionUpdateLight`, the caller a full `TransactionUpdate`. Writing it found a real defect - `stdb.protocol` holds its JSON codec in a module local, so a second connection with a different implementation re-pointed the first one's decoding in silence. Guarded in `connection.new` now, and mutation-proven. - [x] The extension-websocket subprotocol fix is upstream, as far as it can be. Reported as [defold/extension-websocket#67](https://github.com/defold/extension-websocket/issues/67) and answered by [PR #68](https://github.com/defold/extension-websocket/pull/68) from a Defold maintainer, verified across `wasm-web`, `wasm_pthread-web` and both macOS targets. Merging is theirs; #15 stays open until it lands and `patches/` can go. - [x] Released as [3.0.0](https://git.nokopia.com/yuris/spacetimedb-defold/releases/tag/v3.0.0). The docs had been bumped to `archive/v3.0.0.zip` before the tag existed, so every install instruction 404'd; the tag makes them true. ## Parity with the official SDKs (#102, 3.0.0) Shipped together, because they are one decision: the SDK should be recognisable to someone who read the official docs. - [x] **Typed query builder.** `tables.message:where(function(r) return r.sender:eq("yuris") end)` compiles to `SELECT * FROM "message" WHERE "message"."sender" = 'yuris'`. This was listed as impossible in the comparison table and is not: the official builder also compiles to SQL text in the client (`lib/query.ts`'s `toSql`), and the subscription grammar is small and fully specified. Operators `eq ne lt lte gt gte`, combinators `and_ or_ not_`, `left_semijoin`/`right_semijoin`. An unknown column, a value whose type cannot be the column's, a u64 above 2^53 given as a number, a non-hex value for an Identity, an unindexed join column — each raises at the callsite, naming what was wrong. `not_` works despite the grammar having no NOT, by inverting comparisons and applying De Morgan. - [x] **A context first in every callback.** `on_insert(function(ctx, row) end)`, carrying `db`, `reducers`, `procedures`, `event`, and `is_active`/`disconnect`/`subscription_builder`. This is the 3.0.0 break, and every caller in the repo moved with it. - [x] **Connection builder.** `stdb.builder():with_url(...):with_database(...):with_schema(gen) :on_connect(...):build()`, sugar over `stdb.connect`, which stays. - [x] **Reducer and procedure accessors.** `db.reducers.send_message({ ... }, cb)`, generated from the schema, raising on a typo with the real names listed rather than failing later inside the encoder. - [x] **Subscription builder and handle.** `db:subscription_builder():on_applied(...):on_error(...) :subscribe(...)`, with `is_active`, `is_ended`, `unsubscribe`, `unsubscribe_then`. - [x] The example runs in CI (#103). The v2 leg builds it headless and greps for the PASS it prints only once the row it sent comes back through `on_insert`, so the example cannot rot into a quickstart nobody executes. ## Open Nothing here. What is left of the upstream patch is not this repo's to do, and #15 tracks it. ## Not planned Reasons recorded so they need not be re-derived: frame compression (a pure-Lua gzip/brotli inflate is large and slow, and BSATN already halves the bytes); client-side prediction and rollback (a game-architecture decision belonging above the cache, not inside the SDK); an offline write queue (replaying writes against a database that moved on, when reconnect deliberately clears the cache and replays subscriptions); and server-side surfaces such as HTTP handlers, PGWire, submodules and schedule tables (module-authoring and operations, not a client's job). The wiki page carries the longer form.

yuris added the

area:docs

phase:breadth

labels

2026-08-21 02:17:41 +00:00

yuris referenced this issue

2026-08-21 04:41:12 +00:00

BSATN codec — the only route to the v2 message schema #9

yuris changed title from ~~Roadmap: what comes after v1.1.0~~ to Developer experience and documentation

2026-08-21 12:09:05 +00:00

yuris referenced this issue

2026-08-21 13:02:18 +00:00