9 releases 9 tags

RSS feed

• v0.1.0 8768170cd9

  Compare

  v0.1.0 Stable

  yuris released this 2026-08-19 10:04:01 +00:00 | 82 commits to main since this release

  First tagged release of the SpacetimeDB 2.x client SDK for Defold, in Lua. Every capability below
  was exercised against a live clockworklabs/spacetime:v2.8.2 server from inside a real Defold
  engine, and the wire contract it depends on is captured rather than inferred: test/fixtures/session.jsonl
  holds 21 real frames and test/test_fixtures.lua replays them, so a server bump shows up as a
  fixture diff instead of a runtime surprise.

  Features

  Protocol rewritten against captured 2.8.2 wire traffic

  The first implementation was written from the docs and was wrong in four ways, each found by
  publishing a JS chat module to a live 2.8.2 server and capturing the session. v1.json.spacetimedb
  carries the v1 message schema, not v2 - the v2 schema (InitialConnection, QuerySetUpdate,
  ReducerResult) exists only on the v2 BSATN wire, so the first frame on the v1 JSON wire is
  IdentityToken. Under JsonFormat rows and reducer arguments are double-encoded (ByteString:
  JSON strings containing JSON). Row shape depends on the message: InitialSubscription sends named
  objects, TransactionUpdate sends positional arrays, which is why stdb/schema.lua requires
  columns - the declared order is what names the fields. And reducers are addressed by their
  canonical snake_case name, so a JS sendMessage is send_message on the wire.

  stdb/protocol.lua, stdb/cache.lua and stdb/connection.lua were rewritten against those
  findings. protocol.unwrap normalises wrapped scalars ({"__identity__":"0x…"},
  {"__timestamp_micros_since_unix_epoch__":n}) and options ({"some":v} / {"none":[]}).
  Reaching the v2 BSATN wire needs a BSATN codec and is not part of this release.

  Reconnect with backoff, cache drop and subscription replay

  stdb.connect now owns a reconnect state machine rather than leaving resubscribe_all()
  unreachable. On socket close it retries with exponential backoff (backoff_base and
  backoff_cap are options, reconnect = false opts out), reuses the server-issued token so
  identity survives, and on the new socket's IdentityToken clears the cache and replays every
  live subscription. Two pure helpers carry the logic and are unit tested: connection.backoff_delay
  (doubling with a clamp) and connection.set_send, which swaps the transport and drops requests
  owed by the dead socket so their callbacks can never fire late.

  The engine test proves it end to end: the harness runs docker restart under a live client, and
  the run only passes if the client reconnects, replays its subscription and reads the pre-restart
  row back out of the cache.

  Identity and token persistence across restarts

  SpacetimeDB issues an identity and token on first connect. stdb/connection.lua gained an
  injectable { load, save } storage seam, written through on IdentityToken; stdb/init.lua
  defaults it to stdb.sys_storage() over sys.save/sys.load and prefers, in order, the live
  token, an explicit opts.token, then the persisted one. storage = false opts out.

  Verified in-engine, not asserted: conformance runs the headless client twice, the second time with
  no token in the environment, and fails unless both runs report the same identity.

  Droppable subscription query sets

  The legacy Subscribe message replaces the entire subscription set and cannot be undone one query
  at a time. db:subscribe_query(queries) returns a query id over SubscribeMulti and
  db:unsubscribe(qid) drops it over UnsubscribeMulti.

  Three server behaviours are pinned as fixtures. An unsubscribe reply carries the now-unmatched rows
  as deletes, so the cache path that fills on subscribe drains on unsubscribe. The server computes
  the real delta: dropping one of two overlapping subscriptions deletes nothing because the other
  still matches, so a caller needs no query-to-row bookkeeping. An overlapping subscribe returns
  tables:[], contradicting upstream's doc comment, so a UI that repopulates from a subscribe
  payload renders empty - read the cache instead. The single-query forms nest rows as
  .rows.table_rows (a bare TableUpdate) while the multi forms use .update (a DatabaseUpdate
  with a tables array); decoding one as the other silently applies nothing, so both are normalised
  in one place. Reconnect replays query sets under their original query_id so caller-held handles
  stay valid, and a set the server rejected is dropped from the replay list.

  Procedure calls with the return value decoded

  db:call_procedure completes the path to the existing ProcedureResult handler and hands the
  callback a decoded value rather than the raw message. Procedure returns are double-encoded by the
  same ByteString rule as reducer arguments - status.Returned is "2" for a u64 2 and
  '"echo: from lua"' for a string - so protocol.decode_payload unwraps it. Captured from three
  test procedures (u64 return, string return with args, and one that throws).

  Two limits worth knowing. A thrown error and an unknown procedure name produce the same status,
  InternalError, differing only in message, so there is no distinct "no such procedure" variant to
  branch on; request_id correlates directly and is not Option-wrapped. And the HTTP schema
  endpoint has no procedures key at all, so tools/gen-schema.sh cannot emit procedure names even
  though spacetime describe --json lists them.

  Schema generation from a live server

  Column order is load-bearing, and hand-maintaining it is silent: reorder a column in the module and
  every client mis-maps fields with no error anywhere. tools/gen-schema.sh reads
  /v1/database/<db>/schema and emits a Lua module with columns in declaration order, the primary
  key, and canonical reducer names with params. It is wired in rather than optional -
  tools/dev-server.sh regenerates test/fixtures/chat_schema.lua on every publish and
  test/test_fixtures.lua consumes the generated file instead of a hand-typed copy, so a column
  reorder fails the suite with expected id,sender,text, got id,text,sender.

  Headless Defold conformance test

  The extension-websocket binding in stdb/init.lua has no unit coverage and cannot get any. The
  repo is therefore also a Defold library project (game.project with include_dirs=stdb) whose
  bootstrap collection runs main/smoke.script, a headless client, against a live server:
  tools/dev-server.sh brings the server up and publishes the module, tools/conformance.sh builds
  headless with bob and asserts SMOKE: PASS across four phases (round trip, restart survival,
  query set drop, procedure round trip).

  It caught two engine-only bugs on its first run. extension-websocket defaults to binary frames,
  so the server tried to BSATN-decode the JSON and dropped the connection with
  unknown tag 0x7b for sum type ClientMessage; frames are now sent with
  { type = websocket.DATA_TYPE_TEXT }. And require("websocket") fails at build time because
  native extensions register globals, so the binding uses _G.websocket. Running Defold's generic
  Linux binaries on NixOS needs help: defold-patch rewrites the interpreter and rpath, with a
  buildFHSEnv wrapper as fallback, because the FHS env needs user namespaces that unprivileged job
  containers usually deny. On the project's own Forgejo runner the conformance job stays out of the
  blocking gates - it lives in conformance.yml on workflow_dispatch plus a weekly schedule and
  self-reports to issue #7 - while
  stylua, the Lua suites and nix flake check gate every push.

  Consumable as a Defold library dependency

  Forgejo serves /archive/<tag>.zip for any tag, so the dependency URL works from the tag alone,
  and the release workflow publishes notes from the tag annotation read over the API (curl only, no
  toolchain). Verified by building a separate consumer project with no local copy of stdb/,
  resolving the release URL with bob and running the bundle in a real engine.

  Two facts that project came back with, both now in the README. Defold does not resolve dependencies
  transitively, so a consumer that lists only this SDK compiles and then fails at connect with
  extension-websocket missing: add it to game.project dependencies. And Forgejo refuses basic auth
  for archive downloads, which is all bob speaks: measured, https://user:token@host/... returns
  404 and ?token= returns 200, so a private repo needs the token as a query parameter.

  What's Changed

  • release: read the tag annotation from the API, publish a reachable URL in 8768170
  • release: ship as a Defold library dependency, verified by consuming it in 394c228
  • procedures: add call_procedure and decode the returned value in a20f026
  • subscriptions: add droppable query sets (subscribe_query / unsubscribe) in 3528c2b
  • schema: generate table descriptors from the live server in ad3779c
  • identity: persist the server-issued token across app restarts in 7c0fac9
  • reconnect: wire the state machine into the websocket binding in 4c0682a
  • ci: split conformance into a manual workflow in c870e63
  • conformance: patchelf the engine binary instead of bubblewrap in 3c14fdb
  • ci: stop gating conformance on a step output in a3043a7
  • ci: attach server to the job's netns when running containerised in b30a36a
  • ci: grant issues:write and capture runner docker topology in 8989aa8
  • ci: probe a reachable server address instead of assuming localhost in 9e8988d
  • ci: conformance job non-blocking, self-reports to issue #7 in 3be6593
  • engine: headless conformance test — SDK verified inside real Defold in 2b32580
  • protocol: rewrite against captured 2.8.2 wire traffic (v1 schema) in ee90548
  • ci: lean ci devShell, provision json.lua in shellHook in 0a6602c
  • ci: unit tests + stylua + flake check on push and PR in 57a3012
  • spacetimedb-defold: SpacetimeDB 2.x client SDK for Defold — scaffold in d0583b2

  Full Changelog: v0.1.0

  Use as a Defold library

  Add to game.project dependencies:

  https://git.nokopia.com/yuris/spacetimedb-defold/archive/v0.1.0.zip
  https://github.com/defold/extension-websocket/archive/refs/tags/4.2.3.zip
  

  extension-websocket is required: Defold does not resolve dependencies
  transitively, so without it connect fails at runtime.

  Private repo? Forgejo rejects basic auth for archive downloads; append
  ?token=<access-token> to the URL instead.

  Downloads

  • Source code (ZIP)
    16 downloads
  • Source code (TAR.GZ)
    1 download