Usage Guide
Everything past the first connection. The Quickstart builds the
stdb.connect call these sections extend, so the "on_event above" that the
first section refers to is the one in that example.
The SDK speaks one wire format, v2.bsatn.spacetimedb, the binary one. There is
no protocol option and nothing to negotiate. SpacetimeDB 2.x serves the v2
message schema on that subprotocol alone, and every official SDK hardcodes it.
This is a connection with a generated schema.
local gen = require("my_schema")
local db = stdb.connect({
url = stdb.url("ws://127.0.0.1:3000", "my_db"),
-- The generated module, whole. `connect` reads `typespace`, `reducers` and
-- `procedures` off it, so one option covers all four sections.
schema = gen,
})
The four sections can still be passed separately, and an explicit one wins over the same section read off the module.
local db = stdb.connect({
url = stdb.url("ws://127.0.0.1:3000", "my_db"),
schema = stdb.schema.new(gen.tables),
typespace = gen.typespace,
reducers = gen.reducers, -- required to call a reducer
procedures = gen.procedures, -- required to call a procedure
})
They must all come from the same generated module. Mixing generations decodes
rows against columns the server no longer has, and dropping only reducers
still decodes tables fine and fails at the first reducer call instead, which is
the one failure that does not point at the option that caused it. That is the
argument for schema = gen.
stdb.url builds the subscribe URL and requires a scheme on the base, so
ws://127.0.0.1:3000 rather than 127.0.0.1:3000. A bare host raises here, naming the
missing scheme, because concatenating without one produces a URL that fails
inside the engine's websocket and names neither this function nor the reason.
The frame envelope, the message map, the four wire facts the capture settled and what the decoder does with a hostile frame are in Protocol Notes.
A binary wire has four consequences worth stating once, because each is a
raise or a silence rather than a degraded value. Calling a reducer or procedure
raises at the call without reducers and procedures from the generated
module, because BSATN carries neither argument names nor widths (see Generating
the schema). A table whose schema entry has no
generated types is skipped and
reported rather than decoded, so a
hand-written schema decodes nothing at all. A transaction carries no
reducer_call and no caller_identity (Reconnect). And three
payloads stay raw bytes because nothing on the wire names their type. Those are
a one-off query's rows (One-off query rows), and a
reducer's ret_value and Err payload, the last of which reaches the callback
as a printable summary instead (Handles).
3.0.0 adds a ctx-first layer on top of everything above, additive over every
call this page documents. Every row callback below, and both builders'
lifecycle callbacks, now take a ctx as their first argument, one object per
connection built once rather than once per callback, ahead of whatever else
the callback already received.
-- 2.x
db.tables.message:on_insert(function(row) label.set_text("#label", row.text) end)
-- 3.0.0
db.tables.message:on_insert(function(ctx, row) label.set_text("#label", row.text) end)
The row shifts right; nothing else about it changes. ctx carries
ctx.db/ctx.reducers/ctx.procedures, which are db.tables/db.reducers/
db.procedures under the official SDK's own names for them, and ctx.event,
the tagged cause of the batch a callback is inside. Full detail
is in Per-table callbacks; the
db.reducers.<name>/db.procedures.<name> accessor sugar is in
Handles; the subscription-builder alternative to
subscribe_query is in Dropping a subscription.
The page is long and the wiki has no search, so the sections are listed here:
- Coroutine mode, the same calls without a callback
- Per-table callbacks, row callbacks per table, and why deregistration is not optional
- Dropping a subscription, query sets, hold counting, and what a subscribe reply does not tell you
- Handles, the
SubscriptionandCallstate machines - Procedures, a call that returns a value
- Reconnect, backoff, replay, every event
on_eventreceives, and one-off query rows - Connection builder, fluent sugar over
connect, and the lifecycle callbacks derived fromon_event - Event tables, transient rows, broadcast and never cached
- Identity persistence, an identity that survives a relaunch, HTML5 caveats included
- Authenticating with a provider, trading a Steam, Google or Apple credential for a token
- Generating the schema, why column order is load-bearing, and the generator that gets it right
- Views, the relation whose column order nothing else in your project states
- Relations your schema does not describe, rows that are dropped, and how you are told
- Row value types, what each column decodes to, and the two ceilings Lua 5.1 imposes
Coroutine mode
Every asynchronous call above takes a callback. stdb.await/stdb.await_event
are a second style over the same calls, for sequential logic that reads better
without one.
coroutine.wrap(function()
local sub = stdb.await(db.subscribe_query, db, { "SELECT * FROM message" })
local update = stdb.await(db.call_reducer, db, "send_message", { "yuris", "hi" })
end)()
stdb.await(fn, ...) wraps any one-shot (err, ...) callback this SDK uses,
so call_reducer, subscribe_query, call_procedure, one_off_query,
unsubscribe and ping. fn is called as a plain function with one callback
appended, so a method needs its db passed explicitly, while the accessor form
captures it already (stdb.await(db.reducers.send_message, { "yuris", "hi" })).
A truthy err raises inside the coroutine instead of coming back as a value to
check. Catch it around the individual awaited step, not around the whole
coroutine.wrap body. Lua 5.1 cannot yield across pcall and HTML5 is plain
Lua 5.1, so a pcall wrapping the body fails the moment the first await
yields.
coroutine.wrap(function()
local ok, err = pcall(stdb.await, db.reducers.send_message, { "yuris", "hi" })
if not ok then print(err) end
end)()
await wires the handle the awaited call returned as well as the callback,
which is what keeps two of these calls from parking forever. A reducer call
whose socket dies mid-flight settles its Call and skips the plain callback, so
the news rides Call:on_error alone; a subscribe re-issued by the reconnect
replay is re-sent with a fresh no-op callback, so the awaited one is never
called again while Subscription:on_applied fires on every apply. Whichever
surface answers first wins and the rest are inert.
stdb.connect itself has no callback to wrap. It returns synchronously and
reports readiness on on_event, so stdb.await_event(db, "connected") covers
it instead.
local db = stdb.connect({ url = ..., schema = gen })
coroutine.wrap(function()
local evt = stdb.await_event(db, "connected")
local sub = stdb.await(db.subscribe_query, db, { "SELECT * FROM message" })
end)()
await_event waits for a future firing, so an event already delivered is gone.
connected is the exception, because "wait until connected, then resubscribe"
is the natural thing to write and reads this way after a reconnect has already
landed, which is precisely when the next connected may never come. It is
special-cased against the connection's own state and returns immediately when
the socket is already up.
This is additive. An existing opts.on_event handler keeps seeing every event exactly
as before, await_event only forwards to it first. The handle-based APIs,
Subscription:on_applied and Call:on_ok among them, are not wrapped as
awaitables, since they can fire more than once or split by outcome rather than
resolving once; reach for the handle there, coroutine or not. Full signatures on
stdb.await.
Per-table callbacks
on_event above is the firehose. Every connection, subscription and reducer
event, and every row change, goes through one handler. Rows have a second,
narrower route, one accessor per table in the schema.
db.tables.message:on_insert(function(ctx, row) end)
db.tables.message:on_update(function(ctx, old, new) end) -- primary-key tables only
db.tables.message:on_delete(function(ctx, row) end)
db.tables.message:count()
db.tables.message:iter() -- for row in ... do
db.tables.message:find("id", 7) -- by a single-column unique constraint
Every row callback's first argument is ctx, one object per connection,
built once and shared by every callback this SDK calls. Row callbacks here,
stdb.builder()'s lifecycle callbacks, and db:subscription_builder()'s
on_applied/on_error all get the same one (Connection
builder, Dropping a
subscription). ctx.db is db.tables under the
official SDK's own name for it, the same object rather than a copy, and
ctx.reducers/ctx.procedures are db.reducers/db.procedures
(Handles). ctx.event is the tagged cause of the batch this
callback is inside, { tag, ... }, with tag one of reducer,
transaction, subscribe_applied, unsubscribe_applied, error.
ctx.event.reducer is always nil for the reducer tag, since
ReducerResult carries no reducer name and no arguments, only the
transaction it caused, so there is nothing to fill it with. What the cause
does carry is timestamp and outcome ("ok", or "ok_empty" for the
committed-with-no-delta variant). A callback that needs the reducer's name
already has it, since it made the call. ctx:is_active() (not
closed), ctx:disconnect() (db:disconnect(), Reconnect) and
ctx:subscription_builder() (db:subscription_builder(), Dropping a
subscription) are the same three calls reached a
second way, off ctx, so a callback holding only ctx and no closed-over
db can still ask or act.
The two coexist. on_event is unchanged, still fires first, and still carries
every row change in evt.counts.events. Five rules the shape does not state:
-
Callbacks fire once the whole batch is in the cache, one call per row, in the order the server sent them. A callback reading
iter,countorfindsees a complete transaction, never half of one. -
Deregistration is not optional. A game object that registers in
init(self)capturesselfin the closure, and the first row after that object is deleted calls into a corpse. Keep the function and undo it infinal(self).function init(self) self.on_row = function(ctx, row) label.set_text("#label", row.text) end db.tables.message:on_insert(self.on_row) end function final(self) db.tables.message:off_insert(self.on_row) endoff_insert/off_update/off_deletemirrorSubscription:off_applied, the same idiom for undoing a registration, and the same rule of one registration removed, the first match, so a function registered twice needs two calls. -
Attaching or detaching during dispatch takes effect from the next row. The fire-once idiom, calling
off_inserton yourself from inside the callback, is therefore safe on a multi-row batch, and cannot skip the callback behind it. "Next row" means the next row of the batch, not of that table. Register onlogfrom amessagecallback and it is deliveredlog's rows of that same transaction, rows already cached before it existed. -
A callback that throws truncates the rest of the batch. Callbacks are not wrapped in
pcall, so the error propagates out ofconnection.dispatch, as one fromon_eventdoes. The cache is already complete andon_eventalready ran with the whole batch, but every row behind the throw is skipped, including every later table of the same batch, and nothing records which. Likewisedb:close()from inside a callback does not stop the batch, and the remaining rows are still delivered. Wrap the body, or guard on your own flag, if either matters. -
on_updateexists only for a table with a primary key, and raises on one without. A pk-less table's row change arrives as a delete plus an insert that nothing pairs, so such a registration could never fire; raising says so at the callsite instead of going quiet for days.
Frame handling is two calls, and the seam between them is public.
connection.handle_frame(c, bytes) decodes one BSATN frame and hands the result
to connection.dispatch(c, msg), which takes an already-decoded ServerMessage
table and runs everything downstream of the decoder over it, so request
correlation, handle settlement, cache application, event emission and the row
callbacks above. None of that half cares that the wire was binary, which is why
a raise from a row callback surfaces at dispatch and why the unit tests can
drive the whole server-to-client surface from a plain Lua table. An unknown
message ordinal is not a raise either. decode_server hands it back as
{ tag = "Unhandled", ordinal = n } and dispatch spends one unhandled event
on it, leaving the socket intact.
A reconnect empties the cache and no delete callbacks fire for the dropped
rows, because the replayed subscription re-delivers them as inserts. Rebuild derived
state from the subscribed / reconnected events or from
Subscription:on_applied, which re-fires for exactly this reason.
Accessors are namespaced under db.tables rather than sitting flat on db,
because table names belong to whoever wrote the server module and the connection
object already owns cache, send, close, identity and
whatever it grows next. 2.8.2 accepts a table called cache and serves rows from
it, and a module declaring one called tables, the namespace's own name,
publishes just as cleanly and serves its rows too, so neither clash is
hypothetical. A flat accessor would also have made the connection's private field
names public API. Under a 1.0 freeze, adding a field would then break any
consumer whose module has a table by that name. Under the namespace the
construction is total, with no reserved list and nothing refused. db.cache is
the cache, db.tables.cache is the table, and a table genuinely called tables
is db.tables.tables. It also matches the peer SDKs, which namespace the same
way, Godot's client.db.<table> and the official
ctx.db.<table>, where their client/ctx is our db.
db.tables, db.db and ctx.db are one object under three names, not copies,
so a row read or a callback registered through any of them is visible through
the others. db.tables is what this SDK's docs teach, db.db is the
official-SDK-shaped alias, and ctx.db is what a row callback receives.
The redundancy costs exactly one mistake, and it is the obvious one. ctx.db
already is the map, so ctx.db.tables reads like the way in and is not. It
raises, naming the hop and listing the real tables, rather than answering nil
and failing a line later at the typo's first use. So does any other name the
schema does not declare, as in stdb: no such table "mesage"; known: message, user.
With no schema passed at all the message names the option instead, pass schema = gen to stdb.connect, because an empty known-list means the section never
arrived rather than that the module declares nothing.
Dropping a subscription
When what a client watches changes at runtime, a player moving between regions
for instance, a query set is the answer, and it is the only form there is. Subscribe
names a list of query strings under a query_set_id, Unsubscribe drops one by
that id, and nothing on the wire replaces a whole subscription in place.
local sub = db:subscribe_query({ "SELECT * FROM message WHERE room = 1" })
-- ... later
sub:unsubscribe()
A query can also be built instead of hand-assembled. stdb.query compiles a
typed builder to the same SQL string, and subscribe_query accepts the two
forms mixed in one call.
local tables = stdb.query.tables(gen) -- gen: the generated module, same as connect's
db:subscribe_query({
tables.message:where(function(r) return r.room:eq(1) end),
"SELECT * FROM presence",
})
stdb.query is on the facade, so there is no separate require. r.col raises
for a column the table does not declare, naming the ones it does, and every
comparison value is checked against the column's declared type before a string
is ever built, so a typo or a type mismatch fails at the callsite instead of at
the server.
A hand-written string stays perfectly fine for a constant query and is shorter.
Reach for the builder when a filter value comes from the player, because it is
the only form that quotes that value for you. A name, a search term or a room id
typed into a text field is escaped on the way into the SQL, where concatenation
hands the server whatever the player wrote. Full operator, join and escaping
reference on stdb.query.
The server answers an unsubscribe with the now-unmatched rows as deletes, so
the cache drains itself, and the SDK asks for them explicitly, because the wire
makes that payload opt-in. Unsubscribe carries a flag, Default or
SendDroppedRows, and db:unsubscribe always sends the second. Asking for
Default would leave the dropped set's rows cached for the life of the
connection. The behaviours worth knowing, all captured from a live 2.8.2 server:
- A delta is per query, not per subscription. A
TransactionUpdatecarries one entry per query set and, inside it, one per table, and the SDK keeps each set's delta as its own entry rather than merging them, since flattening two would collapse a row's two independent holds into one. Queries are counted per pool, so two query sets whose text is identical share one query, the row arrives once, the first unsubscribe deletes nothing, and the last set to go deletes the rows (dev/test/fixtures/session.jsonlframes 14-17). Two sets with distinct text are two queries, and then a row both match arrives twice and leaves twice. So do not read a subscribe reply as "the full result set of my query". Upstream's own doc comment claims that and it is wrong; read the cache. - The cache counts holds, so a row survives one query dropping it. A change
making a row stop matching one of two queries sends a bare delete in that
query's entry beside a full delete+insert pair in the other's. A row is removed
only when the last query holding it lets go, and the frame reports one
updaterather than adeletebeside it, because adeletefor a rowcache.getstill answers for is a contradiction, not a notification. Without that count the row was silently evicted while the server still held it, recoverable only by resubscribing. - An unsubscribe reply's rows are optional on the wire.
UnsubscribeApplied.rowsis anOption<QueryRows>, populated only because the SDK asked. Absent, the reply is an empty update. The handle and the callback still settle, and the cache simply keeps rows nobody told it to release. - "Nothing came back" has two different shapes, and it is the state that picks one, not the message. A relation is absent, with no entry naming it at all, when the table itself holds no rows, and when a drop released nothing because a set with identical text still holds them. It is present and empty, an entry for the relation carrying zero rows, when the table does hold rows but this query matches none of them, which covers both a narrowed subscribe and dropping a query that had stopped matching. Half of this rule is about subscribing rather than dropping. It lives here because the other delta behaviours do, and a subscribe reply that came back empty is read the same way. So test the rows you decoded, never the presence of the relation.
Holds are counted per primary key, so only a table that declares one gets them. A
pk-less relation, every view included, stores one array entry per
arrival, so a row two queries match is cached twice. One query's delete removes
one copy and the other survives, but a unique index files only one row per value,
so cache.find stops reaching that survivor while cache.iter still does. Give
a relation you subscribe to twice a primary key.
Query sets are replayed after a reconnect under their original query_id, so
the handle you are holding keeps addressing the same subscription.
resubscribe_all empties the cache before it replays them.
Subscription builder
db:subscription_builder() is fluent sugar over subscribe_query above,
official-SDK-shaped.
local handle = db:subscription_builder()
:on_applied(function(ctx) end)
:on_error(function(ctx, err) end)
:subscribe({ "SELECT * FROM message WHERE room = 1" })
-- or, one `SELECT * FROM <table>` per relation the schema declares, sent as one batch:
db:subscription_builder():subscribe_to_all_tables()
It wraps the real Subscription subscribe_query already returns, rather
than replacing it. on_applied/on_error are wired onto that
subscription's own on_applied/on_error, so a reconnect replay still
re-fires on_applied exactly as it does above, and each gets ctx
(Per-table callbacks) in place of the bare msg
subscribe_query's own callback receives.
The handle it returns, SubHandle, answers is_active() and is_ended()
the same way Subscription does, but its unsubscribe(cb) raises on an
already-ended set instead of the no-op Subscription:unsubscribe above
tolerates. A caller who reaches for the builder is assumed not to be racing
its own drop against a rejection, so a second unsubscribe here is a
callsite bug rather than ordinary code. unsubscribe_then(cb) is the same
drop with a continuation, where cb(ctx) fires once it settles, and it raises up
front for the same already-ended reason unsubscribe does.
Handles
subscribe_query and call_reducer return a handle. The (err, ...) callback
argument still works exactly as it always did, since handles are additive, but a
handle keeps state you can ask about later, and splits outcomes the single
callback flattens.
local sub = db:subscribe_query({ "SELECT * FROM message" })
local id = sub.query_id -- the raw id, still accepted by db:unsubscribe
local skipped = sub.undescribed -- relations this apply dropped rows for, or nil
sub:on_applied(function(msg) end) -- fires again after a reconnect replay
sub:off_applied(fn) -- detach one on_applied registration
sub:on_error(function(err) end) -- the server rejected the query set
sub:is_active() -- applied and not dropped
sub:is_ended() -- unsubscribed, or rejected
sub:unsubscribe(cb) -- a no-op if already ended, but cb still fires
sub.state walks pending → sent → applied → ended | error, the same
machine the Rust SDK uses. Three transitions are worth knowing:
- A rejected set is dropped server-side and never replayed. Before handles
the only notice of that was the callback at the time; now
sub.stateis"error"andsub.errorholds the message, whenever you get round to asking. - A reconnect takes an applied handle back to
"sent"until the replayed set lands, because the cache was emptied in between.is_active()is false across that gap; the handle itself stays valid. endedanderrorare terminal. Unsubscribing while the subscribe is still in flight is ordinary code, and the server answers both frames, so the reply arriving behind your drop does not walk the handle back toapplied.
on_applied fires on every apply, including each reconnect replay, because
the replay empties the cache and a consumer holding derived state has to be told
again. That makes registration cumulative, so register once, outside your event
handler, or use off_applied to detach. on_applied returns the handle so
registrations chain, so it is off_applied and not a detacher token.
local call = db:call_reducer("send_message", { "yuris", "hello" })
local id = call.request_id
call:is_pending() -- the transaction has not landed yet
call:on_ok(function(update) end)
call:on_error(function(err) end) -- the reducer itself failed
db.reducers.<name>(args, cb) is sugar over the same call above, one
closure per reducer the generated schema declares, built when connect runs
rather than looked up per call, so an ordinary call touches no metatable.
local call = db.reducers.send_message({ "yuris", "hello" }, function(err) end)
Same handle, same callback shape, the same reducers = gen.reducers
requirement call_reducer already has. This is a name for it, not a second
mechanism. A bare call_reducer(name, ...) skips two checks. An unknown
name raises immediately, stdb: no such reducer "sned_message"; known: send_message, ..., sorted, comma-joined, every reducer the schema names,
rather than reaching the server and failing there. And the argument
count, not type, is checked against the reducer's declared parameters
before anything is sent, stdb: reducer "send_message" takes 2 argument(s) (sender, text), got 1. A wrong argument type is not caught here. It
still reaches the wire and fails however BSATN encoding fails on it.
db.procedures.<name>(args, cb) is the identical accessor over
call_procedure; see Procedures.
call.state is pending until the transaction lands, then ok | error, the
same field name as sub.state, because it is the same concept.
A reducer settles two ways. ReducerOutcome has four variants and they fold
into two, since Ok and OkEmpty commit while Err and InternalError fail. So
call.state reaches ok or error and nothing else, and there is no
on_out_of_energy on the handle. Energy exhaustion has no signal here. There
is no OutOfEnergy variant to set, and nothing else on the wire reports an
account running out, so it arrives as InternalError text like any other host
failure and settles the call as an ordinary error, indistinguishable from one.
Two things a settled reducer hands back are raw bytes, because the generated schema names no type to decode them against:
- A return value stays raw BSATN on
msg.result.ret_value. A generated reducer entry carriesparamsand noreturns, since only a procedure carries one, andcall_procedurestill hands you a decoded value. Thecall_reducercallback shape is unchanged,(err, msg), so the bytes ride along for a caller who knows their own reducer's return type and cost nothing to anyone who does not. - An
Errpayload is BSATN of the reducer's declared error type, which the generated schema does not name either. So the callback,call.errorandreducer_failed.errorget a printable summary,reducer failed (Err, 12 bytes), and the raw payload stays onmsg.result.valuefor you to decode against your own error type.InternalErroris genuine host text and passes through verbatim. Soerris always printable, and a declared error value is never stringified into it.
Registering a callback for a state that has already arrived fires it
immediately, including a cb handed to an unsubscribe that turns out to be a
no-op. Nothing orders your registration against the socket, and an applied frame
can land before you reach the next line, so the alternative would be a callback
that silently never runs.
When a request stops being answerable, its outcome reaches at least one surface
you already hold. That happens whenever a socket goes away for good, whether it
dies and the SDK reconnects, it dies with reconnect = false, or you call
db:close(). It is the one terminal event the SDK can see without a timer.
A Call in flight settles as state = "error", error = "connection lost",
which is what keeps is_pending() honest. The frozen (err, update) callback
is not invoked, since it never fired on a dead socket before. So a reducer's loss
is reachable, on the handle the same call already returned. Reachable is not
delivered, and if the callback is all you keep, you learn only that the
connection dropped, from a connection-level event that names no request. Keep
the Call if you need to know about your own.
call_procedure and one_off_query hand back no handle, so their callback is
the caller's only surface, and it does fire, with ("connection lost"). The
asymmetry with call_reducer is between mechanisms. A reducer's outcome is on
its Call, a procedure's was nowhere, which left a consumer that had disabled a
button until the procedure returned waiting forever. Handles for those two
were the alternative, and were rejected rather than freeze two more public types
into 1.0 to express what (err, ...) already says.
A query set still in flight is failed the same way, with sub:is_ended() and
on_error receiving "connection lost", but only when nothing will
re-issue it. On the reconnect path resubscribe_all replays the set under its
original query_id, so it stays in flight and on_applied fires again;
reporting a failure there would be the lie.
A callback that raises during this sweep does not cost the other callers theirs. Each is settled independently and the first error is re-raised once every one of them has been told.
One thing handles deliberately do not add is timeouts. A call whose socket is
still alive lives until it is answered. A timeout needs an owner in the update
loop and a defined interaction with reconnect replay; that is a behaviour
change, not a handle feature. It also belongs in connect options rather than
as a per-call argument, and adding it later is backwards-compatible.
What handles do not carry is the coroutine style, and that lives beside them
rather than on them. stdb.await wraps the (err, ...) callback and the handle
together, without changing either. See Coroutine mode.
One memory note, since Defold ships to mobile. sub.message retains the last
applied payload for as long as the set is live, and the connection holds the
handle, so it is retained whether or not you keep a reference. The cost scales
with row width, so a 4000-row set of 57-byte rows retains about 0.4 MB, and reaches
2 MB at around 470 bytes a row. It is re-stored rather than accumulated on each
replay, so it does not grow, but subscribe to what you need.
Procedures
A reducer mutates and tells you nothing; a procedure returns a value. Call
one by its canonical name and the callback receives the value decoded, which
needs procedures = gen.procedures passed to connect, because the decode runs
against the returns type that generated entry carries.
Without it the raw BSATN bytes go through instead, which is still usable by a
caller who knows the procedure.
db:call_procedure("echo", { "from defold" }, function(err, value)
if err then return end
print(value) -- echo: from defold
end)
db.procedures.echo({ "from defold" }, function(err, value) end) is the
same accessor sugar db.reducers.<name> is, so one closure per procedure the
schema declares, the same unknown-name raise and argument-count check, the
same callback shape. See Handles for the full contract, since nothing
about it is procedure-specific beyond the name it reads.
Two things to know, both captured from a live server:
- Failures are all
InternalError. A procedure that throws and a name that does not exist produce the same status, differing only in message, and there is no separate "no such procedure" variant to branch on.ProcedureStatushas exactly two variants,ReturnedandInternalError, so there is nothing else the reply could be. - Replies arrive out of request order. Five calls sent as 1, 2, 3, 4, 5 came
back 1, 4, 5, 2, 3 in one run and 1, 4, 5, 3, 2 in another, so not even the
permutation is stable. A failure is answered without a transaction at all,
with
timestamp: 0and a zero execution duration, so nothing holds it behind the calls that do real work, and it can land ahead of one sent before it. Correlating onrequest_idis therefore not an optimisation, it is the only thing that pairs a reply with its call. The SDK does that for you; a consumer reading raw frames has to.
Reconnect
connect reconnects on its own. When the socket drops it retries with
exponential backoff (1s doubling to 30s by default), reuses the
server-issued token so the identity survives, then clears the cache and
replays every live subscription, because stale rows must not outlive the gap.
Each delay carries a deterministic per-client spread of 50 to 99.95% of the slot,
keyed on the identity rather than an RNG, so a server restart does not bring
the whole player base back in one instant. The spread needs an identity, and
before the first socket succeeds connect seeds it from the persisted one, so
a fleet relaunching into an outage scatters on its saved identities instead of
sharing one ladder. That saved identity is not server-confirmed at that point;
a stale one still lands the client on a deterministic slot of its own, which is
all the spread asks of it. Only a genuinely fresh client, nothing persisted
yet, or storage = false, retries on the exact 1s/2s/4s ladder, and one new
client is not a herd.
stdb.connect({
...,
reconnect = true, -- default; false to manage it yourself
backoff_base = 1, -- seconds
backoff_cap = 30,
})
db:close() -- stop reconnecting and close the socket
db:disconnect() is the same function under the official SDK's spelling,
one function under two names, so a caller of either gets identical teardown.
db:state() reports where the connection is right now, for a UI that wants
to draw a "reconnecting…" banner instead of inferring it from the event
stream. It walks "connecting" → "connected" → ("reconnecting" while a retry is
scheduled, back to "connected" once it lands) → "closed" after
db:close(). db:is_connected() is db:state() == "connected".
"closed" is terminal. Nothing moves the state off it, and a frame that
arrives afterwards is discarded rather than handled, so a UI that tears down
on "closed" does not have to defend against a later connected event. That
matters on HTML5, where closing during the handshake does not actually close
the socket and the server's frames keep arriving.
db:latency() is a smoothed round-trip figure, updated by every correlated
request/response this SDK already exchanges, a reducer call, a subscribe or
a one-off query, so an idle connection reads whatever the last real
exchange measured. db:ping(cb) forces a fresh one on demand with the
cheapest possible round trip. Its callback is (err, seconds), the same shape
every other async entry point here uses, so stdb.await(db.ping, db) returns
the latency instead of raising it. Neither starts a timer or a polling loop;
call ping on your own schedule if you want a live readout.
on_event delivers these events.
evt.type |
when |
|---|---|
connected |
InitialConnection received; evt.identity, evt.token |
reconnected |
new socket up; evt.identity, and evt.subscriptions for how many sets were replayed |
reconnecting |
retry scheduled; evt.attempt, evt.delay |
disconnected |
socket closed; evt.code, the close code. There is no reason key |
socket_error |
the socket reported a transport error; evt.error. A reconnect is scheduled, and a socket that reports both this and disconnected still only retries once |
subscribed |
a SubscribeApplied was applied to the cache; evt.request_id, evt.query_id, evt.counts |
unsubscribed |
a query set was dropped; evt.request_id, evt.query_id, evt.counts |
subscription_failed |
query rejected and the query set dropped; evt.error, evt.request_id, evt.query_id |
transaction |
rows changed; evt.counts.events lists the row changes, evt.message is the frame |
event |
one row of an event table; evt.table, evt.row. Never cached. See Event tables |
reducer_failed |
reducer returned an error; evt.error, evt.message |
procedure_result |
a ProcedureResult returned; evt.value, evt.request_id, evt.message |
procedure_failed |
a ProcedureResult failed; evt.error, evt.request_id, evt.message |
undescribed |
rows arrived for a relation your schema does not describe, and nothing was cached; evt.table, evt.dropped (a count of the rows that went unstored), evt.reason. See Relations your schema does not describe |
unhandled |
a frame whose ServerMessage ordinal this SDK does not know; evt.tag, and the ordinal on evt.message.ordinal |
one_off_result / one_off_failed |
a one-off query was answered; evt.request_id, evt.tables, evt.error, and evt.matched. See One-off query rows |
A reducer's result and the transaction it caused arrive together, in one
frame, because ReducerOutcome::Ok carries the transaction inline. What the
wire separates is the caller's own outcome from everybody else's broadcast, and
that is where the bytes are saved. So a transaction event carries neither
reducer_call nor caller_identity, since a ReducerResult names no reducer and
echoes no arguments, and the pairing comes from your own call_reducer
callback, which fires for the same frame. The SDK's own conformance smoke needed
a shim for precisely this, so budget for one if you need to attribute a broadcast
to its caller.
The correlated replies, procedure_* and one_off_*, are emitted whether or
not they matched a request you sent, because an answer nobody is waiting for is
the loudest symptom of a misrouted id. evt.matched is false on a one-off
response that found no caller; the callback you passed to one_off_query is not
invoked in that case and your query stays in flight.
Each entry in evt.counts.events is { type, table, row }.
e.type |
when |
|---|---|
insert |
a row appeared |
delete |
a row went away |
update |
one batch changed a row in place, so e.row is the new row and e.old the previous one. Only tables declared with a pk produce this, because without a key there is no way to tell a change from unrelated churn, so pk-less tables always report a delete and an insert. A delete and an insert of the same key always pair, so a row re-sent unchanged still reports as an update, and on the subscribe/unsubscribe path that is query churn rather than an edit. Pairing is by key, so a change to the key's own value is deliberately not an update, because the delete and the insert carry different keys, nothing pairs, and both halves surface as a delete and an insert. Pairing needs the key to be a comparable scalar, so a table keyed by an Identity pairs only on a generated schema; keyed by a u64 above 2^53 it pairs the wrong rows. Both in Row value types. |
counts.inserts and counts.deletes count row operations, so a single update
still adds one to each, and a delete that only released one of several holds is
counted too, even though it emitted no event and removed no row.
Entries follow wire order. An unpaired delete keeps its position ahead of the
inserts; a synthesized update appears at its insert's position.
Pairing is scoped to one updates entry, because that is how the server sends
it. A live 2.8.2 server puts both halves of a pk change, the delete of the old
row and the insert of the new one, inside the same entry. That is what makes
the update event possible at all; split across two entries the pairing could
never fire. It is also why pairing stops at the entry boundary. The same row can
appear in the deltas of several unrelated query sets, and joining across them
would fabricate an update out of two independent subscriptions.
The consequence to know is that a row can report more than one update for a
single change. The SDK keeps one entry per distinct query the row matches, and
two query sets carrying the same query text share one query and so produce one
entry between them, not one each. That is exactly how insert and delete
already behave, so a consumer that was already idempotent needs no change. The
frame behind this claim is captured, not assumed.
dev/test/fixtures/session.jsonl frame 24 is
edit_message against a real server, and the headless conformance run repeats
the check end to end through the engine's own socket.
The conformance test proves the reconnect path against a real server restart. It kills the server after the first round trip and asserts the row comes back.
One-off query rows
A one-off's rows arrive encoded. evt.tables[i].rows is an array of byte
strings, because a one-off projects an arbitrary relation your schema need not describe
at all, and BSATN with no types cannot be decoded. Correlation is on
evt.request_id, the id the SDK allocated, because the server echoes no id of
its own. table_name is spelled as you would expect. Decoding is yours, because
only you know the shape you selected.
local pv2 = require("stdb.protocol_v2")
local row = pv2.decode_typed_bsatn(t.rows[1], db.cache.schema[t.table_name], db.cache.typespace)
That is right only for SELECT *. A def is positional, so decoding a
projection against the table's def reads the wrong columns, and it need not
raise. SELECT sender, text FROM message is caught, because the string's length
prefix gets read as the u64 and the widths stop lining up. But SELECT id, text, sender has the same three widths with two swapped, so it decodes clean and
hands back sender and text reversed, silently. Pass a def describing the
projection in its SELECT order, or select whole rows. This is why the SDK never
decodes these for you, and both cases are pinned in
dev/test/test_protocol_v2.lua.
Connection builder
stdb.builder() is fluent sugar over the stdb.connect(opts) call at the
top of this page, official-SDK-shaped.
local gen = require("my_schema")
local db = stdb.builder()
:with_url("ws://127.0.0.1:3000")
:with_database("chat")
:with_token(saved_token) -- optional
:with_schema(gen) -- the whole generated module
:on_connect(function(ctx, identity, token) end)
:on_connect_error(function(ctx, err) end)
:on_disconnect(function(ctx, err) end)
:build()
with_schema(gen) hands the whole generated module to connect, which reads
typespace, reducers and procedures off it, so one call covers what the
four-option form at the top of this page sets by hand. build() is a thin
translation into connect. It duplicates none of connect's validation or
logic and calls it exactly once, so nothing assembled this way can diverge from
a hand-built options table. Skipping with_url or with_database raises at
build() naming whichever is missing, rather than three frames later inside
websocket.connect with a message about a URL nobody here wrote.
Everything the fluent setters do not cover is reached through a seed table,
stdb.builder(opts), so the builder is never a reason to fall back to raw
connect and lose the three lifecycle callbacks.
local db = stdb.builder({ frame_budget = 0.004, confirmed_reads = false })
:with_url("ws://127.0.0.1:3000")
:with_database("chat")
:with_schema(gen)
:on_connect(function(ctx) end)
:build()
The seed is copied, so with_token/with_schema write into the builder's own
table rather than into yours, and build() passes it through verbatim. An
on_event in the seed keeps firing first, ahead of the lifecycle callbacks
below.
The three lifecycle callbacks are read off the on_event stream this SDK
already emits, since there is no separate wire signal for any of them.
on_connect fires on connected and reconnected (above); on_connect_error
fires on a socket_error seen before the first successful connect;
on_disconnect fires on a socket_error or disconnected seen after one.
That is the builder's own derivation from the events documented above, not a
new fact the wire states, and it mirrors the official SDKs' own split
between a connection that never came up and one that dropped after it did.
Every callback receives the same ctx row callbacks do
(Per-table callbacks).
Event tables
Event tables are transient rows, such as damage numbers, hit indicators and sound
cues, broadcast to subscribers and never stored. The wire keeps them apart from
the start. A table's rows in a transaction arrive as either a PersistentTable
delta, inserts and deletes, or an EventTable batch, and only the first reaches
the cache. The second arrives as its own on_event type.
on_event = function(evt)
if evt.type == "event" then
-- evt.table is the relation's name, evt.row the decoded row
spawn_damage_number(evt.row.amount, evt.row.at)
end
end
One event per row, and the row is never cached. It is not in cache.iter,
not counted by cache.count, and produces no insert. That is the point of
them. Rows still arrive fully decoded against the generated types, and the event
is emitted after the rest of the transaction is in the cache, so a handler
reading iter or find sees the complete transaction the event belongs to.
Proven end to end against a live server, not only in unit tests. The dev module
carries a ping event table (note and n) and an emit_ping reducer, and the
conformance smoke asserts the event callback fires with decoded fields and
that cache.iter holds nothing for that table, natively and in the browser.
Identity persistence
SpacetimeDB issues an identity and token on first connect. The SDK saves them
with sys.save and reuses them on the next launch, so a player keeps the same
identity across app restarts, with no login and no account system.
stdb.connect({ ..., storage = false }) -- opt out
stdb.connect({ ..., storage = stdb.sys_storage("my-slot") }) -- custom slot
stdb.connect({ ..., token = my_token }) -- supply your own
storage is an injectable { load, save } seam, so tests substitute a plain
table and never touch the engine. Conformance proves the real thing on both
targets. The native harness runs the headless client twice and the HTML5 harness
loads the page twice, the second time with no token supplied, and both fail
unless the two runs report the same identity.
HTML5 persists too, with caveats worth knowing. The mechanism is in the engine
source at Defold 1.13.1 (574678c). sys.save on the Emscripten target is a
plain fopen/fwrite/fclose and calls no sync itself
(engine/script/src/script_sys.cpp:202-218). The durability comes from the HTML5
loader, com.dynamo.cr/com.dynamo.cr.bob/src/com/dynamo/bob/bundle/resources/web/dmloader.js,
which mounts the save root /data (engine/dlib/src/dlib/js/library_sys.js:7) as
IDBFS and patches FS.close so every close starts an asynchronous
FS.syncfs(false) MEM→IndexedDB (mount and patch at :1204-1212, the sync itself
at :1279). At boot the reverse sync runs before main() (:1221), which is what
restores a prior save.
A syncfs is mount-wide. :1279 passes no path, and IDBFS diffs the whole
local set against the whole remote set by mtime and writes every difference. That
matters more than it sounds. A save whose flush fails is still sitting in the
in-memory layer with a differing mtime, so the next flush that succeeds writes
it, and any FS.close anywhere in the app starts one. Single failures are
self-healing.
Concurrent syncs are coalesced. One already in flight sets _syncNeeded, and the
in-flight sync re-enters when it finishes (:1289-1291). Note that re-entry lives
inside the FS.syncfs callback, so a coalesced sync is dropped rather than
deferred if the flush it was queued behind is the one that trips the breaker below.
On the happy path, then, every sys.save starts a flush. There are two things it
does not protect you from.
Once three flushes fail in a row, persistence is dead for the lifetime of the
page. The error path (:1282-1284) does a console.info and increments
_syncTries without retrying, and only the boot direction retries (:1132-1135).
The permanence is not the counter. :1297 resets _syncTries to 0, so on the
counter alone the breaker would skip one flush and resume. It is the
_syncInProgress flag set at :1276, which the tripped branch never clears
because the only line that clears it (:1280) is inside the FS.syncfs callback
that branch never issues. From then on persistentSync can only ever set
_syncNeeded and never starts another sync. Nothing is raised to Lua.
The commit is asynchronous. sys.save returns before IndexedDB commits, and
the loader installs no beforeunload or exit safety net. A reload with no token
recovers the same identity, and the check fails loudly when persistence is
disabled, both verified. What that does not prove is a tab closed in the
instant between the save and the commit. If you have a shutdown path that reloads
or navigates straight after a save, another fire-and-forget syncfs buys you
nothing, because you need the reload inside the callback. That callback fires once the
sync settles, from the transaction's oncomplete on success or its
onerror/onabort on failure, so treat it as an ordering guarantee rather than a
durability one. The snippet below reloads either way, and inspects err if you
want to distinguish them.
if stdb.is_html5() then
-- `err` is non-null if the flush failed; the reload happens either way, so log
-- it rather than discarding it.
html5.run("FS.syncfs(false, function(err){ if (err) console.warn(err); location.reload() })")
else
my_reload() -- native sys.save needs nothing extra; it is durable when it returns
end
The is_html5() guard is not optional. On native targets InitializeHtml5 is an
empty stub (engine/script/src/script_html5.cpp:23-25), so nothing registers
run and the unguarded call raises attempt to call field 'run' (a nil value).
html5.run evals in the
engine's own JS scope, which is why FS is reachable at all
(engine/script/src/script_html5_js.cpp:95-106). On success it returns the eval's
stringified result, but a JS exception is re-raised as a Lua error rather than
returned (:112-117 calls luaL_error), so wrap it in pcall if you adapt it.
The return value is not a success signal either. Both snippets here evaluate to
"undefined" and "false" respectively when they work.
This snippet does bypass the loader's _syncInProgress guard and can overlap the
flush sys.save just triggered. The cost is a duplicated mount-wide pass, not a
lost write, since both passes diff against current state inside a transaction the browser
serialises against the other. Do not "fix" that by polling _syncInProgress
until it clears, because the wedge above is exactly the state where that poll never
terminates and your reload never happens. A bounded wait is the safe form of that
idea.
The SDK deliberately issues no flush of its own, and the reason is not that it
would be redundant, because above the happy path it would not be. It is that there is no
durability signal available to Lua at any price the SDK can pay. sys.save returns
a boolean about the in-memory write; FS.syncfs's only outcome channel is a JS
callback; html5.run is a synchronous eval returning a string. Reporting durability
would mean an async JS→Lua bridge and a new public function, to tell you something
whose only remedy is the snippet above. The consumer-side line delivers the whole
benefit with none of the surface.
If you have already seen the loader's retry-count warning in the console, the wedge has a one-line escape hatch. It reaches into loader-private state, so it is a last resort rather than something to run routinely.
if stdb.is_html5() then html5.run("Module._syncInProgress = false") end
That re-arms the breaker but flushes nothing by itself. _syncTries was already
reset when the breaker tripped, so clearing the flag only lets the next
persistentSync through. Follow it with a sys.save (reconnecting is enough, since
the SDK saves the token) to actually get the pending data written.
Finally, one environment has no durability at all. Where window.indexedDB and its
vendor-prefixed forms are all absent, in a non-browser or heavily locked-down
context, not ordinary private browsing, which does expose the API,
Module.persistentStorage is false from the gate at :1303. The loader still
attempts the mount, since it does not consult the flag first, but persistentSync
(:1167) and the boot-time preSync (:1125) both early-return on it, so
sys.save writes only the in-memory layer. Private modes bite differently. They
expose indexedDB but can fail open(), which surfaces as the failing-flush path
and the breaker above. Nothing in the SDK can change either, and no error is
raised. The identity simply will not survive a reload.
Authenticating with a provider
connect on its own gives every player an anonymous identity, fine for a demo,
useless the moment a player has an account. stdb.auth.exchange trades a
credential the player already has (a Steam session ticket, a Google or Apple ID
token, your own session cookie) for a SpacetimeDB token you hand to connect.
stdb.auth.exchange({
url = "https://api.example.com/auth/spacetimedb",
provider = "steam", -- your name for it; this is the cache key
credential = steam_session_ticket, -- the credential you are trading in
storage = stdb.sys_storage("auth"), -- caching is opt-in; see below
}, function(err, token)
if err then
print(err) -- names a status, never the credential
return
end
stdb.connect({
url = stdb.url("wss://example.com", "chat"),
token = token,
})
end)
The endpoint is yours, not the database's. That is not a preference, it is
what 2.8.2 leaves you. /v1/identity/token, /v1/token, /v1/auth and
/v1/identity/exchange all answer 404; the only identity endpoint that exists,
POST /v1/identity, mints an anonymous one. Pointing the server at a
third-party OIDC issuer is not on offer either, since spacetimedb-standalone start
exposes a single --jwt-pub-key-path/--jwt-priv-key-path keypair, with no
issuer list and no JWKS URL. So something you control has to verify the provider
credential and obtain the identity. url is only a URL, so if a later server
version grows an endpoint, point it there and nothing else changes.
The credential travels in the request body, as {"token":"<credential>"},
with Content-Type: application/json and no other header. Note the two names.
The option is credential, the body field is token. They differ because
token means a SpacetimeDB token everywhere else here, including what this call
hands back, while the body shape is the one your own endpoint reads, so it stays
as it is. It is deliberately not an Authorization header, either. The endpoint
is not the credential's resource server, and a header invites a proxy to forward
it as if it were.
Retries are the part worth knowing. A transport failure or a 5xx is retried,
three more attempts, 0.5s doubling, and a 4xx is not. A rejected credential
will not be accepted on the second ask, so retrying one would only make the game
wait 3.5 seconds before telling the player they are not signed in. The timeout is
per attempt, not for the whole exchange, and a timed-out attempt still goes on
to the next retry.
| Option | Default | Meaning |
|---|---|---|
url |
none | Required; your exchange endpoint |
provider |
none | Required; the cache key, a name you choose ("steam", "google"). Empty string is rejected |
credential |
none | Required; the provider credential. The request body carries it as token |
json |
the engine's json global |
Only needed off-engine, where there is no such global to default to. This call and stdb.schema.fetch are the two that speak HTTP and therefore the two that use it at all; connect does not take it |
http |
stdb.http_seam() |
Injectable { post } seam over http.request |
timer |
the timer global |
Injectable; needs only delay(seconds, repeating, fn) |
storage |
nil |
{ load, save } seam. nil means no caching at all |
timeout |
10 |
Seconds, per attempt |
retries |
3 |
Additional attempts after the first |
backoff_base |
0.5 |
Seconds between retries, doubling |
A missing url, provider or credential raises rather than reaching the
callback, and so does a json that is neither passed nor ambient. Those are
programming errors, and the SDK raises for
those everywhere else too. Everything a server or a network can do reaches cb
as one of four strings.
err |
When |
|---|---|
stdb.auth: exchange rejected with status N |
A 4xx, immediately; or a 5xx that outlived every retry |
stdb.auth: transport failure after N attempts |
The last attempt got no answer at all; earlier ones may have failed differently |
stdb.auth: exchange timed out after N attempts |
The last attempt hit timeout; earlier ones may have failed differently |
stdb.auth: response had no token |
The body carried no token string, or would not decode at all. Both are the same thing to you, and a malformed body never escapes as a Lua error |
Notice what is absent. The credential and the exchanged token appear in no error
string and no log line, and a response body is never interpolated into a message.
dev/test/test_auth.lua collects every string the module produces, including
callback errors, raised errors and prints, and fails if either secret is in one.
A leak here would be invisible until it turned up in someone's crash report.
Nothing is decoded out of the token, either. There is no JWT parsing in this SDK
and no base64url decoder. The cache key is provider, a name you supply,
because you already know which provider you used. An unverified claim read out of
a token is not an authorization decision, and this SDK will not be the thing that
mistakes one for it.
The cache is opt-in, and its token is not validated
storage defaults to nil, so there is no caching and nothing written to disk.
That is the opposite of connect, whose storage defaults to
stdb.sys_storage(), and the difference is deliberate. An identity token is the
SDK's own to persist; a provider-exchanged token is your credential, and the SDK
will not silently write it to a save file. Pass a { load, save } seam, the same
shape identity persistence uses, and it caches under the
provider name, merged rather than overwritten, so two providers coexist without
evicting each other. A cache hit makes zero requests.
The whole record nests under one auth key, as { auth = { steam = "…" } }, so
it cannot collide with connect's { identity, token } record, which reaches
the same kind of file through the same stdb.sys_storage(name) seam. Flat, only
the slot name kept the two apart, and provider is a name you choose, so no
reserved-name list could have fixed it. A provider = "token" would have read
connect's identity token straight back as an exchange result, having made no
request at all. Exchanging now leaves any connect record in the slot untouched,
and connect returns the favour, since its identity save merges into the slot
rather than rewriting it, so the two coexist in a shared slot in both directions.
Giving the exchange its own slot name, as the example above does, is still the
tidier arrangement, and it is just no longer load-bearing.
What the cache does not do is check the token is still good. Verifying one
needs a round trip, and there is no expiry parsing here, which would need exactly
the JWT decode above. So a hit hands back whatever string was stored, however
old, and a stale token surfaces as a failure at connect, not here. The one
thing it does check is the type. A well-formed store whose entry is not a string,
a tampered value, or a seam of your own returning something odd, is ignored and
the exchange proceeds, rather than handing a number, or a function, on to
connect. A malformed save file is a different matter. Defold's sys.load
raises on a truncated or version-mismatched file, and that raise comes straight
back out of auth.exchange rather than being swallowed, exactly as it does for
connect's identity file.
The cache does do one thing silently. A save that raises is swallowed, because
the token is already in hand and losing it to a failed cache write would be the
worse outcome. So a save directory that is read-only or full costs you nothing
visible, with no error and no log, except that the next launch exchanges again.
Clearing the cache is your move. If connect reports the token was rejected,
drop the entry and exchange again. Nothing in the SDK does it for you, and nothing
in the SDK can tell you the token went bad before the server does.
Generating the schema
Column order is not cosmetic, and on a binary wire it is the whole of how a row
is read. A BSATN row is its field values concatenated, with no names, no count and
no delimiters, so position is the only thing that identifies a field. Two orders
carry that weight. columns[i] and types[i] are parallel arrays, so a reorder
pairs every column after it with the wrong tag, and because the tag also fixes
the field's width, a mispaired tag does not stop at one column, it walks the
cursor off and misreads the rest of the row. The typespace orders do the same
one level down. A sum_ value arrives as a bare ordinal byte and the variant
list is what names it, and a product_ value is its fields in declaration
order. Both are read by protocol.decode_typed_bsatn, the decoder every table
carrying types goes through (see Row value types).
Hand-maintaining that is not maintainable at all. A table whose entry has no
types is skipped and reported rather than decoded, so a hand-written schema
gets you nothing. Generate it instead, since the generator also covers
views, which nothing else can describe.
Two routes produce the module, and both walk the same
stdb.typetags transform,
so a fetched schema and a written one cannot disagree. In the editor,
Project ▸ SpacetimeDB: Regenerate schema writes /my_schema.lua at your
project root; it needs database under [spacetimedb] in game.project, and
raises naming that key when it is absent rather than guessing a module name. In
Lua, to_lua turns a fetched schema into the same text.
stdb.schema.fetch({ base_url = "http://127.0.0.1:3000", database = "chat" }, function(err, gen)
if err then print(err) return end
print(stdb.schema.to_lua(gen, "http://127.0.0.1:3000/v1/database/chat"))
end)
local generated = require("my_schema")
local db = stdb.connect({
...,
-- Whole module: the tables, the typespace, and the argument types a reducer
-- or procedure call encodes itself against.
schema = generated,
})
db:call_reducer(next(generated.reducers), { ... })
Per table it emits the columns in declaration order, a type tag per column, the
primary key, the columns under a unique constraint and the indexes; per
view, the columns and types plus view = true; per reducer and
procedure, the parameter names, their type tags and, for procedures only, the type
of the return value. Reducer and procedure names are the canonical wire ones, so
a module exporting sendMessage appears here as send_message.
["message"] = {
columns = { "id", "sender", "text" },
types = { "u64", "string", "string" },
pk = "id",
unique = { "id" },
indexes = {
{ name = "message_id_idx_btree", columns = { "id" }, unique = true },
},
},
Type tags are flat strings, so decoding a column costs one lookup. The tags are
bool, i8..i256, u8..u256, f32, f64, string, opt_<inner> for an
Option, vec_<inner> for an array, identity, connection_id, timestamp,
time_duration, uuid, schedule_at, unit, result, sum_<Name> /
product_<Name> for a type the module declared, and unknown when, and only
when, nothing matched. Nothing a 2.8.2 JS module can publish reaches it, because a
compound type needs a name (an anonymous one is rejected at publish, an empty one
by the identifier check), so every sum and product arrives either named or as one
of the built-in shapes above. result reports the shape only (ordinal 0 is ok, 1
is err), not the two payload types, because a flat tag carries one inner type.
reducers and procedures are not optional if you call anything. A BSATN
argument tuple has to be encoded against the parameter's declared type, so
without them call_reducer and call_procedure have nothing to encode against.
schema = gen carries both; passing the sections by hand works too. A call made
without them raises at the callsite and names the option, stdb: no reducers are known, so "send_message" cannot be called - pass reducers = gen.reducers to stdb.connect (or schema = gen), because an empty known-list means the section
never arrived rather than that the module declares nothing. A reducer whose
arguments you have already encoded yourself can be called with an args byte
string, which is passed through.
Argument encoding goes all the way up. The wide types encode as well as decode,
so identity, connection_id, uuid, i128/u128 and i256/u256 each go
out in exactly the form the decoder hands back, and a value read out of a row can
be passed straight into a reducer call. An identity is "0x" and 64 hex digits,
case-insensitive, prefix required; a connection_id, uuid or u128 is an
exact decimal digit string, and i128 the same with an optional leading -; a
u256 is "0x" and up to 64 hex digits, minimal width, no padding needed, and
i256 the same with an optional - before the 0x. A malformed string raises
rather than encoding something else, whether the prefix is wrong, a character is
non-hex, there are too many digits, or the value does not fit the width. A plain
Lua number is rejected for the 128- and 256-bit widths, because a double cannot
carry them exactly, and shipping a silently rounded value is worse than raising.
The special types are recognised structurally, by the magic field names sats
hides inside them, never by column name. A column called timestamp that holds
a u64 is tagged u64, and a column called x that holds a Timestamp is
tagged timestamp.
That holds in the other direction too, which is the direction that would
silently corrupt a decode. A module type whose shape collides with a built-in
keeps its own name. enum Perhaps { Some, None } is tagged sum_Perhaps, not
opt_*, even though the default case-conversion policy canonicalises its
variants to the some/none that Option uses. The two are distinguishable
because a genuine Option or special type is never a typespace entry, since it
arrives inlined, while a module type is reached by name.
That covers the special types as well as Option. A module product named
FakeIdentity whose single field is the magic __identity__ is tagged
product_FakeIdentity after its own name, not identity.
A sum_<Name> or product_<Name> tag is a name, not a decoding, so a
typespace section declares what it names.
typespace = {
["Point"] = { kind = "product", fields = { "x", "y" }, types = { "i32", "i32" } },
["Status"] = { kind = "sum", variants = { "idle", "busy" }, types = { "unit", "unit" } },
},
Variants are in ordinal order because that is how values arrive. A sum on
the wire is a single ordinal byte followed by that variant's payload, and
nothing else, so Status::Idle above is the one byte 00, since a unit payload
adds no bytes at all. There is no variant name anywhere in the frame, so the
declared list is the only thing that turns that byte back into "idle", and
inserting a variant anywhere but the end retags every value after it.
The same holds at every depth, which is why typespace field order is as
load-bearing as columns.
| shape | on the wire |
|---|---|
module sum (sum_Status) |
ordinal u8, then the variant's payload |
module product (product_Point) |
its fields concatenated in declaration order, nothing else |
Option |
00 and the value, or 01 and nothing |
array (vec_<T>) |
a u32 count, then each element |
string, vec_u8 |
a u32 byte length, then the bytes |
identity |
32 raw little-endian bytes, rendered 0x… big-endian on the way out |
Nothing there is keyed by a name, and nothing carries a length that would let a
decoder resynchronise after a wrong guess. One misread width shifts every field
behind it. Each tag's layout is pinned per type in
dev/test/test_protocol_v2.lua.
Compound uniqueness lives only on indexes[].unique, while unique holds plain
column names, for the single-column lookups a client can actually use.
dev/test/fixtures/chat_schema.lua is that generated file, committed, and the
fixture test reads it rather than a hand-typed copy. Two checks hold it to the
module, and they catch different halves of the same mistake.
dev/tools/conformance.sh re-derives the schema from the module it has just
published, normalises it through stylua, and diffs the result against the
committed copy, ignoring only line 1's source URL (whose port follows
STDB_PORT). Reorder a column in
dev/server/index.js and that diff fails, printing the one command that refreshes
the fixture. It is also the only thing that holds the tool to account rather
than just its output. A generator resolving the wrong typespace entry, emitting a
view with its underlying table's column order, the exact mis-mapping views exist
to avoid, passes the whole Lua suite, because the suite only ever reads the
committed file.
Refresh the fixture and the suite fails in turn, naming the frames that depend on the order.
FAIL generated column order matches the module, which decodes the TransactionUpdate rows in frames 4/5/24/48: expected id,sender,text, got id,text,sender
Views
A view is a relation like a table. It arrives under its own table_name,
routes through the same cache path, and the generator emits it into tables
alongside the real ones, so stdb.schema.new(gen.tables) covers it with no extra
wiring.
["message_flip"] = {
columns = { "text", "sender", "id" },
types = { "string", "string", "u64" },
view = true,
},
One thing about it is unlike a table, and it is captured rather than inferred
(dev/test/fixtures/session.jsonl frames 37-42, a view over message
projecting it reversed). The column order is the view's own, not the
underlying table's. message is (id, sender, text); the view above returns
(text, sender, id), and a BSATN row is nothing but its fields in order, so
handing the cache the table's column list maps every field to the wrong name,
with id taking the text and text taking the u64, which is worse than mapping
nothing, and it is why a view cannot be described by reusing the table it
projects. A table has the same exposure, since every row of every relation is
positional here; what makes a view the case that bites is that its order is one
nothing else in your project states.
A generated view has no primary key, so it is cached as an append-only list. A
changed projection is a delete and an insert, never one update, and
on_update raises rather than silently never firing. That is not an omission the
generator could fix. A view's module source may declare one primary-key column,
and it does not reach /v1/database/<db>/schema at all, which describes a view
with its name, params, visibility, anonymity, index and return type, and no key of
any kind. view = true exists so that raise can say "view" instead of sending you
looking for an annotation that cannot help.
A hand-written schema may still give a view = true entry a pk, and it is
honoured exactly as a table's, so the relation is keyed, paired, with on_update
firing. That is the view whose projection includes the key of the table it reads,
and the flag says where the relation came from, not what the cache may do with it.
Both return forms spacetimedb.view accepts are emitted. A view returning an
Option rather than an array, the form that yields at most one row, declares
its row product through the some variant of a two-variant sum, and that is a
product like any other, so its column order is just as recoverable. On a live
2.8.2 such a view delivers a single positional row such as ["second",2], in the
order the view declared, decoded the same way. Which row that is belongs to the
module and the server, not to this SDK. The view here returns
rows[rows.length-1], and nothing in the protocol promises which row iter()
yields last. The none case carries no row, and carries no entry for the
relation either, which is not the same shape as an empty one; see Dropping a
subscription.
Relations your schema does not describe
A schema may describe as few relations as you like. Describing only the tables you use is normal, and the headless conformance run does exactly that. The cost is stated once, here. Rows of a relation your schema does not describe are not cached. They cannot be. The declared column order is the only thing that names the fields of a positional row, so with no definition there is nothing to decode against.
What happens instead is that the relation is skipped whole, with no row stored,
no insert counted and no row callback fired, and reported.
on_event = function(evt)
if evt.type == "undescribed" then
print(evt.table, evt.dropped, evt.reason) -- message_flip 1 no schema entry
end
end
The same report is on evt.counts.undescribed, as
{ { table = name, dropped = n, reason = why } }, and the field is absent when
nothing was skipped, so a consumer subscribed only to relations it declared sees
an unchanged counts and no extra events. reason is "no schema entry" for a
relation the schema never mentions, or "schema entry has no columns" for a
definition that describes none (only reachable by skipping stdb.schema.new,
which rejects it).
Two more reasons come from the binary wire, and the first is the one a
hand-written schema meets. A table needs generated types. Nothing on the
wire names a column or states its width, so an entry without them cannot be
decoded at all, and there is no degraded, untyped reading to fall back on. Such a
relation is skipped with the reason binary rows need generated types. The
second catches a types list that is present but unusable, whether that is a
sum_<Name> or product_<Name> tag with no typespace entry to declare it, a
vec_ of a zero-width element, a tag the decoder does not know, or a types
list shorter than columns. Any of those gives binary rows need a decodable type for column "<name>". Both are checked once per relation rather than once
per row, because the alternative is a raise from inside the decoder where every
other cause here degrades, and both report through exactly the two surfaces
above. Views are included, so no decodable types means no rows.
A subscribe_query handle carries the same report, so the case that needs it
most does not depend on an on_event at all. Subscribing to a relation you have
not described is the one place the loss is otherwise invisible. The server
applied the set, so the callback fires with err = nil and is_active() is
true, while every row is dropped. sub.undescribed is what says so.
local sub = db:subscribe_query({ "SELECT * FROM my_view" }, function(err) end)
sub:on_applied(function()
for _, s in ipairs(sub.undescribed or {}) do
print(s.table, s.dropped, s.reason) -- my_view 3 no schema entry
end
end)
It is rewritten on every apply, nil included, because the set re-applies after a reconnect, and a report left over from the previous apply would be an accusation the handle could never withdraw. An unsubscribe reply does not touch it, since that handle is already ended.
It is a report, not a raise, deliberately. A raise from inside
cache.apply_database_update would abort the frame's table loop, so subscribing
to one view would drop the rows of every relation you did describe that the
frame listed after it and leave the subscribe callback parked forever, and it
would repeat on every reconnect, since the same frame arrives again. cache.find
and stdb.schema.new raise because a caller made a mistake at a callsite; here
the server sent something your schema cannot describe, which is the same class of
problem as an unknown message tag, and gets the same answer.
Before this, such rows were stored as {}, one fieldless row per arrival,
counted as an insert, with no error anywhere; and a delete of a row the relation
never held evicted the last row it did hold, because a fieldless row matches
everything.
Row value types
A cached row's columns are decoded against the schema's declared types, so an
Identity column is the server's 0x… hex string, a Timestamp is micros as
a number, an enum is a named variant, and a 128-bit integer is exact decimal
digits. That is the contract from 1.0 on, and the table below is all of it.
What switches it is one thing, whether the table's schema entry carries types.
A generated entry has them, and its rows go through
protocol.decode_typed_bsatn to the values tabulated below. An entry without
them cannot be decoded at all, so the relation is skipped and reported instead.
See Relations your schema does not
describe.
A row that decodes cleanly but leaves bytes over raises, naming the fix,
stdb.protocol_v2: decode_typed_bsatn: 8 byte(s) left after decoding a row of 3 column(s) - the schema is behind the server, regenerate the module. That is a
generated module the server has moved past, and cursor exhaustion is the only
structural self-check BSATN admits. Silence there was worse than a raise. A
column added at the end decoded every row "fine" and dropped the new field
forever, and one added in the middle misread every column behind it. The check
runs after the row is built and before the store is touched, so no half-applied
frame is exposed.
The typespace has to reach the connection. schema = gen carries it, since
connect reads that section off the module; passing it as its own option is the
other spelling. It is required for any table with an enum or a nested product
column, and unused by one without.
local gen = require("my_schema")
local db = stdb.connect({ ..., schema = gen })
-- Or, section by section.
local db = stdb.connect({
...,
schema = stdb.schema.new(gen.tables),
typespace = gen.typespace,
})
typespace is a second option rather than a schema key, because table names are
chosen by whoever wrote the server module. 2.8.2 will happily serve a table
called typespace, and stdb.schema.new rejects any entry without a columns
list anyway. Omitting it is not a degradation here, it is a loss. A sum_<Name>
or product_<Name> column has no knowable width without the typespace entry
that declares it, so the reader cannot find where the column after it starts
either. The relation is skipped and reported before a row is read, with reason
binary rows need a decodable type for column "<name>", exactly as a missing
types list is. A table whose columns are all built-in shapes needs no
typespace.
You can also decode a row yourself, which is what the cache does.
local row = stdb.protocol.decode_typed_bsatn(encoded, gen.tables.all_types, gen.typespace)
-- row.status -> { tag = "busy" } not { 1, {} }
-- row.point -> { x = 3, y = -4 } named from the declared field order
-- row.at -> 1787163313278160 micros, unwrapped
-- row.u_128 -> "340282366920938463463374607431768211455" exact u128
stdb.protocol.decode_bytes(bytes, tag, typespace) does one value, for a reducer
parameter or a procedure return (generated.procedures.foo.returns). It raises
if a byte is left over, since that means the tag is not the type the server
encoded.
What comes out, per tag.
| tag | Lua value |
|---|---|
bool, string, i8..u32, f32, f64 |
as-is, and an f32 is the exact single the server stored, see below |
i64, u64 |
number, and it rounds past 2^53, see below |
i128, u128 |
exact decimal digit string, always a string |
i256, u256 |
0x-prefixed lowercase hex, minimal width ("0x0", "-0x8000…") |
vec_u8 |
Lua byte string, since the bytes arrive raw rather than as a list of elements |
vec_<T> |
array, each element decoded |
opt_<T> |
the decoded value, or nil |
sum_<Name> |
{ tag = "<variant>", value = <decoded payload> } |
product_<Name> |
{ <field> = <decoded>, ... }, named from the declared field order |
identity |
0x-prefixed hex string |
connection_id, uuid |
exact decimal digit string |
timestamp, time_duration |
micros, as a number |
schedule_at |
{ tag = "Interval" | "Time", value = <micros> } |
unit |
nil |
unknown, result, anything unresolvable |
the value, unchanged |
An f32 is the exact single the server stored. BSATN carries the four bytes
the server holds, so a stored 0.1 decodes to 0.10000000149011612, which is
that single's exact value written as a double. The consequence is narrower than
it sounds. A value that fits 24 mantissa bits is exact and compares equal to its
literal, so 0, -1.5, 0.25, 100.5 and any integer up to 2^24 all do, while
0.1, 0.2 and 3.14 do not. Compare an f32 with a tolerance rather than
==, or declare the column f64.
Three things there are decisions rather than consequences, and all three are
asserted against captured server output in dev/test/test_fixtures.lua. The
first comes with a caveat about what a consumer may do with the value.
128-bit integers are strings. A u128 is 16 raw little-endian bytes on the
wire and Lua 5.1 has one number type, a double, so u128::MAX cannot be a Lua
number at all. The reader walks those bytes straight into decimal digits and
hands back a string. Narrow values are strings too, so the type never depends on
the magnitude. 256-bit values come back as 0x hex instead, since the digits are
there either way, and turning 32 bytes into decimal would mean bignum division
in Lua for no gain. Both forms are lossless.
ConnectionId and Uuid are 128-bit too, and they get the same digits rather
than the canonical 0x… / hyphenated text an Identity gets. That is a choice,
not a limit, since connection_id_to_hex is exact, so the decoder could render
it here. The reason is that an Identity arrives as 32 bytes that have exactly
one conventional rendering, while these two arrive as a plain 128-bit integer
that has several. Rendering at decode time would put a formatting decision
(32-digit hex? hyphenated? both?) on the frozen path for two column types that
are rare in row data, and it would cost a digit-array division per such column.
Digits are the raw value; connection_id_to_hex is the renderer, and it is one
call away. Compare with ==, render for display.
Treat that string as an identifier, not a number. Lua 5.1 coerces a numeric
string in arithmetic, so row.serial + 1 silently yields a rounded double and
raises nothing, so the value survives arrival and is destroyed by first use. And
</> are lexicographic, not numeric. "9007199254740993" < "10000000000000000"
is false, table.sort on a 128-bit column gives 10000000000000000 < 9007199254740993 < 999, and for i128 the - prefix inverts it again ("-8" < "-9" is true). Compare these values with ==, which is exact because the
representation is canonical, and leave 128-bit arithmetic and ordering to the
server.
i64/u64 stay numbers, so a u64 primary key above 2^53 is not safe. This
is the one limit here that cannot be fixed, only bounded, because Lua 5.1 has a
single number type, a double, exact for integers up to
2^53 = 9007199254740992 and no further. Above that some neighbouring integers
share a double, half the adjacent pairs just past 2^53 and effectively all of
them near u64's maximum, so two distinct server keys become one Lua key and the
cache merges the two rows, so last arrival wins, count is one short, and
nothing raises. The digits do survive on the wire, and returning them as text
would fix it, but that retypes message.id from number to string for every
consumer of every u64 column, which is a bigger break than the bound is worth.
So u64 stays a number.
In practice an auto-increment key needs 9.0e15 rows to reach the ceiling. A u64
holding a snowflake id, a hash or an externally-minted id does not, and such a
table wants a string or 128-bit key instead, since i128/u128 decode to exact
decimal digit strings, so keys stay distinct at any width. Pinned both ways in
dev/test/test_cache.lua.
An Identity primary key works, on a generated schema. player keyed by the
connecting identity is the canonical SpacetimeDB pattern, and the whole of it is
that the key must be a comparable scalar.
-- `db.identity` is the hex string `InitialConnection` handed you, and it is
-- exactly what the column decodes to.
local me = stdb.cache.get(db.cache, "player", db.identity)
local mate = stdb.cache.find(db.cache, "player", "sponsor", db.identity)
A hand-written schema cannot reach the mis-keyed case at all, which is the
one improvement a binary wire hands you for free. With no types there is
nothing to decode a row against, so the relation is skipped and reported rather
than cached under a key that only ever matches itself. Generate the schema.
Some(nil) is not distinguishable from None. An Option normalises to the
value or nil, and a Lua table field holding nil is absent, so Some(None)
collapses. In practice nothing produces it. A JS module writing null into an
Option<Option<i32>> column emits the outer none, captured and asserted. If
you must tell them apart, the outer ordinal is the first byte of that column's
slice in the raw row, 0 for some and 1 for none, read before decoding.
spacetimedb-defold
Using the SDK
Working on the SDK