api: settle the v1.1.0 surface before it freezes #68

Merged

yuris merged 4 commits from feat/api-surface-v11 into main 2026-08-20 16:23:17 +00:00

Conversation 0 Commits 4 Files changed 7 +301 -126

yuris commented 2026-08-20 15:36:55 +00:00

Owner

Copy link

Phase 5 of the v1.1.0 milestone (#59): the API-surface review of the new
surface only. v1.0.0's freeze review found eight renames worth making, and this
is the same question for everything added this milestone — the last moment any of
it is free. Zero external consumers, so every change here is a clean cutover: no
aliases, no shims, no deprecated paths.

Four changes, each from the review, each demonstrated rather than argued from
taste.

1. auth.exchange's token option becomes credential

opts.token was the provider credential, but token means "a SpacetimeDB
token" in every other public place: connect{token=}, db.token,
stdb.with_token(url, token), the ?token= query parameter, the persisted
{identity, token} record, and auth.exchange's own cb(err, token).

The clinching evidence is that the implementation already refused the name — it
read local credential = required(opts, "token") because the result needed the
word token. One function body needed two words for two things and only the
public key stayed ambiguous. Both meanings appeared within twelve lines of the
module's own example, and the headline example is not a token at all: a Steam
session ticket is a ticket.

The HTTP body field stays token — that is the wire shape and belongs to the
caller's backend. The README now states the mapping explicitly, and a test
asserts it directly, plus that no credential field leaks onto the wire.

Error string is now stdb.auth: credential is required.

2. The http_seam callback drops err

Was cb(err, status, body), with err set only for a transport failure and
every HTTP status — 4xx and 5xx included — arriving with err = nil. Two facts
killed it:

• The value was discarded. The only consumer did if err then return failed("... transport failure ..."). The first parameter of a public contract
  carried no information to its only reader.
• It inverted the rule the README states for the whole library ("Callbacks
  are (err, ...) — err is nil on success"). http_seam is a documented,
  injectable seam, so third parties will write one, and the natural
  implementation sets err for a 401.

Observed cost of that natural mistake, before this change: four requests for a
credential rejected on the first, reported as a transport failure — silently
defeating the 4xx-no-retry rule the README calls the part worth knowing.

Now post(url, headers, body, cb) calls back (status, body), with status == 0
meaning nothing answered. That is Defold's own signal, already verified against
engine source, so there is one way to say "no response" instead of two. The seam
became a pure pass-through and the two duplicated transport-failure branches in
auth.lua merged into one. Error strings are unchanged.

3. The auth cache nests under its own key

The record was a flat { [provider] = token } with provider caller-chosen,
sharing a namespace with connect's { identity, token } — only the slot name
kept them apart. Both directions were reproduced: provider = "token" on a
shared slot returned connect's identity token as the exchange result with zero
requests, and connect persisting an identity wiped every cached provider.

The SDK already made this argument in the opposite direction: table accessors are
namespaced under db.tables precisely because "table names belong to whoever
wrote the server module", and that construction has no reserved list. A flat
provider map had two reserved words and refused nothing.

Now { auth = { [provider] = token } }. Merge semantics preserved on both axes.

The implementer deviated from the review's suggested cached.auth and cached.auth[provider] and type-checked both levels instead — correctly: bare
truthiness introduces a raise class the flat version could not produce, since
{ auth = 99 } raises on indexing, and { auth = "junk" } with
provider = "format" reaches string.format through the string metatable, which
is the exact leak the existing string check exists to stop, one level down.

4. connection.spread_for becomes connection.backoff_spread

Right on meaning, wrong on family: spread_for was the only _for-suffixed name
in the public surface, among backoff_delay, settle_pending, resubscribe_all,
set_send, identity_to_hex, decode_typed. backoff_spread sits next to the
only function it is ever passed to — in the README table, in autocomplete, and
alphabetically.

backoff_delay's signature is untouched and its spread parameter keeps its
name. The review also confirmed the trailing-optional-multiplier shape is right
and that taking the identity directly would be worse, since it keeps
backoff_delay a pure numeric function testable by exact value.

5. Completing #3: connect now merges rather than overwriting

Nesting fixed two of the three collisions. The third survived, and the
implementer flagged it rather than letting it pass: the IdentityToken handler
wrote save({identity=..., token=...}) wholesale, so every connect dropped
the auth key regardless of nesting. Auth preserving connect's record is only
half of a mutual arrangement.

The handler now loads, sets the two fields, and saves. connect's own behaviour
is unchanged — it only ever reads persisted.token back — and a shared slot is a
supported configuration rather than a misuse, since sys_storage(name) always
took a caller-chosen name.

6. Two stale archive figures, from the archive gate

.gitattributes:4 still said "54 entries before, 11 after"; re-measured, those
are 56 and 12, so the repo stated the pair twice with conflicting values while
the README pointed at .gitattributes as corroboration. And the README credited
those counts to "the zip a consumer fetches", which actually enumerates 13 —
the forge prepends a spacetimedb-defold/ directory entry. Both reworded to
attribute the counts to git archive.

Verification

lua dev/test/runner.lua            7 passed, 0 failed
stylua --check stdb dev/test       exit 0, silent
conformance.sh                     PASS, identity persisted across a restart
html5-conformance.sh               PASS, identity survived a reload
grep -rn spread_for                no matches anywhere

The four v1.0.0 exact-value backoff assertions remain byte-identical.

Mutation-checked: reverting the wholesale save fails the suite; breaking the
cache nesting and the status == 0 branch each fail it too.

One process note worth recording

Adding the new test locals tipped test_connection.lua over Lua 5.1's ceiling of
200 locals per function. The runner printed that crash before its per-file ok
lines, so reading the tail of the output showed a false green — I nearly took it.
The block is now scoped in do ... end with a comment, since the file will hit
this again, and the lesson is that this runner's summary must be grepped rather
than tailed.

Phase 5 of the v1.1.0 milestone (#59): the API-surface review of the **new** surface only. v1.0.0's freeze review found eight renames worth making, and this is the same question for everything added this milestone — the last moment any of it is free. Zero external consumers, so every change here is a clean cutover: no aliases, no shims, no deprecated paths. Four changes, each from the review, each demonstrated rather than argued from taste. ## 1. `auth.exchange`'s `token` option becomes `credential` `opts.token` was the *provider* credential, but `token` means "a SpacetimeDB token" in every other public place: `connect{token=}`, `db.token`, `stdb.with_token(url, token)`, the `?token=` query parameter, the persisted `{identity, token}` record, and `auth.exchange`'s **own** `cb(err, token)`. The clinching evidence is that the implementation already refused the name — it read `local credential = required(opts, "token")` because the result needed the word `token`. One function body needed two words for two things and only the public key stayed ambiguous. Both meanings appeared within twelve lines of the module's own example, and the headline example is not a token at all: a Steam session ticket is a ticket. The HTTP body field stays `token` — that is the wire shape and belongs to the caller's backend. The README now states the mapping explicitly, and a test asserts it directly, plus that no `credential` field leaks onto the wire. Error string is now `stdb.auth: credential is required`. ## 2. The `http_seam` callback drops `err` Was `cb(err, status, body)`, with `err` set only for a transport failure and every HTTP status — 4xx and 5xx included — arriving with `err = nil`. Two facts killed it: - **The value was discarded.** The only consumer did `if err then return failed("... transport failure ...")`. The first parameter of a public contract carried no information to its only reader. - **It inverted the rule the README states for the whole library** ("Callbacks are `(err, ...)` — `err` is nil on success"). `http_seam` is a *documented, injectable* seam, so third parties will write one, and the natural implementation sets `err` for a 401. Observed cost of that natural mistake, before this change: **four requests for a credential rejected on the first**, reported as a transport failure — silently defeating the 4xx-no-retry rule the README calls the part worth knowing. Now `post(url, headers, body, cb)` calls back `(status, body)`, with `status == 0` meaning nothing answered. That is Defold's own signal, already verified against engine source, so there is one way to say "no response" instead of two. The seam became a pure pass-through and the two duplicated transport-failure branches in `auth.lua` merged into one. Error strings are unchanged. ## 3. The auth cache nests under its own key The record was a flat `{ [provider] = token }` with `provider` caller-chosen, sharing a namespace with `connect`'s `{ identity, token }` — only the slot name kept them apart. Both directions were reproduced: `provider = "token"` on a shared slot returned `connect`'s identity token as the exchange result with zero requests, and `connect` persisting an identity wiped every cached provider. The SDK already made this argument in the opposite direction: table accessors are namespaced under `db.tables` precisely because "table names belong to whoever wrote the server module", and that construction has no reserved list. A flat provider map had two reserved words and refused nothing. Now `{ auth = { [provider] = token } }`. Merge semantics preserved on both axes. The implementer deviated from the review's suggested `cached.auth and cached.auth[provider]` and type-checked both levels instead — correctly: bare truthiness introduces a raise class the flat version could not produce, since `{ auth = 99 }` raises on indexing, and `{ auth = "junk" }` with `provider = "format"` reaches `string.format` through the string metatable, which is the exact leak the existing string check exists to stop, one level down. ## 4. `connection.spread_for` becomes `connection.backoff_spread` Right on meaning, wrong on family: `spread_for` was the only `_for`-suffixed name in the public surface, among `backoff_delay`, `settle_pending`, `resubscribe_all`, `set_send`, `identity_to_hex`, `decode_typed`. `backoff_spread` sits next to the only function it is ever passed to — in the README table, in autocomplete, and alphabetically. `backoff_delay`'s signature is untouched and its `spread` parameter keeps its name. The review also confirmed the trailing-optional-multiplier shape is right and that taking the identity directly would be worse, since it keeps `backoff_delay` a pure numeric function testable by exact value. ## 5. Completing #3: `connect` now merges rather than overwriting Nesting fixed two of the three collisions. The third survived, and the implementer flagged it rather than letting it pass: the `IdentityToken` handler wrote `save({identity=..., token=...})` **wholesale**, so every connect dropped the `auth` key regardless of nesting. Auth preserving connect's record is only half of a mutual arrangement. The handler now loads, sets the two fields, and saves. `connect`'s own behaviour is unchanged — it only ever reads `persisted.token` back — and a shared slot is a supported configuration rather than a misuse, since `sys_storage(name)` always took a caller-chosen name. ## 6. Two stale archive figures, from the archive gate `.gitattributes:4` still said "54 entries before, 11 after"; re-measured, those are 56 and 12, so the repo stated the pair twice with conflicting values while the README pointed at `.gitattributes` as corroboration. And the README credited those counts to "the zip a consumer fetches", which actually enumerates 13 — the forge prepends a `spacetimedb-defold/` directory entry. Both reworded to attribute the counts to `git archive`. ## Verification ``` lua dev/test/runner.lua 7 passed, 0 failed stylua --check stdb dev/test exit 0, silent conformance.sh PASS, identity persisted across a restart html5-conformance.sh PASS, identity survived a reload grep -rn spread_for no matches anywhere ``` The four v1.0.0 exact-value backoff assertions remain byte-identical. Mutation-checked: reverting the wholesale save fails the suite; breaking the cache nesting and the `status == 0` branch each fail it too. ## One process note worth recording Adding the new test locals tipped `test_connection.lua` over Lua 5.1's ceiling of 200 locals per function. The runner printed that crash *before* its per-file `ok` lines, so reading the tail of the output showed a false green — I nearly took it. The block is now scoped in `do ... end` with a comment, since the file will hit this again, and the lesson is that this runner's summary must be grepped rather than tailed.

yuris added 2 commits 2026-08-20 15:36:55 +00:00

api: settle the v1.1.0 surface before it freezes ... 3a35e75450

Four renames from the API-surface review, plus two stale doc figures. All of
this is unreleased, so there are no consumers and no shims: each old name is
gone, not aliased.

`auth.exchange`'s `token` option becomes `credential`. `token` means a
SpacetimeDB token everywhere else here - `connect{token=}`, `db.token`,
`with_token`, the `?token=` parameter, the persisted `{identity, token}` record,
and `exchange`'s own `cb(err, token)` result - and both meanings appeared within
a dozen lines of the module's worked example. The implementation already refused
the name internally (`local credential = required(opts, "token")`). The request
BODY field stays `token`: that shape belongs to the caller's backend, not to
this SDK, and the README now says so explicitly since option and field differ.

The `http_seam` callback drops `err` and calls back `(status, body)`, where
status 0 means nothing answered. The old `err` was discarded by its only
consumer, and it inverted the `(err, ...)` rule the README states for the whole
library - so a third party writing this documented, injectable seam naturally
set `err` for a 401, which turned a credential rejected on the first ask into
four requests reported as a transport failure. Status 0 is Defold's own signal,
already the seam's exact test, so there is now one way to say it instead of two
that can disagree. The seam is a pass-through and auth's two transport-failure
branches merge into one.

The auth cache nests under its own `auth` key. Flat, `{ [provider] = token }`
collided with `connect`'s `{ identity, token }` record - same shape of file,
same `sys_storage(name)` seam, only the slot name between them - and it bit both
ways: `provider = "token"` returned connect's identity token as an exchange
result having made zero requests, and a connect save wiped every cached
provider. `provider` is caller-chosen, so no reserved-name list can fix that;
the SDK already namespaced table accessors under `db.tables` on exactly this
argument. Both levels are type-checked rather than merely truthy, because a
tampered `{ auth = 99 }` would raise on the index.

`connection.spread_for` becomes `connection.backoff_spread` - correct on
meaning, wrong on family, being the only `_for` name beside `backoff_delay`,
`settle_pending`, `resubscribe_all`, `set_send`. It now sits next to the only
function it is ever passed to. `backoff_delay`'s `spread` parameter is
positional and unchanged.

Finally, two figures re-measured rather than trusted: `.gitattributes` claimed
54 entries before and 11 after where `git archive` gives 56 and 12, and the
README attributed those counts to the fetched zip, which enumerates one more
because the forge prepends a `spacetimedb-defold/` directory entry.

Not fixed, and worth a follow-up: nesting stops auth from clobbering connect,
but connect still rewrites its slot wholesale at connection.lua:725, so an
identity saved into a shared slot still drops the cache. That is frozen v1.0.0
behaviour and out of scope here; the README now tells callers to give the
exchange its own slot name, as the documented example already does.

connect: merge the identity record instead of writing the slot whole ... 90cb455bc2

Completes the third direction of the auth-cache nesting. The API review found
the flat {[provider]=token} record shared a key namespace with connect's
{identity, token}, and nesting under `auth` fixed two of the three collisions:
provider="token" no longer reads back connect's identity token, and auth's own
save now preserves a connect record.

The third stayed broken, as the previous commit's report correctly flagged: the
IdentityToken handler wrote `save({identity=..., token=...})` wholesale, so
every connect dropped the `auth` key. Nesting cannot fix that from auth's side -
auth preserving ours is only half a mutual arrangement.

So the handler now merges: load, set identity and token, save. A storage slot is
not exclusively connect's, and it never was - `sys_storage(name)` always took a
caller-chosen slot name, so a consumer pointing both at one slot is a supported
configuration rather than a misuse. Behaviour for connect itself is unchanged,
since it only ever reads `persisted.token` back.

Pinned by a test asserting both directions, including the case of a slot that
holds ONLY an auth cache. Mutation-checked: reverting to the wholesale save fails
the suite.

One incidental find while adding it: the new locals tipped test_connection.lua
over Lua 5.1's ceiling of 200 locals per function, which surfaced as
"main function has more than 200 local variables" - and, worth recording, the
runner reported that crash BEFORE the per-file ok lines, so reading the tail of
the output showed a false green. The block is scoped in `do ... end` with a
comment saying why, since the file will hit this again.

yuris referenced this pull request 2026-08-20 16:01:40 +00:00

v1.1.0 release checklist #59

yuris force-pushed feat/api-surface-v11 from 90cb455bc2 to 328c8b500a 2026-08-20 16:20:44 +00:00 Compare

yuris added 1 commit 2026-08-20 16:22:59 +00:00

review: correct ten README claims the truth check measured as wrong ... fd6f511bd6

Phase 5's README behavioural-truth gate re-ran every claim this milestone added,
by execution against a live 2.8.2 on the same CPU model the section names. Ten
were wrong. The load-bearing point about the numeric ones: the section hedges
with "this is one machine", and a hedge about MACHINES does not excuse a number
that misses on the same silicon.

The one I most deserved to be caught on: "at the 30s cap a shared slot is two
clients 15ms apart rather than simultaneous". Clients sharing a slot share the
multiplier, so they share the delay exactly - 0ms apart, demonstrated with two
real identities both landing on 0.9390 and both on 28.170000s. 15ms is the gap
between ADJACENT slots. Measured occupancy at 20,000 clients: all 1000 slots
used, 35 clients in the busiest, arriving in the same millisecond. The row now
says what quantisation actually buys - a herd smeared across the window in 15ms
steps - instead of implying collisions are harmless.

"The first connect has no spread" understated it the same way: EVERY attempt
before the first successful connect is unspread, so a fleet relaunching into an
outage retries on the exact 1, 2, 4, 8, 16, 30 ladder - which is the herd the
paragraph claims to prevent. Now stated plainly, pointing at #65.

Frame-budget numbers, all re-measured:

- 100k per-row is 3.55us/row and +24%, not 4.08 and +33%. Two independent
  sweeps, one prefix-matched to the README's own documented command.
- The typed penalty is 6-17%, median 11%, not 13-20%. That claim was also
  internally inconsistent in a machine-independent way: the section's own
  crossing pair (6100 hand-written against 5700 typed) encodes 7%, which cannot
  coexist with 13-20%. Now "roughly a tenth", with the band, and the typed
  crossing corrected to ~5400.
- "Run-to-run spread is 6-11% throughout" is neither the floor nor the ceiling of
  the harness's own output: 25 native summary points printed 1% to 21%.
- 5000-row per-row cost is 2.9us/row, not 3.06.
- The BSATN ceiling is quoted as the observed 83-88% band and 6-8x rather than
  the optimistic edge, and says outright that these are bounds, not predictions.
- "Medians over at least six samples" was my arithmetic, not the tool's output:
  the harness prints two independent 3-sample medians per invocation and pools
  nothing. Now says the two are pooled by hand.

Two smaller ones:

- The transport-failure error row said "Nothing answered, N times", which also
  fires when only the LAST attempt got no answer. The timeout row directly below
  already carried the right hedge; now both do.
- "an empty stub" and "raises attempt to index global 'html5'" cannot both be
  true, and the reviewer executed both cases. The stub reading is the one the
  citation supports: InitializeHtml5 is empty, so nothing registers `run` and the
  raise is "attempt to call field 'run' (a nil value)".

The gate also confirmed the archive wording this branch already carries is exact
(git archive 56 -> 12, forge zip one more), and that the blocking auth-cache
paragraph corrected in the previous commit now matches the code in both merge
directions.

yuris merged commit 952cf1194c into main 2026-08-20 16:23:17 +00:00

yuris referenced this pull request from a commit 2026-08-20 16:23:18 +00:00

api: settle the v1.1.0 surface before it freezes (#68)

yuris referenced this pull request 2026-08-20 16:31:55 +00:00

v1.1.0 release checklist #59

yuris referenced this pull request 2026-08-20 16:32:45 +00:00

v1.1.0 release checklist #59

yuris referenced this pull request 2026-08-22 03:53:12 +00:00

patches: carry upstream PR 68 instead of our narrower fix #106

yuris referenced this pull request from a commit 2026-08-22 03:58:25 +00:00

patches: carry upstream PR 68 instead of our narrower fix (#106)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area:defold

Defold engine integration

  

area:docs

Documentation and examples

  

area:infra

Flake, CI, secretspec

  

area:protocol

Wire protocol conformance

  

area:testing

Test harness and fixtures

  

kind:bug

Something broken

  

kind:chore

Maintenance

  

phase:breadth

UnsubscribeMulti, procedures, BSATN

  

phase:core

Core SDK layers (protocol/cache/connection)

  

phase:hardening

Reconnect, token persistence, errors

  

phase:slice

Vertical slice: first playable end-to-end

No labels

area:defold

area:docs

area:infra

area:protocol

area:testing

kind:bug

kind:chore

phase:breadth

phase:core

phase:hardening

phase:slice

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

yuris/spacetimedb-defold!68

No description provided.