Protocol Notes
yuris edited this page 2026-08-22 17:21:29 +07:00

Page revisions

13 Commits

Author SHA1 Message Date
yuris
cf73bde7b3 docs: humanize the prose across the wiki
No technical claim changed. This is a pass over how the pages read, not
what they say.

Removed 419 em and en dashes, which were the most consistent tell in the
text, along with mechanical boldface and explanatory mid-sentence colons.
Then the judgement work the humanizer skill asks for: rule-of-three
padding, announced counts, signposting, passive voice with the actor
hidden, verbless noun-phrase lists that should have been sentences, and
filler.

Every measured number, file and line reference, version, quoted error
string and piece of reasoning survives. That specificity is what makes
these pages worth reading, and over-editing would have flattened it.

Two accuracy fixes fell out of the pass, both found by reading closely
enough to rewrite:

  * API-Reference misquoted the editor command's failure. The real
    message is `stdb: no schema from <url>. Is the server running, and is
    curl on PATH?`, with a period, not the em dash the page had.
  * Roadmap still listed "an example project you can open and run" under
    what is missing. `dev/example-chat` has been exactly that for a
    while, with its own `game.project` and a headless variant CI runs.
    What is genuinely still absent is a staged tutorial of the kind
    upstream ships for Unity, Godot and Unreal, so the entry now says
    that instead.
2026-08-22 17:21:29 +07:00
yuris
38402d71ca docs: 4.2.4 with no patch, and the quickstart moves here
extension-websocket 4.2.4 fixes defold/extension-websocket#67, so the
HTML5 subprotocol patch is gone. Every page that carried the patching
instructions, a vendored-fork workflow, or a 4.2.3 pin is corrected.

Quickstart is now the SDK's tutorial rather than a section of the
README. It walks an unpublished module through to a connected game that
replicates positions between clients, and it absorbed the parts of the
old page the README version did not carry.

API-Reference and Usage-Guide are brought in line with PR #111:
`schema = gen`, `json` no longer required, `stdb.query` on the facade,
`db:ping` as `(err, seconds)`, `stdb.await`/`await_event` including the
Lua 5.1 limit on yielding across `pcall`, `stdb.builder(opts)`, the full
`on_event` type list with payload keys, and a section for the mistakes
that now raise instead of failing quietly.

Protocol-Notes and Roadmap no longer describe `stdb/protocol.lua` or the
v1 JSON wire in the present tense; both went in 4.0.0.
2026-08-22 16:57:42 +07:00
08c5c09077 wiki: one wire — drop the v1 JSON pages and claims for 4.0.0
The SDK deleted the v1 JSON wire in 4.0.0, so the wiki stops offering it.

V1-Opt-Out is deleted outright: it documented an opt-out that no longer
exists. Its nav entry and Home row go with it, and all 11 inbound links
across the Usage Guide are gone.

Every page that named a removed API is corrected rather than trimmed:
the protocol option, light, json on connect, Builder:with_protocol, the
legacy whole-set db:subscribe, subscription_replaced, and
Call:on_out_of_energy. json stays documented on auth.exchange and
schema.fetch, which are HTTP and still need it. The API Reference gains
connection.dispatch, decode_bytes and decodable_tag; the Usage Guide
gains the fourth undescribed reason, which the page had been missing
independently of this change.

A table with no types is now always skipped and reported. Four places
described an untyped positional fallback that no longer exists.

Protocol Notes keeps its v1 findings as archive, reframed: they are still
true of the server, the committed fixtures were captured over that wire,
and the decoder that reads them survives as dev/test/crosscheck_v1.lua -
a cross-check on the BSATN reader, since BSATN is opaque and a wrong type
belief returns a plausible number instead of raising.

One factual error corrected on the way through. The Roadmap claimed the
official TypeScript SDK 'prefers v3, falls back to v2, and does not offer
v1 at all'. It is on v1.bsatn.spacetimedb
(packages/sdk/src/db_connection_impl.ts:233) - a protocol version behind
this SDK. The wire-ceiling section now tables where every official client
actually sits, read off source, and says what v3 actually is: not a new
message schema but permission to coalesce several v2 ServerMessages into
one payload.

Install URLs at v4.0.0. Benchmark tables measured on the JSON wire keep
their numbers and gain a note saying so - re-running the sweep on the
current branch is what retires them, and inventing replacements would be
worse than dated figures.
2026-08-22 14:51:00 +07:00
eca4c8ad31 Gzip frames are decoded, as of 3.1.0
The server's default scheme is brotli and it compresses anything over 1KiB, so
the connect URL never omits the parameter - it asks for gzip when a gunzip is in
reach. Protocol Notes had the envelope wrong for the compressed case: the tag
byte replaces the message's leading byte rather than prefixing the stream, so
the gzip magic sits at byte 2. Roadmap moves frame compression out of Not
planned, where the stated reason - that a pure-Lua inflate would be large and
slow - stopped applying the moment the engine shipped one.
2026-08-22 12:06:22 +07:00
662d17cf06 docs: make the default wire the spine, and split the two audiences
The Usage Guide documented the JSON opt-out for 1144 lines before saying it
was not the default. The v2 material is folded into the sections it belongs
to, and the v1-only knobs - light, on_out_of_energy, the whole-set subscribe,
the older message shapes - move to V1-Opt-Out.

Provenance moves rather than disappearing: captures, byte tables and the
upstream Rust citations to Protocol-Notes, the benchmark narratives to
Development. Home and the sidebar now read as two paths, one for a game using
the SDK and one for someone changing it.

Also documents what the SDK gained: stdb.schema.fetch, stdb.schema.to_lua and
the Project menu command that writes the generated module. Four code blocks
were false and are fixed - two syntax errors, load vs loadstring on 5.1, and
a quickstart still telling consumers to run a script they never receive.
2026-08-21 20:00:10 +07:00
eaafcb8406 Flip the documented default to the v2 BSATN wire
`connect()` negotiates `v2.bsatn.spacetimedb` as of 2.0.0, so every page that
called v1 the default and v2 an opt-in was wrong the moment it merged. The
inversion is mechanical in most places — the `protocol` default, the two
SUBPROTOCOL rows, the version pins — and structural in three.

Quickstart was not merely stale, it could not run. Its sample connected with a
hand-written schema, subscribed with the legacy `subscribe` and called a reducer
with no `reducers`: on the default wire that decodes nothing, raises, and cannot
encode the call. Schema generation moves ahead of the first connect, where it is
now a prerequisite rather than an upgrade, and the sample passes `reducers` and
`procedures` and uses `subscribe_query`.

Usage-Guide's "everything above describes the default" is now the wrong way
round and cannot be fixed by swapping a word, so it says so plainly instead: the
page documents the opt-out first, an unqualified statement above is v1's, and the
differences are enumerated in the v2 section. The reducer contract grew the
second half it never had — v2 has two outcomes, and `on_out_of_energy` never
fires because nothing replaces it. API-Reference keeps the light-mode
investigation, scoped to the wire that still has a light mode.

Five things wrong beyond the flip, found by checking the pages against the
worktree rather than against the brief:

  * Development and Roadmap both said conformance runs v2 by dispatch only. It
    has been a matrix over both wires on every automatic trigger since #91.
  * Both said the HTML5 tier reports instead of gating. It gates now — it had to
    before v2 could be the default, since it is the only place the wasm build of
    the binary wire runs.
  * The smoke is seven phases on v2 and six on v1, not five, and the phase that
    prints PASS is the bulk apply, not the identity-keyed table.
  * The generated-schema sample called a reducer without passing `reducers`,
    which the flip turned from redundant into broken.
  * session_v2.jsonl holds 33 frames, not 29; the capture grew and the sentence
    describing it did not.

`STDB_BULK_TYPED` is marked as the v1 knob it now is; the typed description is
not optional on a wire whose rows carry no names.
2026-08-21 18:24:35 +07:00
yuris
a60153f138 State the measurement conditions on the latency numbers
The 20.0/10.5ms pair at 10k rows was published as if it were a baseline. A
re-run of the same commit measured every native absolute 2-3x higher, with
loadavg at 11.06 on 16 cores and the harness flagging 47% spread -- and the
10k ratio at 0.59-0.65 against the published 0.53. That is a material
difference, so say so instead of quoting the flattering half.

Wire-byte tables are unchanged; they are byte counts. The structural claims
stand and are now stated as structural: the big-integer quoting bucket cannot
exist on a binary wire and is enforced by a zero-ATTEMPTS assertion rather
than merely observed, and the HTML5 pre bucket collapses from the largest
share of the frame on v1 to a non-dominant one on v2.
2026-08-21 13:49:39 +07:00
9a17a8b9ef Correct what v2 separates, and what it costs
The v2 section claimed v2 delivers a reducer result separately from the
transaction it caused. That is backwards: ReducerResult::Ok carries the
transaction inline and arrives as one frame, and the capture pins that a
bare TransactionUpdate never arrives for one's own call. What v2 actually
separates is the caller's reducer_call from everybody else's broadcast,
which is where the bytes are saved.

"The wire is what changes, not the values" was false for everything
except cached rows. A transaction event carries no reducer_call, a
reducer's return value and Err payload stay raw BSATN, one-off rows stay
encoded, and on_out_of_energy cannot fire. A consumer porting to v2 needs
that list, not a reassurance.

Also document the wide reducer-arg types now that they encode, and add a
hostile-input note to the protocol notes: an unvalidated RowOffsets table
turns 232 KB of frame into 388 MB of heap, so the offsets are checked as
a partition of rows_data, array counts are bounded by the bytes left, and
a frame that decodes with bytes to spare was not the frame we thought.
2026-08-21 13:11:16 +07:00
261e79e1c2 Quote the latency win as a range, and say which frame the 46% is
Both sweeps were already in the table; the surrounding prose and the roadmap
quoted only the faster one, which reads as a flat 2x. It is 1.5-1.9x at 10k
rows. The byte share is the subscription readback specifically \u2014 the frame
PHASE6 times \u2014 not the whole session, which lands at 45.9% and rounds the
same.
2026-08-21 12:29:29 +07:00
ec08967921 Publish the measured v1-vs-v2 numbers
Wire bytes and apply latency, both wires against one local 2.8.2 server:
v2 rows are 46% of v1's bytes, client frames 31%, and a 10k-row first
subscription applies in 10.5ms against 20.0ms native. Includes the five-bucket
split (the big-integer quoting pass is structurally absent on v2, and row
construction falls 6.8ms to 1.9ms alongside decoding), the browser tier, and
the note that both wires bill into the same buckets with matching decode call
counts. Roadmap and the frame-compression entry cite the measurement instead of
the estimate. Usage-Guide records that the event type is proven end to end
against the dev module's ping event table.
2026-08-21 12:27:56 +07:00
4e7cf2bdfa Say plainly that the v1-vs-v2 numbers are not measured yet
The placeholder was an HTML comment, so a reader saw an empty heading. State
the pending comparison and the measured v1 baseline it will be compared
against; the comment stays for whoever fills the table in.
2026-08-21 12:15:19 +07:00
701c419b4d Document the opt-in v2 BSATN protocol
Protocol-Notes gains the v2 frame envelope, the ClientMessage/ServerMessage
map and the four wire facts the live capture settled: the timestamp is micros
despite upstream documenting nanoseconds, identity hex is the byte-reverse of
the wire bytes, Option/Result ordinals, and a client frame carrying no
compression byte is accepted. Benchmark figures are marked as pending rather
than guessed.

Usage-Guide documents connect({ protocol = "v2" }), the event-table event
type, the generated-types requirement, light being v1-only, subscribe_query
replacing the legacy subscribe, and on_out_of_energy being unreachable on v2.
API-Reference covers the new public surface. Roadmap moves the BSATN item to
done, pending only the decision to flip the default.
2026-08-21 12:13:25 +07:00
yuris
6bfe41b8ad docs: import documentation from README (#74) 2026-08-21 00:30:14 +07:00