cache: skip a relation the schema cannot describe, and emit views #49

Merged

yuris merged 4 commits from feat/42-views into main 2026-08-20 04:47:23 +00:00

Conversation 1 Commits 4 Files changed 18 +1007 -112

yuris commented 2026-08-20 03:52:28 +00:00

Owner

Copy link

Closes #42 — the last open 1.0 blocker.

table_def returned {} for a relation the schema does not describe, so def.columns was nil, decode_row(encoded, nil) set columns = {}, the mapping loop ran zero times and returned {}. An empty row was stored and counted as an insert, with no error anywhere. Every field of every view row was silently discarded, reachable by an ordinary SELECT * FROM my_view.

It was worse than lossy. Those fieldless rows were reachable through the ordinary read API: count() reported a row that had no fields, iter() handed out a table where every access is nil, and — because a pk-less delete scans for an equal row and pairs({}) never falsifies a match — a delete of a row the relation never held evicted the one it did hold. A consumer who registers nothing at all is therefore strictly better off now: the relation is simply empty, which is honest and inert.

Both halves of the issue are done, deliberately.

The loud half — skip and report, not raise

apply_database_update skips an undescribed relation whole and reports it in counts.undescribed, which emit_rows turns into an undescribed event.

Not a raise, and the reason is concrete: a raise aborts the frame's tables loop, so it would drop the rows of relations the consumer did describe and park the subscribe callback, repeating on every reconnect. handle_frame has no pcall at any of apply_database_update's four call sites, so it would propagate into Defold's websocket callback and kill the connection because someone subscribed to a view.

The loss a raise causes is also order-dependent, which is the strongest form of the argument. Replaying one frame containing an undescribed ghost and a described item, against a patched cache that raises:

frame order	described item rows
ghost first	0 — lost
item first	1
skip (this PR), either order	1, always

db_update.tables is ordered by the server, so a raise would lose described rows nondeterministically.

This follows the unhandled-event pattern handle_frame already uses for an unknown message tag — loud without being fatal — rather than cache.find's raise, which is a programmer error on a call the consumer controls.

Discoverability: the report reaches the handle, not only on_event

An event is only audible if you registered on_event, and the natural way to consume a view is subscribe_query({"SELECT * FROM my_view"}, cb). On that path the SDK affirmatively reported success — callback(nil, msg), handle applied, is_active() true — while every row was dropped, and on_event defaults to a no-op with no logging facility anywhere in stdb/.

Those signals are all correct (the server really did apply the set) and callback(err, msg) is frozen, so the report goes on the handle instead:

local sub = db:subscribe_query({ "SELECT * FROM my_view" })
sub:on_applied(function()
  for _, s in ipairs(sub.undescribed or {}) do
    print(s.table, s.rows, s.reason)  -- my_view  3  no schema entry
  end
end)

sub.undescribed is the same { { table, rows, reason } } list under the same name, written before on_applied fires and rewritten on every apply — nil included, since the set re-applies after a reconnect and a stale report would be an accusation the handle could never withdraw. The legacy M.subscribe returns a bare request_id with no handle to carry it, and its docstring says so.

The generator half

gen-schema.sh resolves misc_exports[].View's return type to the view's own column order and emits it into tables with view = true.

Both return forms are emitted. The Array form is {Array: {Ref: n}}; the Option form — a view yielding at most one row — declares the same kind of product through the some variant of a two-variant sum, so one helper resolves either and everything downstream, typespace seeding included, is identical. Verified end to end on a live 2.8.2: SELECT * FROM message_latest delivers ["second",2], positional in the view's declared (text, id) order, and decodes through schema.new(gen.tables) to text = "second" and id = 2 as a number. Reusing the underlying table's (id, sender, text) would have put the u64 in text. The dev module carries that view so the second resolution hop stays covered.

A view's own pk is still absent: a View at that endpoint carries name, params, is_public, is_anonymous, index and return_type — no key of any kind, even when the module source declares one — so no pk is emitted rather than guessed, and a changed view row is a delete plus an insert instead of one update event.

That ordering is the part that could have been worse than the raise, so it is measured. Frames 37–42 are a new capture block proving:

• a view's readback rows are positional where a table's are named
• a view is incrementally maintained
• unsubscribe returns positional deletes

The tool itself is now checked, not just its output. Nothing re-derived chat_schema.lua, so a generator resolving the wrong typespace entry — emitting a view with its underlying table's column order, the exact silent mis-mapping #42 exists to prevent — passed the whole Lua suite, which only ever reads the committed fixture. conformance.sh regenerates the schema from the module it just published and diffs it against the committed copy, ignoring line 1's source URL whose port follows STDB_PORT. That mutation now fails the run. dev-server.sh no longer rewrites the fixture on every publish: a clobber can never detect drift, because it destroys the copy it would have to compare against.

Verification

6 passed, 0 failed; stylua clean. Native and HTML5 conformance both PASS, including PHASE5's identity-keyed check, with the new schema diff green inside the native run.

Reverting the guard fails three files by name. The review's mutation sweep found two survivors and both are closed: two undescribed relations in one frame are now pinned in test_cache.lua (two entries, wire order, own row counts) and in test_connection.lua (two notices), so skipped[#skipped + 1] → skipped[1] fails both files; and the generator's wrong-ref mutation fails the conformance diff.

On #42's acceptance wording

#42's Acceptance line asks for "a test that an unknown relation raises rather than storing an empty row." Nothing raises, and that is deliberate: #42's own Work bullet authorises the deviation — "should raise, or emit a distinct event" — and the table above is why the event is the better half of that latitude. Recorded here because the issue text will outlive the PR.

Frozen surface

• undescribed, on counts, as an event, and now on the Subscription handle: one concept, one name. It mirrors unhandled for an unknown message tag — same class of problem, same bare-adjective form.
• Payload { type, table, rows, reason }. rows is a count, not a list: there is nothing to hand over, since the rows were never decoded, and it is the vocabulary counts already uses for inserts and deletes. Documented as a count at both surfaces, because the sibling row on an insert or delete event is a row.
• counts stays additive: with nothing skipped the keys are exactly deletes, events, inserts, and no consumer anywhere iterates pairs(counts).

Closes #42 — the last open 1.0 blocker. `table_def` returned `{}` for a relation the schema does not describe, so `def.columns` was nil, `decode_row(encoded, nil)` set `columns = {}`, the mapping loop ran zero times and returned `{}`. **An empty row was stored and counted as an insert, with no error anywhere.** Every field of every view row was silently discarded, reachable by an ordinary `SELECT * FROM my_view`. It was worse than lossy. Those fieldless rows were reachable through the ordinary read API: `count()` reported a row that had no fields, `iter()` handed out a table where every access is nil, and — because a pk-less delete scans for an equal row and `pairs({})` never falsifies a match — a delete of a row the relation never held evicted the one it did hold. A consumer who registers nothing at all is therefore strictly better off now: the relation is simply empty, which is honest and inert. Both halves of the issue are done, deliberately. ## The loud half — skip and report, not raise `apply_database_update` skips an undescribed relation whole and reports it in `counts.undescribed`, which `emit_rows` turns into an `undescribed` event. **Not a raise**, and the reason is concrete: a raise aborts the frame's `tables` loop, so it would drop the rows of relations the consumer *did* describe **and** park the subscribe callback, repeating on every reconnect. `handle_frame` has no `pcall` at any of `apply_database_update`'s four call sites, so it would propagate into Defold's websocket callback and kill the connection because someone subscribed to a view. The loss a raise causes is also **order-dependent**, which is the strongest form of the argument. Replaying one frame containing an undescribed `ghost` and a described `item`, against a patched cache that raises: | frame order | described `item` rows | |---|---| | `ghost` first | **0 — lost** | | `item` first | 1 | | skip (this PR), either order | **1, always** | `db_update.tables` is ordered by the server, so a raise would lose described rows nondeterministically. This follows the `unhandled`-event pattern `handle_frame` already uses for an unknown message tag — loud without being fatal — rather than `cache.find`'s raise, which is a programmer error on a call the consumer controls. ## Discoverability: the report reaches the handle, not only `on_event` An event is only audible if you registered `on_event`, and the natural way to consume a view is `subscribe_query({"SELECT * FROM my_view"}, cb)`. On that path the SDK affirmatively reported success — `callback(nil, msg)`, handle `applied`, `is_active()` true — while every row was dropped, and `on_event` defaults to a no-op with no logging facility anywhere in `stdb/`. Those signals are all correct (the server really did apply the set) and `callback(err, msg)` is frozen, so the report goes on the handle instead: ```lua local sub = db:subscribe_query({ "SELECT * FROM my_view" }) sub:on_applied(function() for _, s in ipairs(sub.undescribed or {}) do print(s.table, s.rows, s.reason) -- my_view 3 no schema entry end end) ``` `sub.undescribed` is the same `{ { table, rows, reason } }` list under the same name, written before `on_applied` fires and rewritten on every apply — nil included, since the set re-applies after a reconnect and a stale report would be an accusation the handle could never withdraw. The legacy `M.subscribe` returns a bare request_id with no handle to carry it, and its docstring says so. ## The generator half `gen-schema.sh` resolves `misc_exports[].View`'s return type to the view's **own** column order and emits it into `tables` with `view = true`. **Both return forms are emitted.** The Array form is `{Array: {Ref: n}}`; the Option form — a view yielding at most one row — declares the same kind of product through the `some` variant of a two-variant sum, so one helper resolves either and everything downstream, typespace seeding included, is identical. Verified end to end on a live 2.8.2: `SELECT * FROM message_latest` delivers `["second",2]`, positional in the view's declared `(text, id)` order, and decodes through `schema.new(gen.tables)` to `text = "second"` and `id = 2` as a number. Reusing the underlying table's `(id, sender, text)` would have put the u64 in `text`. The dev module carries that view so the second resolution hop stays covered. A view's own pk is still absent: a View at that endpoint carries name, params, `is_public`, `is_anonymous`, `index` and `return_type` — no key of any kind, even when the module source declares one — so no `pk` is emitted rather than guessed, and a changed view row is a delete plus an insert instead of one update event. That ordering is the part that could have been worse than the raise, so it is measured. Frames 37–42 are a new capture block proving: - a view's readback rows are **positional** where a table's are named - a view is incrementally maintained - unsubscribe returns positional deletes **The tool itself is now checked, not just its output.** Nothing re-derived `chat_schema.lua`, so a generator resolving the wrong typespace entry — emitting a view with its underlying table's column order, the exact silent mis-mapping #42 exists to prevent — passed the whole Lua suite, which only ever reads the committed fixture. `conformance.sh` regenerates the schema from the module it just published and diffs it against the committed copy, ignoring line 1's source URL whose port follows `STDB_PORT`. That mutation now fails the run. `dev-server.sh` no longer rewrites the fixture on every publish: a clobber can never detect drift, because it destroys the copy it would have to compare against. ## Verification `6 passed, 0 failed`; stylua clean. **Native and HTML5 conformance both PASS**, including PHASE5's identity-keyed check, with the new schema diff green inside the native run. Reverting the guard fails three files by name. The review's mutation sweep found two survivors and both are closed: two undescribed relations in one frame are now pinned in `test_cache.lua` (two entries, wire order, own row counts) and in `test_connection.lua` (two notices), so `skipped[#skipped + 1]` → `skipped[1]` fails both files; and the generator's wrong-ref mutation fails the conformance diff. ## On #42's acceptance wording #42's Acceptance line asks for *"a test that an unknown relation **raises** rather than storing an empty row."* Nothing raises, and that is deliberate: #42's own Work bullet authorises the deviation — *"should raise, **or emit a distinct event**"* — and the table above is why the event is the better half of that latitude. Recorded here because the issue text will outlive the PR. ## Frozen surface - `undescribed`, on `counts`, as an event, and now on the Subscription handle: one concept, one name. It mirrors `unhandled` for an unknown message tag — same class of problem, same bare-adjective form. - Payload `{ type, table, rows, reason }`. `rows` is a **count**, not a list: there is nothing to hand over, since the rows were never decoded, and it is the vocabulary `counts` already uses for `inserts` and `deletes`. Documented as a count at both surfaces, because the sibling `row` on an insert or delete event *is* a row. - `counts` stays additive: with nothing skipped the keys are exactly `deletes, events, inserts`, and no consumer anywhere iterates `pairs(counts)`.

yuris added 1 commit 2026-08-20 03:52:28 +00:00

cache: skip a relation the schema cannot describe, and emit views (#42) ... 3b19e55942

Subscribing to a view stored one fieldless row per projection and counted
each as an insert, with no error anywhere. `table_def` returned `{}` for an
unknown relation, so `def.columns` was nil, and `protocol.decode_row` with no
columns is total loss: `columns` defaults to `{}`, both of its loops become
`ipairs({})`, and it returns `{}` for any input at all — a named readback row
included. Reachable by an ordinary `SELECT * FROM my_view`, and the same
failure class this project has spent #8, #17, #21, #26 and #46 eliminating.

Downstream it was worse than empty. A pk-less `remove` scans for a row equal
to the one being deleted field by field; `pairs({})` never falsifies the
match, so the first candidate always won and a delete evicted the LAST row of
the relation — even a delete of a row the relation never held.

SKIP AND REPORT, NOT RAISE. `apply_database_update` now skips such a relation
whole and records it in `counts.undescribed`, which `emit_rows` turns into an
`undescribed` event. A raise would have aborted the frame's table loop: the
relations listed after the undescribed one would never be applied, `counts`
would never be returned, so `emit_rows` never runs and a `subscribe` callback
parks forever. Subscribing to one view would therefore corrupt the tables the
consumer DID describe — strictly worse than the bug being fixed — and it would
repeat on every reconnect, since the same frame arrives again. `cache.find`
and `schema.new` raise because a CALLER made a mistake at a callsite with a
stack; here the server sent something the schema cannot describe, which is the
same class of problem as an unknown message tag, and gets the same answer.
test_cache.lua had already pinned "unknown table does not raise — the server
can send tables the client didn't declare"; that decision stands.

Recovering field names from a named readback was considered and rejected. It
would make one relation's behaviour depend on which message carried the row —
populated on subscribe, then silently stale, because its TransactionUpdate
rows stay positional and undecodable. Every row of a view is positional in
every message anyway, so for the case #42 is about there is no named half to
recover.

"No definition" is two causes, kept apart because they point at different
mistakes. No entry at all is ordinary and supported: a hand-written schema
describes the tables its consumer uses and no others, and the headless
conformance run is exactly that — it omits `profile`, `all_types` and the new
view, and passes unchanged, PHASE5 included. An entry with no columns is an
author slip, so `schema.new` now rejects `columns = {}` at construction, and
the cache still reports it for the unvalidated `cache.new(defs)` spelling.
`counts.undescribed` is absent when nothing was skipped, so a consumer
subscribed only to relations it describes sees a byte-identical `counts` and
no extra events.

VIEWS ARE NOW EMITTED, because the capture made it small. gen-schema.sh reads
`misc_exports[].View`, resolves `return_type`'s `{Array: {Ref: n}}` to that
product for the columns, order and types, seeds the typespace from them, and
writes the entry into `tables` with `view = true` — so `schema.new(gen.tables)`
covers a view with no consumer change. Only the Array form: `spacetimedb.view`
also accepts an Option return for a view yielding at most one row, nothing here
has seen one on the wire, and it is left undescribed where the guard above
reports it rather than described by inference. A view's own primary-key column
does not reach the schema endpoint at all, so no `pk` is emitted rather than a
guess; `view = true` is what lets tables.lua's `on_update` raise say "view"
instead of sending a reader after an annotation that cannot help.

PROVEN ON THE WIRE, not argued. `dev/server/index.js` gains a view over
`message` projecting it as (text, sender, id) — deliberately reversed, so a
decoder reusing the table's order lands a u64 in `text`. Frames 37-42 are a
third capture block, appended so 1-36 stay byte-identical, and they settle
three things: a view's readback rows are POSITIONAL where a table's are named
(`["before subscribe","yuris",1]` in a SubscribeMultiApplied), a view IS
incrementally maintained (a TransactionUpdate carries the new projection under
the view's name), and an unsubscribe returns its rows as positional deletes.
The tests assert against those bytes, on both a schema that describes the view
and one that does not.

Also: capture-fixtures.mjs now truncates at the Nth IdentityToken so a third
block is re-capturable, and `protocol.decode_row`/`decode_typed` document that
an empty column list is total loss rather than lenient.

The generated schema fixture and session.jsonl were captured from port 4900
(own state dir), not the shared 3000 server, which is why the provenance URL in
chat_schema.lua's header moved.

Verified: 6 passed, 0 failed; stylua clean; native and HTML5 conformance both
PASS including PHASE5. Reverting the guard fails three test files by name
("the skip was reported at all", "an undescribed relation emits its own
event", "an undescribed view stores nothing"). 16 mutations of the new logic —
each half of the guard, the row accounting, the report shape, the event and its
ordering, the schema.new rejection, the view flag and the generated column
order — were all caught; two survivors found gaps in the new tests and both
gaps were closed.

yuris commented 2026-08-20 04:12:56 +00:00

Author

Owner

Copy link

Review — automated reviewer, posted for the record

Review — PR #49 feat/42-views (closes #42)

Verdict: closes #42. No blockers. Four should-fixes, three nits. The core fix is correct, verified against live 2.8.2 wire bytes, and well tested.

Suite 6 passed, 0 failed; stylua clean; native conformance PASS (both runs, incl. PHASE5); HTML5 conformance PASS (incl. PHASE5). #39's phase-2 flake did not trigger.

---

1. Should-fixes

S1. The subscribe callback reports unqualified success while every row is dropped

stdb/connection.lua:658 (InitialSubscription), :691 (subscription_applied) — severity should-fix

The PR argues the skip is "audible without a raise" via on_event. It is — if you registered on_event. But the natural way to consume a view is subscribe_query({"SELECT * FROM my_view"}, cb), and that path is silent. Verified:

subscribe callback: err=nil   msg carries counts=false
sub:on_applied  : handle carries counts=false
sub:is_active() = true

The consumer is told the subscription succeeded, the handle goes active, and every row was discarded. on_event defaults to function() end (connection.lua:19) and there is no logging facility anywhere in stdb/*.lua — no print, no sys.*, nothing in cache/connection/schema/tables. So for a non-listener the loss is undiscoverable.

This contradicts the repo's own doctrine, stated in tables.lua:115-118 to justify on_update raising: silence is indistinguishable from "the server sent nothing yet". Here the SDK chose silence on the one path where the consumer is provably present and holding a callback.

Fix (additive, non-breaking): do not touch callback(err, msg) — connection.lua:739 freezes that signature. Instead set the report on the Subscription handle (sub.undescribed) in subscription_applied, where counts is already in scope at :691, and document it on on_applied. A view subscriber holds that handle by construction.

S2. The generator's view-order resolution has no automated guard

dev/tools/gen-schema.sh:331 — severity should-fix

This is the one thing whose failure mode is silent mis-mapping, i.e. worse than the bug being fixed. It is unguarded. Changing line 331 from ($v.return_type.Array.Ref) as $ref to (0) as $ref regenerates:

["item_flip"] = { columns = { "id", "alpha", "beta" }, ... }   -- the TABLE's order

…the exact corruption #42 exists to prevent — and the suite still passes 6/6. The Lua suite only reads the committed chat_schema.lua; nothing re-derives it from the tool.

Mitigating, and worth crediting: the committed fixture is a faithful witness. I regenerated chat_schema.lua from a live server and it reproduced byte-for-byte except line 1's URL comment. And test_fixtures.lua does pin the order — flipping the fixture's view columns to the table's order is CAUGHT.

Fix: conformance.sh already regenerates the schema. Have it diff the result against the committed copy (ignoring line 1) and fail on drift. One diff closes the only silent-corruption path left in the change.

S3. The stated reason for omitting the Option-returning view is now false

dev/tools/gen-schema.sh:160-162 and :321-324 — severity should-fix (docs, since 1.0 freezes them)

Both comments say the Option form is left undescribed because "NOTHING in this repo has ever seen one on the wire". I put one on the wire. SELECT * FROM item_one on a live 2.8.2 delivered a row:

SubscribeMultiApplied → item_one → ["a1",4097]

It is reachable by an ordinary select, exactly like the view in #42's own headline. It is also fully describable: the endpoint reports return_type.Sum.variants[0].algebraic_type.Ref = 9, which resolves to Product{only_alpha: String, id: U64} — as resolvable as the Array form, via the same columns/coltypes helpers.

The loud path is correct behaviour for it today, and I confirmed it works end to end: undescribed table=item_one reason="no schema entry" rows=1, nothing cached. So this is not a correctness bug. But the justification is falsified, and it is about to be frozen. Either emit it (resolve the some variant's Ref — a one-line jq addition) or restate the reason as a deliberate scope decision rather than an empirical claim that is no longer true.

S4. Two undescribed relations in one frame is untested — mutation 7 survived

stdb/cache.lua:346 — severity should-fix (test gap only)

Mutating skipped[#skipped + 1] = {...} to skipped[1] = {...} SURVIVED the suite. That collapses every skip report in a frame into one slot, so a consumer subscribed to two undescribed relations hears about only one — breaking the "one notice per skipped relation" promise the README makes and test_connection.lua:263 asserts (but only for a single relation).

The behaviour is correct; only the coverage is missing. Verified with a frame carrying ghost_a, item, ghost_b:

undescribed entries: 2
  [1] ghost_a rows=1
  [2] ghost_b rows=1
events: undescribed:ghost_a, undescribed:ghost_b, subscribed

Fix: one frame with two distinct undescribed relations, asserting #counts.undescribed == 2 and that entries [1]/[2] name each relation in wire order. That kills the mutation and pins the documented promise.

---

2. Nits

N1. schema.lua:32-34 contradicts schema.new's own validation. The header says a view "has no primary key and cannot have one", but schema.new({v = {columns={"a","b"}, view=true, pk="a"}}) is accepted, and the view then gets pk pairing and a working on_update (verified: events=1 type=update). The behaviour is benign and arguably a useful escape hatch for a hand-written schema whose view projects its table's key — so keep it and fix the comment. The generator never emits pk on a view, so nothing generated is affected.

N2. evt.rows is a count in an API where row is a row. Payload is { type, table, rows, reason }, but counts.events[].row is the row itself. evt.rows reading as "the rows" is a live trap. row_count or skipped avoids it. Frozen if unfixed.

N3. chat_schema.lua:1 records an ad-hoc port. Line 1 now says http://127.0.0.1:4900, which appears nowhere else in the repo (gen-schema.sh defaults 3000, dev-server.sh uses STDB_PORT:-3000). Anyone regenerating on the default port dirties the file again. Regenerate against 3000 before merge.

---

3. The skip-and-report decision — reasoning verified, all three limbs

"A raise drops the rows of relations the consumer did describe" — TRUE, and worse than claimed: it is order-dependent. I patched cache.lua to raise and replayed the same frame in both orders:

frame order	raise?	described item rows
ghost first, item second	yes	0 — lost
item first, ghost second	yes	1 — survives
current code (skip), either order	no	1, always

Since the order of db_update.tables is the server's choice, a raise would lose described rows nondeterministically. That is a stronger argument than the PR makes.

"Parks the subscribe callback" — TRUE by construction. connection.lua:658 applies the update before clearing c.pending[msg.request_id] and calling p.callback(nil, msg) at :659-663. A raise at 658 leaves the pending entry forever and never fires the callback. Same shape at :691.

"No pcall at any of the four call sites" — TRUE. Sites are connection.lua:658, 691, 742, 781; the file's only pcall is :597 (settle_one, unrelated). handle_frame (:889) has none, and init.lua:201 calls conn:handle_frame(evt.message) bare inside the extension-websocket callback. A raise would reach Defold's callback and kill the connection.

graph LR
  A[frame with ghost + item] --> B{undescribed?}
  B -->|skip| C[item applied, ghost reported, callback fires]
  B -->|raise| D[tables loop aborts]
  D --> E[later relations dropped]
  D --> F[counts never returned]
  F --> G[emit_rows never runs]
  G --> H[subscribe callback parked forever]

Judgement: right to freeze. But the strongest argument is not the one the PR makes. The PR leans on the event; the real case is that the old behaviour actively corrupted state reachable through the ordinary read API. Running origin/main's cache against a view frame:

OLD: view insert -> inserts=1  cached view rows=1  undescribed=nil
  stored row has 0 fields; row.text = nil
OLD: delete of a row the view NEVER held -> deletes=1, view rows now 0

So count() reported a row that had no fields, iter() handed out a table where every access is nil, and a delete of a row the relation never held evicted the row it did hold. The phantom delete is confined to the undescribed relation's own store — a bogus delete against a described pk-less table did not evict, so the PR's "evicts the LAST row of the relation" is precise and does not cross relation boundaries.

Is a consumer who never registers on_event genuinely better off? Yes — unambiguously. Before: count()/iter() lied. After: the relation is simply empty, which is honest and inert. Silence plus an accurate absence beats silence plus fieldless garbage that answers count() with a falsehood. That holds with no handler registered at all. But nothing makes the loss discoverable for that consumer (see S1) — and the subscribe-callback case is worse than merely undiscoverable, because it affirmatively reports success.

The absent-vs-columnless boundary — a hand-written schema can break, but only one already broken

case	before	now
relation absent from schema	{} rows stored + counted	skipped, reason = "no schema entry"
entry present, columns absent	same	skipped, "schema entry has no columns"
entry present, columns = {} via schema.new	accepted, then every row → {}	raises (schema.lua:92-100)
entry present, columns = {} via bare cache.new	same	skipped, "schema entry has no columns"

Omitting tables entirely still works, which is the documented hand-written contract. The only new rejection is columns = {} — a spelling that could never have worked, since it decoded every row of that relation to a fieldless table. It now fails at construction, loudly, with a message naming the list. Correct call: a hand-written setup that was genuinely working cannot break, because it either named its columns (unaffected) or omitted the table (unaffected).

---

4. The generator's view emission — verified end to end on my own server

Published a probe module on a private 2.8.2 (port 5000, own state dir — the shared 3000 was untouched): table item (id, alpha, beta) with views deliberately mis-ordered.

Column order is correct, and the field names land on the right values. Generated:

["item_flip"] = { columns = { "beta", "alpha", "id" }, types = { "string", "string", "u64" }, view = true },

Wire: view readback ["b1","a1",1] positional; the table's readback named {"id":1,"alpha":"a1","beta":"b1"}. Decoded through schema.new(gen.tables) + cache.apply_database_update:

ok item_flip.beta = b1        ok item_flip.alpha = a1
ok item_flip.id = 1           ok item_flip.id type = number
ALL FIELD MAPPINGS CORRECT

The view's own order, not the table's — no mis-mapping. Had the table's order been used, beta would hold 1 and id would hold "b1".

• Join + computed column: item_join → columns = {label, combined, alpha_len, id}, types = {string, string, i32, u64}; wire ["tag-one","a1/b1",2,1]; decoded correctly. Columns existing in no single table work automatically, because the resolution is of the view's declared product.
• types for a view column: identical vocabulary to a table's. A nested product/sum column also seeds the typespace: item_nested → types = {product_Point, sum_Status, u64} with Point/Status emitted. The diff's added seeding line (gen-schema.sh:376-377) is load-bearing — without it those tags would name nothing.
• Name collision with a table: impossible. The server rejects it at publish — Error: name 'item' is used for multiple entities. So $tables + $views concatenating into one Lua constructor (:439) cannot produce a duplicate key that silently overrides a table. Worth knowing the safety is the server's, not the generator's, but it holds.
• Option form: correctly excluded — the endpoint reports {Sum: {variants:[some,none]}}, not {Option:…}, so select(.return_type.Array.Ref != null) skips it. See S3.

---

5. The frozen surface

Name undescribed: good enough to freeze. It mirrors the unhandled precedent for an unknown message tag — same class of problem, same bare-adjective form, and that parallel is exactly the argument the PR makes. "Undescribed" is also more accurate than "unknown": the server knows the relation fine; this schema does not describe it.

Payload { type, table, rows, reason }: see N2 on rows. Two smaller points, both permanent: reason is a human sentence and consumers must string-match it to tell the two causes apart (README documents both literals) — a stable code plus the prose would age better; and table_id is on the wire but not in the payload, which is the only stable handle if a name is ever reused. Neither is worth blocking on, but both deserve a decision rather than a default.

Every counts consumer is additive-only — confirmed. With nothing skipped, counts keys are exactly deletes, events, inserts and undescribed = nil, byte-identical to before. No consumer anywhere iterates pairs(counts), so a new optional key cannot be observed by accident. Full sweep: smoke.script:174,303; test_tables.lua:395-397; test_connection.lua:131; test_fixtures.lua (many); test_cache.lua; README:204,238,501,509.

• #21's pairing: pair_by_key sits inside the new else branch (cache.lua:357), reachable only for described relations. Semantics unchanged — test_fixtures.lua:431-439 still asserts one update event with inserts=1, deletes=1.
• #28's index maintenance: rides inside store/remove, also else-only. Index structures are built in cache.new from def.unique; a skipped relation has no def, so there is nothing to maintain and nothing half-maintained.
• tables.dispatch: takes evt.counts.events, initialized at cache.lua:329 and always present — including on the malformed-db_update early return at :330-332. All four emit_rows sites pass a real counts.
• Lookup paths on a skipped relation: count→0, get→nil, iter→0 rows, find→ raises table "ghost" has no unique column "id". table_def's {} really is safe for lookups, as its new doc comment claims.

Should a view = true def get a db.tables.<name> facade? Yes — and all six behave sanely. tables.install builds one per c.state.schema entry, so views get one automatically. On a pk-less view:

method	behaviour	verdict
count()	2	correct
iter()	2 rows	correct
find(col, val)	raises has no unique column	correct — a view never gets unique, so find is permanently unavailable; the raise is clear and matches pk-less tables
on_insert / on_delete	register; dispatch fires del:b1, ins:b3	correct
on_update	raises view "v" has no primary key…	correct, and the view wording works

Removing the facade would be wrong: count/iter/on_insert/on_delete are precisely what a view consumer needs and are most of the surface. The view = true flag earns its keep in exactly one place — the require_pk message — but that place matters, and mutation 6 confirms it is pinned.

---

6. Fixture integrity — clean

• Frames 1–36 byte-identical. cmp of git show origin/main:…session.jsonl against head -36 of HEAD → exit 0. Numstat 6 0: six insertions, zero deletions. The capture tool's truncate-and-reappend hazard did not materialize.
• 36 → 42 lines. New frames in order: IdentityToken, TransactionUpdate (clear_all), TransactionUpdate (send_message, not yet subscribed), SubscribeMultiApplied (message_flip, ["before subscribe","yuris",1]), TransactionUpdate (incremental message_flip), UnsubscribeMultiApplied (both rows as deletes).
• chat_schema.lua: 8 insertions, 1 deletion. The lone deletion is line 1's provenance comment (N3). All four pre-existing tables (all_types, message, player, profile) keep character-identical columns/types/pk/unique/indexes; typespace (Kind, Point, Status) byte-identical. Only real addition is the message_flip entry. (No new typespace entries — the view's FlipRow lives in index.js, not the fixture.)
• All 8 reducers and 3 procedures survive in both chat_schema.lua and dev/server/index.js; index.js is +22/−0, a pure addition.
• Frame 24 still carries exactly 2 updates entries on message (#44's regression evidence) — re-measured with jq.
• Frames 33/34 still the identity-keyed player case (#35's PHASE5): identity-typed identity/sponsor pair, score 0 → 42.

---

7. Sanity and the mutation sweep

Suite 6 passed, 0 failed. stylua --check . clean. Native conformance PASS both runs (fresh token, then token-less identity persistence across a process restart), PHASE5 included. HTML5 conformance PASS, PHASE5 included.

Spot-checked 12 mutations: 11 caught, 1 survived.

#	mutation	result
1	guard revert (why = nil; restore pre-#42)	CAUGHT — exactly three files by name: test_cache.lua, test_connection.lua, test_fixtures.lua. Claim verified precisely.
2	undecodable empty-columns branch unreachable	CAUGHT (test_cache.lua)
3	row tally drops the deletes term	CAUGHT (test_cache.lua)
4	undescribed emitted after, not before, the frame event	CAUGHT (test_connection.lua)
5	schema.new empty-columns rejection removed	CAUGHT (test_cache.lua)
6	require_pk loses the view wording	CAUGHT (test_tables.lua)
8	the two reason strings made identical	CAUGHT (three files)
9	skip stops reporting (silent drop, still no raise)	CAUGHT (three files)
10	undescribed event drops the table name	CAUGHT (test_connection.lua)
12	fixture's view column order flipped to the table's	CAUGHT (test_fixtures.lua)
7	skipped[#skipped+1] → skipped[1]	SURVIVED → S4
11	gen-schema.sh resolves the wrong typespace ref	SURVIVED → S2

The author's headline claim — "reverting the guard fails three files by name" — is exact.

---

8. Does it close #42?

Yes. All five Work items are done: the loud path, the generator emission (verified live), the view = true marking, the index.js view plus fixture capture, and an explicit, documented decision to store view rows as an append-only list.

One acceptance line is met in substance rather than literally, and it is worth stating in the PR body since the issue text will outlive it. #42's Acceptance asks for "a test that an unknown relation raises rather than storing an empty row." Nothing raises. But #42's own Work bullet authorizes the deviation — "should raise, or emit a distinct event" — and the deviation is verifiably better than the raise would have been (§3). Recommend the PR body note that the Acceptance wording was consciously superseded by the Work bullet's latitude, so a future reader does not read the mismatch as an oversight.

Nothing I found causes data loss or silent mis-mapping in the shipped code. S2 is a future risk, not a present defect — the generator is correct today and I proved it on live bytes; it is simply unguarded against regression.

## Review — automated reviewer, posted for the record # Review — PR #49 `feat/42-views` (closes #42) **Verdict: closes #42. No blockers.** Four should-fixes, three nits. The core fix is correct, verified against live 2.8.2 wire bytes, and well tested. Suite `6 passed, 0 failed`; stylua clean; native conformance **PASS** (both runs, incl. PHASE5); HTML5 conformance **PASS** (incl. PHASE5). #39's phase-2 flake did not trigger. --- ## 1. Should-fixes ### S1. The `subscribe` callback reports unqualified success while every row is dropped `stdb/connection.lua:658` (`InitialSubscription`), `:691` (`subscription_applied`) — severity **should-fix** The PR argues the skip is "audible without a raise" via `on_event`. It is — *if* you registered `on_event`. But the natural way to consume a view is `subscribe_query({"SELECT * FROM my_view"}, cb)`, and that path is silent. Verified: ``` subscribe callback: err=nil msg carries counts=false sub:on_applied : handle carries counts=false sub:is_active() = true ``` The consumer is told the subscription succeeded, the handle goes active, and every row was discarded. `on_event` defaults to `function() end` (`connection.lua:19`) and there is **no logging facility anywhere in `stdb/*.lua`** — no `print`, no `sys.*`, nothing in cache/connection/schema/tables. So for a non-listener the loss is undiscoverable. This contradicts the repo's own doctrine, stated in `tables.lua:115-118` to justify `on_update` raising: *silence is indistinguishable from "the server sent nothing yet"*. Here the SDK chose silence on the one path where the consumer is provably present and holding a callback. **Fix (additive, non-breaking):** do **not** touch `callback(err, msg)` — `connection.lua:739` freezes that signature. Instead set the report on the Subscription handle (`sub.undescribed`) in `subscription_applied`, where `counts` is already in scope at `:691`, and document it on `on_applied`. A view subscriber holds that handle by construction. ### S2. The generator's view-order resolution has no automated guard `dev/tools/gen-schema.sh:331` — severity **should-fix** This is the one thing whose failure mode is *silent mis-mapping*, i.e. worse than the bug being fixed. It is unguarded. Changing line 331 from `($v.return_type.Array.Ref) as $ref` to `(0) as $ref` regenerates: ```lua ["item_flip"] = { columns = { "id", "alpha", "beta" }, ... } -- the TABLE's order ``` …the exact corruption #42 exists to prevent — and **the suite still passes 6/6**. The Lua suite only reads the committed `chat_schema.lua`; nothing re-derives it from the tool. Mitigating, and worth crediting: the committed fixture *is* a faithful witness. I regenerated `chat_schema.lua` from a live server and it reproduced byte-for-byte except line 1's URL comment. And `test_fixtures.lua` does pin the order — flipping the fixture's view columns to the table's order is CAUGHT. **Fix:** `conformance.sh` already regenerates the schema. Have it `diff` the result against the committed copy (ignoring line 1) and fail on drift. One `diff` closes the only silent-corruption path left in the change. ### S3. The stated reason for omitting the Option-returning view is now false `dev/tools/gen-schema.sh:160-162` and `:321-324` — severity **should-fix** (docs, since 1.0 freezes them) Both comments say the Option form is left undescribed because "NOTHING in this repo has ever seen one on the wire". I put one on the wire. `SELECT * FROM item_one` on a live 2.8.2 delivered a row: ``` SubscribeMultiApplied → item_one → ["a1",4097] ``` It is reachable by an ordinary select, exactly like the view in #42's own headline. It is also **fully describable**: the endpoint reports `return_type.Sum.variants[0].algebraic_type.Ref = 9`, which resolves to `Product{only_alpha: String, id: U64}` — as resolvable as the Array form, via the same `columns`/`coltypes` helpers. The loud path *is* correct behaviour for it today, and I confirmed it works end to end: `undescribed table=item_one reason="no schema entry" rows=1`, nothing cached. So this is not a correctness bug. But the justification is falsified, and it is about to be frozen. Either emit it (resolve the `some` variant's `Ref` — a one-line jq addition) or restate the reason as a deliberate scope decision rather than an empirical claim that is no longer true. ### S4. Two undescribed relations in one frame is untested — mutation 7 survived `stdb/cache.lua:346` — severity **should-fix** (test gap only) Mutating `skipped[#skipped + 1] = {...}` to `skipped[1] = {...}` **SURVIVED** the suite. That collapses every skip report in a frame into one slot, so a consumer subscribed to two undescribed relations hears about only one — breaking the "one notice per skipped relation" promise the README makes and `test_connection.lua:263` asserts (but only for a single relation). The **behaviour is correct**; only the coverage is missing. Verified with a frame carrying `ghost_a`, `item`, `ghost_b`: ``` undescribed entries: 2 [1] ghost_a rows=1 [2] ghost_b rows=1 events: undescribed:ghost_a, undescribed:ghost_b, subscribed ``` **Fix:** one frame with two distinct undescribed relations, asserting `#counts.undescribed == 2` and that entries `[1]`/`[2]` name each relation in wire order. That kills the mutation and pins the documented promise. --- ## 2. Nits **N1. `schema.lua:32-34` contradicts `schema.new`'s own validation.** The header says a view "has no primary key **and cannot have one**", but `schema.new({v = {columns={"a","b"}, view=true, pk="a"}})` is accepted, and the view then gets pk pairing and a working `on_update` (verified: `events=1 type=update`). The behaviour is benign and arguably a useful escape hatch for a hand-written schema whose view projects its table's key — so keep it and fix the comment. The generator never emits `pk` on a view, so nothing generated is affected. **N2. `evt.rows` is a count in an API where `row` is a row.** Payload is `{ type, table, rows, reason }`, but `counts.events[].row` is the row *itself*. `evt.rows` reading as "the rows" is a live trap. `row_count` or `skipped` avoids it. Frozen if unfixed. **N3. `chat_schema.lua:1` records an ad-hoc port.** Line 1 now says `http://127.0.0.1:4900`, which appears nowhere else in the repo (`gen-schema.sh` defaults 3000, `dev-server.sh` uses `STDB_PORT:-3000`). Anyone regenerating on the default port dirties the file again. Regenerate against 3000 before merge. --- ## 3. The skip-and-report decision — reasoning verified, all three limbs **"A raise drops the rows of relations the consumer did describe" — TRUE, and worse than claimed: it is order-dependent.** I patched `cache.lua` to raise and replayed the same frame in both orders: | frame order | raise? | described `item` rows | |---|---|---| | `ghost` first, `item` second | yes | **0 — lost** | | `item` first, `ghost` second | yes | 1 — survives | | **current code (skip), either order** | no | **1, always** | Since the order of `db_update.tables` is the server's choice, a raise would lose described rows *nondeterministically*. That is a stronger argument than the PR makes. **"Parks the subscribe callback" — TRUE by construction.** `connection.lua:658` applies the update *before* clearing `c.pending[msg.request_id]` and calling `p.callback(nil, msg)` at `:659-663`. A raise at 658 leaves the pending entry forever and never fires the callback. Same shape at `:691`. **"No `pcall` at any of the four call sites" — TRUE.** Sites are `connection.lua:658, 691, 742, 781`; the file's only `pcall` is `:597` (`settle_one`, unrelated). `handle_frame` (`:889`) has none, and `init.lua:201` calls `conn:handle_frame(evt.message)` bare inside the extension-websocket callback. A raise would reach Defold's callback and kill the connection. ```mermaid graph LR A[frame with ghost + item] --> B{undescribed?} B -->|skip| C[item applied, ghost reported, callback fires] B -->|raise| D[tables loop aborts] D --> E[later relations dropped] D --> F[counts never returned] F --> G[emit_rows never runs] G --> H[subscribe callback parked forever] ``` **Judgement: right to freeze.** But the strongest argument is not the one the PR makes. The PR leans on the event; the real case is that the **old behaviour actively corrupted state reachable through the ordinary read API**. Running `origin/main`'s cache against a view frame: ``` OLD: view insert -> inserts=1 cached view rows=1 undescribed=nil stored row has 0 fields; row.text = nil OLD: delete of a row the view NEVER held -> deletes=1, view rows now 0 ``` So `count()` reported a row that had no fields, `iter()` handed out a table where every access is `nil`, and a delete of a row the relation never held evicted the row it did hold. The phantom delete is confined to the undescribed relation's own store — a bogus delete against a *described* pk-less table did **not** evict, so the PR's "evicts the LAST row of the relation" is precise and does not cross relation boundaries. **Is a consumer who never registers `on_event` genuinely better off? Yes — unambiguously.** Before: `count()`/`iter()` lied. After: the relation is simply empty, which is honest and inert. Silence plus an accurate absence beats silence plus fieldless garbage that answers `count()` with a falsehood. That holds with no handler registered at all. **But nothing makes the loss discoverable for that consumer** (see S1) — and the `subscribe`-callback case is worse than merely undiscoverable, because it affirmatively reports success. ### The absent-vs-columnless boundary — a hand-written schema can break, but only one already broken | case | before | now | |---|---|---| | relation absent from schema | `{}` rows stored + counted | skipped, `reason = "no schema entry"` | | entry present, `columns` absent | same | skipped, `"schema entry has no columns"` | | entry present, `columns = {}` via `schema.new` | **accepted**, then every row → `{}` | **raises** (`schema.lua:92-100`) | | entry present, `columns = {}` via bare `cache.new` | same | skipped, `"schema entry has no columns"` | Omitting tables entirely still works, which is the documented hand-written contract. The **only** new rejection is `columns = {}` — a spelling that could never have worked, since it decoded every row of that relation to a fieldless table. It now fails at construction, loudly, with a message naming the list. **Correct call:** a hand-written setup that was genuinely working cannot break, because it either named its columns (unaffected) or omitted the table (unaffected). --- ## 4. The generator's view emission — verified end to end on my own server Published a probe module on a private 2.8.2 (port 5000, own state dir — the shared 3000 was untouched): table `item (id, alpha, beta)` with views deliberately mis-ordered. **Column order is correct, and the field names land on the right values.** Generated: ```lua ["item_flip"] = { columns = { "beta", "alpha", "id" }, types = { "string", "string", "u64" }, view = true }, ``` Wire: view readback `["b1","a1",1]` **positional**; the table's readback **named** `{"id":1,"alpha":"a1","beta":"b1"}`. Decoded through `schema.new(gen.tables)` + `cache.apply_database_update`: ``` ok item_flip.beta = b1 ok item_flip.alpha = a1 ok item_flip.id = 1 ok item_flip.id type = number ALL FIELD MAPPINGS CORRECT ``` The view's own order, not the table's — no mis-mapping. Had the table's order been used, `beta` would hold `1` and `id` would hold `"b1"`. - **Join + computed column:** `item_join` → `columns = {label, combined, alpha_len, id}`, `types = {string, string, i32, u64}`; wire `["tag-one","a1/b1",2,1]`; decoded correctly. Columns existing in no single table work automatically, because the resolution is of the view's *declared product*. - **`types` for a view column:** identical vocabulary to a table's. A nested product/sum column also **seeds the typespace**: `item_nested` → `types = {product_Point, sum_Status, u64}` with `Point`/`Status` emitted. The diff's added seeding line (`gen-schema.sh:376-377`) is load-bearing — without it those tags would name nothing. - **Name collision with a table:** **impossible.** The server rejects it at publish — `Error: name 'item' is used for multiple entities`. So `$tables + $views` concatenating into one Lua constructor (`:439`) cannot produce a duplicate key that silently overrides a table. Worth knowing the safety is the server's, not the generator's, but it holds. - **Option form:** correctly excluded — the endpoint reports `{Sum: {variants:[some,none]}}`, not `{Option:…}`, so `select(.return_type.Array.Ref != null)` skips it. See S3. --- ## 5. The frozen surface **Name `undescribed`: good enough to freeze.** It mirrors the `unhandled` precedent for an unknown message tag — same class of problem, same bare-adjective form, and that parallel is exactly the argument the PR makes. "Undescribed" is also more accurate than "unknown": the server knows the relation fine; this schema does not describe it. **Payload `{ type, table, rows, reason }`:** see N2 on `rows`. Two smaller points, both permanent: `reason` is a human sentence and consumers must string-match it to tell the two causes apart (README documents both literals) — a stable code plus the prose would age better; and `table_id` is on the wire but not in the payload, which is the only stable handle if a name is ever reused. Neither is worth blocking on, but both deserve a decision rather than a default. **Every `counts` consumer is additive-only — confirmed.** With nothing skipped, `counts` keys are exactly `deletes, events, inserts` and `undescribed = nil`, byte-identical to before. **No consumer anywhere iterates `pairs(counts)`**, so a new optional key cannot be observed by accident. Full sweep: `smoke.script:174,303`; `test_tables.lua:395-397`; `test_connection.lua:131`; `test_fixtures.lua` (many); `test_cache.lua`; `README:204,238,501,509`. - **#21's pairing:** `pair_by_key` sits inside the new `else` branch (`cache.lua:357`), reachable only for described relations. Semantics unchanged — `test_fixtures.lua:431-439` still asserts one update event with `inserts=1, deletes=1`. - **#28's index maintenance:** rides inside `store`/`remove`, also `else`-only. Index structures are built in `cache.new` from `def.unique`; a skipped relation has no def, so there is nothing to maintain and nothing half-maintained. - **`tables.dispatch`:** takes `evt.counts.events`, initialized at `cache.lua:329` and always present — including on the malformed-`db_update` early return at `:330-332`. All four `emit_rows` sites pass a real `counts`. - **Lookup paths on a skipped relation:** `count`→0, `get`→nil, `iter`→0 rows, `find`→ raises `table "ghost" has no unique column "id"`. `table_def`'s `{}` really is safe for lookups, as its new doc comment claims. **Should a `view = true` def get a `db.tables.<name>` facade? Yes — and all six behave sanely.** `tables.install` builds one per `c.state.schema` entry, so views get one automatically. On a pk-less view: | method | behaviour | verdict | |---|---|---| | `count()` | 2 | correct | | `iter()` | 2 rows | correct | | `find(col, val)` | raises `has no unique column` | correct — a view never gets `unique`, so `find` is permanently unavailable; the raise is clear and matches pk-less tables | | `on_insert` / `on_delete` | register; dispatch fires `del:b1, ins:b3` | correct | | `on_update` | raises `view "v" has no primary key…` | correct, and the `view` wording works | Removing the facade would be wrong: `count`/`iter`/`on_insert`/`on_delete` are precisely what a view consumer needs and are most of the surface. The `view = true` flag earns its keep in exactly one place — the `require_pk` message — but that place matters, and mutation 6 confirms it is pinned. --- ## 6. Fixture integrity — clean - **Frames 1–36 byte-identical.** `cmp` of `git show origin/main:…session.jsonl` against `head -36` of HEAD → exit 0. Numstat `6 0`: six insertions, **zero deletions**. The capture tool's truncate-and-reappend hazard did not materialize. - **36 → 42 lines.** New frames in order: `IdentityToken`, `TransactionUpdate` (clear_all), `TransactionUpdate` (send_message, not yet subscribed), `SubscribeMultiApplied` (`message_flip`, `["before subscribe","yuris",1]`), `TransactionUpdate` (incremental `message_flip`), `UnsubscribeMultiApplied` (both rows as deletes). - **`chat_schema.lua`: 8 insertions, 1 deletion.** The lone deletion is line 1's provenance comment (N3). All four pre-existing tables (`all_types`, `message`, `player`, `profile`) keep character-identical `columns`/`types`/`pk`/`unique`/`indexes`; `typespace` (`Kind`, `Point`, `Status`) byte-identical. Only real addition is the `message_flip` entry. (No new typespace entries — the view's `FlipRow` lives in `index.js`, not the fixture.) - **All 8 reducers and 3 procedures survive** in both `chat_schema.lua` and `dev/server/index.js`; `index.js` is +22/−0, a pure addition. - **Frame 24** still carries exactly **2** `updates` entries on `message` (#44's regression evidence) — re-measured with jq. - **Frames 33/34** still the identity-keyed `player` case (#35's PHASE5): identity-typed `identity`/`sponsor` pair, score 0 → 42. --- ## 7. Sanity and the mutation sweep Suite `6 passed, 0 failed`. stylua `--check .` clean. Native conformance **PASS** both runs (fresh token, then token-less identity persistence across a process restart), PHASE5 included. HTML5 conformance **PASS**, PHASE5 included. Spot-checked **12 mutations: 11 caught, 1 survived.** | # | mutation | result | |---|---|---| | 1 | **guard revert** (`why = nil`; restore pre-#42) | **CAUGHT — exactly three files by name:** `test_cache.lua`, `test_connection.lua`, `test_fixtures.lua`. Claim verified precisely. | | 2 | `undecodable` empty-columns branch unreachable | CAUGHT (`test_cache.lua`) | | 3 | row tally drops the `deletes` term | CAUGHT (`test_cache.lua`) | | 4 | `undescribed` emitted after, not before, the frame event | CAUGHT (`test_connection.lua`) | | 5 | `schema.new` empty-columns rejection removed | CAUGHT (`test_cache.lua`) | | 6 | `require_pk` loses the `view` wording | CAUGHT (`test_tables.lua`) | | 8 | the two `reason` strings made identical | CAUGHT (three files) | | 9 | skip stops reporting (silent drop, still no raise) | CAUGHT (three files) | | 10 | `undescribed` event drops the table name | CAUGHT (`test_connection.lua`) | | 12 | fixture's view column order flipped to the table's | CAUGHT (`test_fixtures.lua`) | | 7 | `skipped[#skipped+1]` → `skipped[1]` | **SURVIVED** → S4 | | 11 | `gen-schema.sh` resolves the wrong typespace ref | **SURVIVED** → S2 | The author's headline claim — *"reverting the guard fails three files by name"* — is exact. --- ## 8. Does it close #42? **Yes.** All five Work items are done: the loud path, the generator emission (verified live), the `view = true` marking, the `index.js` view plus fixture capture, and an explicit, documented decision to store view rows as an append-only list. One acceptance line is met in substance rather than literally, and it is worth stating in the PR body since the issue text will outlive it. #42's Acceptance asks for *"a test that an unknown relation **raises** rather than storing an empty row."* Nothing raises. But #42's own Work bullet authorizes the deviation — *"should raise, **or emit a distinct event**"* — and the deviation is verifiably better than the raise would have been (§3). Recommend the PR body note that the Acceptance wording was consciously superseded by the Work bullet's latitude, so a future reader does not read the mismatch as an oversight. Nothing I found causes data loss or silent mis-mapping in the shipped code. S2 is a *future* risk, not a present defect — the generator is correct today and I proved it on live bytes; it is simply unguarded against regression.

yuris referenced this pull request from a commit 2026-08-20 04:36:25 +00:00

views: report the skip on the subscribe handle, emit the Option form (#42)

yuris added 1 commit 2026-08-20 04:36:25 +00:00

views: report the skip on the subscribe handle, emit the Option form (#42) ... 4f3faf6e79

Review follow-up. Four should-fixes and three nits from the #49 review, none
of which reopens the skip-and-report decision — that was re-derived by
experiment and held.

TOLD SUCCESS WHILE EVERY ROW IS DROPPED. Subscribing to an undescribed
relation reported `err = nil`, moved the handle to `applied` and left
`is_active()` true, while the frame's rows went nowhere. Both of those are
correct — the server really did apply the set — so neither may change, and
`callback(err, msg)` is frozen. What was missing was any way to LEARN of the
loss from the one object such a consumer holds by construction, since
`on_event` defaults to a no-op and there is no logging facility anywhere in
`stdb/`. `subscription_apply` now writes `counts.undescribed` to
`sub.undescribed`: the same list, under the same name, on the handle. It is
written BEFORE `on_applied` fires, so a handler reading the handle sees the
report for the frame it was called for, and it is written on every apply, nil
included — the set re-applies after a reconnect, and a stale report would be
an accusation the handle could never withdraw. The legacy `M.subscribe` hands
back a bare request_id with no handle to carry it, and its docstring now says
so rather than implying parity.

THE GENERATOR IS NOW RE-DERIVED WHERE IT RUNS. Nothing checked the tool's
output against anything: mutating the view's typespace resolution so it
emitted the underlying TABLE's column order — the exact silent mis-mapping
#42 exists to prevent — passed 6/6, because the Lua suite only ever reads the
committed fixture. conformance.sh now regenerates the schema from the module
it just published and diffs it against the committed copy, ignoring line 1's
source URL, whose port follows STDB_PORT. That mutation now fails the run.

dev-server.sh no longer rewrites the fixture on every publish. A clobber can
never DETECT drift — it destroys the copy it would have to compare against —
and it is what wrote an ad-hoc port into the committed provenance line.

THE OPTION-RETURNING VIEW IS EMITTED, not excused. The comments claimed
nothing in this repo had ever seen one on the wire; the review put one there,
so the claim was false and about to be frozen. It is also describable: the
`some` variant of its two-variant sum points at a product exactly like the
Array form's, so `viewref` resolves either and everything downstream is
identical, seeding of the typespace included. Verified end to end against a
live 2.8.2 — `SELECT * FROM message_latest` delivers `["second",2]`,
positional in the view's own (text, id) order, and decodes through
`schema.new(gen.tables)` to text = "second" and id = 2 as a number. Reusing
the table's (id, sender, text) would have put the u64 in `text`. The dev
module gained that view so the second resolution hop stays covered by the
diff above rather than by argument.

TWO UNDESCRIBED RELATIONS IN ONE FRAME. `skipped[#skipped + 1]` collapsed to
`skipped[1]` survived the suite: a consumer subscribed to two views would
hear about one. The behaviour was already right; only the assertion was
missing. test_cache.lua now pins two entries in wire order with their own row
counts, and test_connection.lua pins the two notices the connection layer
makes of them. Both files fail under that mutation.

Nits. schema.lua said a view "cannot have" a primary key, but `schema.new`
accepts `view = true` with a `pk` and then pairs updates on it (verified: one
update event) — the behaviour is a useful escape hatch for a hand-written
schema whose view projects its table's key, so the comment moved to what is
actually true, that no GENERATED view carries one. `evt.rows` stays a count
and is now documented as one at both surfaces: it is the vocabulary `counts`
already uses for `inserts` and `deletes`, the event copies that entry
verbatim, and there is no list to hand over because the rows were never
decoded. The fixture's provenance line records the default port again.

6 passed, 0 failed; stylua clean; native conformance PASS both runs including
PHASE5, with the new schema diff green inside it.

yuris added 1 commit 2026-08-20 04:40:58 +00:00

views: name what the schema endpoint actually reports for a view (#42) ... 3de32c12e8

Three exactness fixes on top of the review follow-up.

The conformance drift check printed `cp <tmpfile> <fixture>` to refresh the
fixture, and that temp file lives in the nix shell TMPDIR — gone the moment
the failing run exits. It now prints the generate-and-format pipeline, which
is runnable while the server it just started is still up.

"The endpoint reports the name, params, visibility and return type of a View,
and nothing else" was close but not true: a View also carries `index` and
`is_anonymous`. The load-bearing half — that it carries no key of any kind —
is now what the comment says, in gen-schema.sh and in the README.

The README still said a view "cannot have" a primary key in one more place
than the schema.lua header did. A generated one never has one; a hand-written
entry may, and it is honoured.

yuris added 1 commit 2026-08-20 04:45:31 +00:00

gen-schema: point the stylua note at the tool that now runs it (#42) d8c7cd8a7b

yuris merged commit b336e946d1 into main 2026-08-20 04:47:23 +00:00

yuris referenced this pull request from a commit 2026-08-20 04:47:24 +00:00

cache: skip a relation the schema cannot describe, and emit views (#49)

yuris referenced this pull request 2026-08-20 04:49:29 +00:00

v1.0.0 release checklist #24

yuris referenced this pull request 2026-08-20 04:49:29 +00:00

v1.0.0 release checklist #24

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area:defold

Defold engine integration

  

area:docs

Documentation and examples

  

area:infra

Flake, CI, secretspec

  

area:protocol

Wire protocol conformance

  

area:testing

Test harness and fixtures

  

kind:bug

Something broken

  

kind:chore

Maintenance

  

phase:breadth

UnsubscribeMulti, procedures, BSATN

  

phase:core

Core SDK layers (protocol/cache/connection)

  

phase:hardening

Reconnect, token persistence, errors

  

phase:slice

Vertical slice: first playable end-to-end

No labels

area:defold

area:docs

area:infra

area:protocol

area:testing

kind:bug

kind:chore

phase:breadth

phase:core

phase:hardening

phase:slice

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

yuris/spacetimedb-defold!49

No description provided.