Codegen: emit column types, indexes and reducer signatures (foundation for 1.0) #25

New issue

Closed

opened 2026-08-19 15:42:13 +00:00 by yuris · 0 comments

yuris commented

2026-08-19 15:42:13 +00:00

Owner

dev/tools/gen-schema.sh fetches the full module schema and then throws away almost all of it. This is the foundation issue for 1.0: several other features are blocked on the generator keeping what it already has in hand.

The evidence

The generator reads .typespace.types[ref].Product.elements and takes only .name.some:

def columns($ref):
  (.typespace.types[$ref].Product.elements // [])
  | map(.name.some // "unnamed");

Every element carries its type right next to the name. Live 2.8.2 server, chat module:

[
  { "name": { "some": "id" },     "algebraic_type": { "U64": [] } },
  { "name": { "some": "sender" }, "algebraic_type": { "String": [] } },
  { "name": { "some": "text" },   "algebraic_type": { "String": [] } }
]

Also discarded: primary_key beyond [0], every unique constraint, every index, and reducer parameter types (only names are kept).

What to emit

Extend the generated module to carry, per table: column names and order (as now), pk, column types, unique constraints, and indexes. Per reducer and procedure: parameter names and types.

Type encoding in the generated Lua wants to be a compact string tag rather than nested tables — the Godot SDK settled on exactly this and it is the right shape for a table-driven decoder (a BSATN_TYPES map per row class, with opt_/vec_ prefixes for nesting). Something like:

message = {
  columns = { "id", "sender", "text" },
  types   = { "u64", "string", "string" },
  pk      = "id",
  unique  = { "id" },
}

Special types are structurally detectable

Confirmed against v2.8.2 (crates/sats/src/product_type.rs:14-18) and by compiling a probe against the sats serializer: each special type is a single-field product with a magic field name, so detection needs no name heuristics and no guessing.

Type	Detection rule
`Identity`	Product, one element named `__identity__` (U256)
`ConnectionId`	Product, one element named `__connection_id__` (U128)
`Timestamp`	Product, one element named `__timestamp_micros_since_unix_epoch__` (I64)
`TimeDuration`	Product, one element named `__time_duration_micros__` (I64)
`Uuid`	Product with the uuid magic tag
`ScheduleAt`	Sum with variants named `Interval` / `Time`
`Option<T>`	Sum with variants named `some` / `none`

Note that unit type variants serialize as {"U64":[]} — an empty array, not null — which the parser should encode explicitly rather than discover.

Why this is the foundation

Blocked on this issue: #23 (special-type conversion must know which columns are special), typed per-table facades, unique-index lookups, typed reducer stubs, and any future BSATN work — a wire codec is impossible without column types.

Acceptance

dev/test/fixtures/chat_schema.lua regenerates with types, unique constraints and reducer parameter types, and dev/test/test_fixtures.lua asserts them against the live-captured schema rather than a hand-written expectation. A table with a compound pk, a unique non-pk column, and an index must be covered — which means dev/server/index.js needs such a table, so add one.

`dev/tools/gen-schema.sh` fetches the full module schema and then throws away almost all of it. This is the foundation issue for 1.0: several other features are blocked on the generator keeping what it already has in hand. ## The evidence The generator reads `.typespace.types[ref].Product.elements` and takes **only** `.name.some`: ```jq def columns($ref): (.typespace.types[$ref].Product.elements // []) | map(.name.some // "unnamed"); ``` Every element carries its type right next to the name. Live 2.8.2 server, `chat` module: ```json [ { "name": { "some": "id" }, "algebraic_type": { "U64": [] } }, { "name": { "some": "sender" }, "algebraic_type": { "String": [] } }, { "name": { "some": "text" }, "algebraic_type": { "String": [] } } ] ``` Also discarded: `primary_key` beyond `[0]`, every unique constraint, every index, and reducer parameter **types** (only names are kept). ## What to emit Extend the generated module to carry, per table: column names and order (as now), pk, **column types**, unique constraints, and indexes. Per reducer and procedure: parameter names **and** types. Type encoding in the generated Lua wants to be a compact string tag rather than nested tables — the Godot SDK settled on exactly this and it is the right shape for a table-driven decoder (a `BSATN_TYPES` map per row class, with `opt_`/`vec_` prefixes for nesting). Something like: ```lua message = { columns = { "id", "sender", "text" }, types = { "u64", "string", "string" }, pk = "id", unique = { "id" }, } ``` ### Special types are structurally detectable Confirmed against v2.8.2 (`crates/sats/src/product_type.rs:14-18`) and by compiling a probe against the sats serializer: each special type is a **single-field product with a magic field name**, so detection needs no name heuristics and no guessing. | Type | Detection rule | |---|---| | `Identity` | Product, one element named `__identity__` (U256) | | `ConnectionId` | Product, one element named `__connection_id__` (U128) | | `Timestamp` | Product, one element named `__timestamp_micros_since_unix_epoch__` (I64) | | `TimeDuration` | Product, one element named `__time_duration_micros__` (I64) | | `Uuid` | Product with the uuid magic tag | | `ScheduleAt` | Sum with variants named `Interval` / `Time` | | `Option<T>` | Sum with variants named `some` / `none` | Note that unit type variants serialize as `{"U64":[]}` — an empty array, not `null` — which the parser should encode explicitly rather than discover. ## Why this is the foundation Blocked on this issue: #23 (special-type conversion must know which columns are special), typed per-table facades, unique-index lookups, typed reducer stubs, and any future BSATN work — a wire codec is impossible without column types. ## Acceptance `dev/test/fixtures/chat_schema.lua` regenerates with types, unique constraints and reducer parameter types, and `dev/test/test_fixtures.lua` asserts them against the live-captured schema rather than a hand-written expectation. A table with a compound pk, a unique non-pk column, and an index must be covered — which means `dev/server/index.js` needs such a table, so add one.

yuris referenced this issue

2026-08-19 15:43:23 +00:00

v1.0.0 release checklist #24

yuris referenced this issue

2026-08-19 15:43:56 +00:00

BSATN codec — the only route to the v2 message schema #9

yuris referenced this issue

2026-08-19 16:11:51 +00:00

Identity-keyed tables are broken in the cache (pk decodes to a table) #35

yuris referenced this issue from a commit

2026-08-19 16:11:58 +00:00

codegen: keep the module's type information (#25)