yuris/life-service
1
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages 1 Wiki Activity Actions

Two-Phase Commit Journaling & Idempotency Fingerprinting #5

New issue
Closed
opened 2026-09-22 17:58:53 +00:00 by yuris · 0 comments
yuris commented 2026-09-22 17:58:53 +00:00
Owner
Copy link

Summary

Implement the two-phase commit boundary around external mutations: commit a prepared operation intent and canonical request fingerprint in SQLite before remote dispatch, enforce idempotency replay versus conflict detection, and atomically commit the confirmed result or uncertainty state alongside timeline events upon remote completion.

Background & Technical Rationale

Interacting with remote CalDAV servers (such as Radicale) and external services inherently lacks distributed transaction guarantees. Network timeouts, server crashes, and process interruptions can occur at any point. To ensure the Life timeline remains consistent without phantom writes or duplicated actions:

  1. Separation of Failure Boundaries: Local journal commits and remote HTTP requests belong to separate failure boundaries. A local journal failure must strictly block any subsequent network dispatch: the service must NEVER perform an unjournaled remote write.
  2. Phase 1 (Prepared Intent Commit): Before any external HTTP request is sent, the service commits a row in operations with state = 'prepared', recording the canonical request fingerprint SHA-256 hash, actor, target resource, baseline ETag, and intended mutation payload.
  3. Phase 2 (Remote Dispatch): The conditional HTTP mutation (e.g. PUT with If-Match) is dispatched over the network. Crucially, no SQLite transaction is open during this network I/O.
  4. Phase 3 (Result Commit): Upon receiving the HTTP response (or transport error), a single local SQLite transaction atomically updates operations.state to applied, failed, or unknown, records the response payload, and appends the corresponding confirmed fact to life_events.
  5. Canonical Request Fingerprinting & Idempotency:
    • Every mutation request requires an Idempotency-Key (or CLI --operation-id).
    • The canonical fingerprint is a SHA-256 digest covering HTTP method, normalized route, sorted semantic JSON body, baseline ETags, and approval digest. Transport-level headers such as X-Request-ID and authorization tokens are strictly excluded.
    • If an incoming request presents an existing Idempotency-Key and an identical fingerprint, the service returns the recorded outcome without repeating the remote dispatch (verifying Scenario R01).
    • If an incoming request presents an existing Idempotency-Key but a divergent fingerprint, the service rejects the request immediately with idempotency_conflict (HTTP 409 / Exit code 4) without mutating state (verifying Scenario R02).

Exact Technical References & Specifications

Mutation Sequence Diagram

From docs/technical/architecture.md#mutation-sequence:

"This is not a distributed transaction. Remote success and local result commit are separate failure boundaries. A local journal failure blocks new remote writes; an uncertain or unjournaled remote outcome cannot be reported as complete."

sequenceDiagram
  participant U as User through agent/CLI
  participant S as Life service
  participant D as SQLite worker
  participant R as Radicale
  participant H as Hindsight
  U->>S: Approved write, operation ID, baseline ETag
  S->>R: Read current resource and validate baseline
  S->>D: Commit prepared intent
  S->>R: Conditional PUT (no SQLite transaction held)
  R-->>S: Success, conflict or uncertain transport outcome
  S->>D: Commit result event, operation state and optional outbox
  S-->>U: Confirmed result or recoverable operation ID
  S->>H: Deliver durable context asynchronously

Operation State Machine

From docs/technical/timeline-consistency.md#operation-state-machine:

  • prepared: intent durably committed, outcome not durably established. After restart it is potentially dispatched, not automatically safe to replay.
  • applied: intended remote result confirmed and journaled, or a local-only operation committed. Confirmed no-op carries changed:false, not a fake mutation event.
  • failed: evidence proves intended write was not applied, e.g. validation failure or conditional conflict. Authentication/server errors count as failed only where response semantics establish no write.
  • unknown: timeout/disconnection or journal failure leaves remote outcome uncertain.
  • needs_reconciliation: recovery could not establish enough evidence, or a multi-step operation needs reviewed repair. It is not permission to retry blindly.

Normal path: validate authorization/input and baseline → commit prepared intent including UID, target, before-state, intended patched representation and precondition → conditional remote write → transaction appending result and updating operation/projection/outbox → report result. A failed preflight can record failed operation without dispatch. A readback must not substitute a newer external state for the acknowledged write's after-state: retain write evidence and record later differences separately.

On network timeout, attempt to record unknown. On remote success followed by journal failure, respond partial/operation_unknown if possible, with operation ID; never report fully committed. The durable row may still say prepared if storage cannot accept the update. Client-side timeout likewise uses the pre-generated operation ID. Recovery scans prepared/unknown operations before accepting conflicting writes to the same resource."

Idempotency and Approval Specifications

From docs/technical/cli-api.md#idempotency-and-approval:

"All state-changing requests require Idempotency-Key. This includes proposals, context, sync and timeline captures, not read-only POST searches/judgments/reviews. CLI generates a high-entropy operation ID before transmission when --operation-id is absent and includes it in success/failure data. Key scope is the single-user service identity, including rotated tokens for that identity. Canonical request fingerprint includes method, route, semantic JSON body, baseline ETags and approval digest; it excludes transport request IDs and secret tokens.

Same key and fingerprint returns the recorded outcome or current pending state without another remote mutation. Different fingerprint returns idempotency_conflict. Keep operation identity/fingerprint for the retained journal lifetime; never expire it silently while clients can retry. Privacy erasure may remove it only with an explicit replay-risk warning."

Error Identifiers & Exit Codes

From docs/technical/cli-api.md#error-identifiers:

Code Meaning / handling
idempotency_conflict Same key, different semantic request; do not mutate
journal_unavailable No new remote write permitted

From docs/technical/cli-api.md#transport-and-common-contract:

  • Exit 4: Conflict (HTTP 409 / 412).
  • Exit 5: Dependency unavailable (HTTP 503).

Verification Scenarios (R01, R02, R04)

From docs/technical/verification.md#recovery-scenarios:

ID Boundary / injected failure Observable acceptance
R01 Same Idempotency-Key and semantic payload replay Recorded outcome/current pending state; exactly no additional remote dispatch
R02 Same key, changed body/route/ETag idempotency_conflict, no mutation
R04 Journal fails before prepared commit No remote write

Formal Section Citations

  • docs/technical/architecture.md#mutation-sequence: Illustrates the two-phase commit boundary and sequence of operations.
  • docs/technical/timeline-consistency.md#local-durability: Establishes the rule that prepared intent must commit before remote mutation.
  • docs/technical/timeline-consistency.md#operation-state-machine: Defines the lifecycle states prepared, applied, failed, unknown, needs_reconciliation.
  • docs/technical/cli-api.md#idempotency-and-approval: Specifies the canonical fingerprinting algorithm and replay behavior.
  • docs/technical/cli-api.md#error-identifiers: Defines idempotency_conflict and journal_unavailable.
  • docs/technical/verification.md#recovery-scenarios: Formulates normative test scenarios R01, R02, and R04.

What to Build (Step-by-Step)

  1. Implement Canonical Request Fingerprinting (src/history/fingerprint.rs):

    • Construct a canonical representation of mutation requests:
      • HTTP method (e.g. POST, PATCH).
      • Normalized path (e.g. /v1/tasks).
      • Canonicalized JSON body (keys sorted recursively, insignificant whitespace stripped).
      • Baseline ETag (if present).
      • Approval digest (if present).
    • Compute SHA-256 hash formatted as lowercase hex string: sha256:<hex>.

  2. Implement Journal Service (src/history/journal.rs):

    • prepare_operation(op_id, fingerprint, actor, intent_json, baseline_json) -> Result<PrepareResult, JournalError>:
      • Check operations table for existing operation_id.
      • If existing row found:
        • If request_hash == fingerprint: return PrepareResult::Replay(existing_row).
        • If request_hash != fingerprint: return Err(JournalError::IdempotencyConflict).
      • If not found: insert row with state = 'prepared', prepared_at = NOW(), updated_at = NOW().
    • commit_operation_result(op_id, state, result_json, events: Vec<NewLifeEvent>) -> Result<(), JournalError>:
      • In one local SQLite transaction:
        • Update operations row: state = ?, result_json = ?, updated_at = NOW().
        • Insert each event into life_events.

  3. Wire Request Execution Pipeline in API Handlers (src/api.rs):

    • Enforce mandatory Idempotency-Key header on all state-changing endpoints; reject requests missing Idempotency-Key immediately with HTTP 400 invalid_input (exit code 2). The CLI client generates a high-entropy operation ID before transmission when --operation-id is omitted.
    • Calculate canonical fingerprint.
    • Call prepare_operation:
      • If PrepareResult::Replay: immediately return recorded result without invoking downstream backend.
      • If Err(JournalError::IdempotencyConflict): return HTTP 409 with error code idempotency_conflict.
      • If SQLite error occurs: return HTTP 503 with error code journal_unavailable and DO NOT proceed to remote write.
    • Dispatch remote network request (e.g. to Radicale mock/adapter).
    • On response, call commit_operation_result to transition operations to applied or failed.
    • On network timeout, call commit_operation_result with state unknown.

  4. Integration Test Suite for R01, R02, R04 (tests/journal_idempotency.rs):

    • Test Scenario R04: Inject SQLite error during prepare_operation. Assert that zero HTTP requests reach downstream mock and client receives journal_unavailable (503).
    • Test Scenario R01: Issue write request. Assert remote mock called once. Issue identical request with same Idempotency-Key and payload. Assert client receives identical response and mock call count remains 1.
    • Test Scenario R02: Issue write request. Issue second request with same Idempotency-Key but modified title or ETag. Assert client receives HTTP 409 idempotency_conflict, exit code 4, and zero additional mutations occur.

Acceptance Criteria

  • Injected failure in SQLite during prepared intent commit prevents any remote HTTP request from executing (verifying Scenario R04).
  • Submitting the same Idempotency-Key with identical semantic payload returns the recorded result without calling downstream backend (verifying Scenario R01).
  • Submitting the same Idempotency-Key with a different payload or route returns idempotency_conflict (HTTP 409 / Exit code 4) without mutating state (verifying Scenario R02).
  • Transport headers like X-Request-ID do not alter the canonical fingerprint calculation.
  • life_events are appended atomically within the same local transaction that transitions operations to applied.
  • Mandatory Idempotency-Key on State-Changing API Requests: Calling a state-changing API endpoint without the Idempotency-Key header returns HTTP 400 invalid_input (exit code 2); the CLI client automatically generates a high-entropy operation ID before transmission when --operation-id is omitted.
  • Conforms to all normative requirements in docs/technical/verification.md for R01, R02, and R04.

Blocked by

  • #004: SQLite WAL Connection Pool, Worker & Baseline Schema
## Summary Implement the two-phase commit boundary around external mutations: commit a `prepared` operation intent and canonical request fingerprint in SQLite before remote dispatch, enforce idempotency replay versus conflict detection, and atomically commit the confirmed result or uncertainty state alongside timeline events upon remote completion. ## Background & Technical Rationale Interacting with remote CalDAV servers (such as Radicale) and external services inherently lacks distributed transaction guarantees. Network timeouts, server crashes, and process interruptions can occur at any point. To ensure the Life timeline remains consistent without phantom writes or duplicated actions: 1. **Separation of Failure Boundaries**: Local journal commits and remote HTTP requests belong to separate failure boundaries. A local journal failure must strictly block any subsequent network dispatch: the service must NEVER perform an unjournaled remote write. 2. **Phase 1 (Prepared Intent Commit)**: Before any external HTTP request is sent, the service commits a row in `operations` with `state = 'prepared'`, recording the canonical request fingerprint SHA-256 hash, actor, target resource, baseline ETag, and intended mutation payload. 3. **Phase 2 (Remote Dispatch)**: The conditional HTTP mutation (e.g. `PUT` with `If-Match`) is dispatched over the network. Crucially, **no SQLite transaction is open during this network I/O**. 4. **Phase 3 (Result Commit)**: Upon receiving the HTTP response (or transport error), a single local SQLite transaction atomically updates `operations.state` to `applied`, `failed`, or `unknown`, records the response payload, and appends the corresponding confirmed fact to `life_events`. 5. **Canonical Request Fingerprinting & Idempotency**: - Every mutation request requires an `Idempotency-Key` (or CLI `--operation-id`). - The canonical fingerprint is a SHA-256 digest covering HTTP method, normalized route, sorted semantic JSON body, baseline ETags, and approval digest. Transport-level headers such as `X-Request-ID` and authorization tokens are strictly excluded. - If an incoming request presents an existing `Idempotency-Key` and an **identical** fingerprint, the service returns the recorded outcome without repeating the remote dispatch (verifying Scenario R01). - If an incoming request presents an existing `Idempotency-Key` but a **divergent** fingerprint, the service rejects the request immediately with `idempotency_conflict` (HTTP 409 / Exit code 4) without mutating state (verifying Scenario R02). ## Exact Technical References & Specifications ### Mutation Sequence Diagram From `docs/technical/architecture.md#mutation-sequence`: > "This is not a distributed transaction. Remote success and local result commit are separate failure boundaries. A local journal failure blocks new remote writes; an uncertain or unjournaled remote outcome cannot be reported as complete." ```mermaid sequenceDiagram participant U as User through agent/CLI participant S as Life service participant D as SQLite worker participant R as Radicale participant H as Hindsight U->>S: Approved write, operation ID, baseline ETag S->>R: Read current resource and validate baseline S->>D: Commit prepared intent S->>R: Conditional PUT (no SQLite transaction held) R-->>S: Success, conflict or uncertain transport outcome S->>D: Commit result event, operation state and optional outbox S-->>U: Confirmed result or recoverable operation ID S->>H: Deliver durable context asynchronously ``` ### Operation State Machine From `docs/technical/timeline-consistency.md#operation-state-machine`: > - `prepared`: intent durably committed, outcome not durably established. After restart it is potentially dispatched, not automatically safe to replay. > - `applied`: intended remote result confirmed and journaled, or a local-only operation committed. Confirmed no-op carries `changed:false`, not a fake mutation event. > - `failed`: evidence proves intended write was not applied, e.g. validation failure or conditional conflict. Authentication/server errors count as failed only where response semantics establish no write. > - `unknown`: timeout/disconnection or journal failure leaves remote outcome uncertain. > - `needs_reconciliation`: recovery could not establish enough evidence, or a multi-step operation needs reviewed repair. It is not permission to retry blindly. > > Normal path: validate authorization/input and baseline → commit prepared intent including UID, target, before-state, intended patched representation and precondition → conditional remote write → transaction appending result and updating operation/projection/outbox → report result. A failed preflight can record failed operation without dispatch. A readback must not substitute a newer external state for the acknowledged write's after-state: retain write evidence and record later differences separately. > > On network timeout, attempt to record unknown. On remote success followed by journal failure, respond partial/operation_unknown if possible, with operation ID; never report fully committed. The durable row may still say prepared if storage cannot accept the update. Client-side timeout likewise uses the pre-generated operation ID. Recovery scans prepared/unknown operations before accepting conflicting writes to the same resource." ### Idempotency and Approval Specifications From `docs/technical/cli-api.md#idempotency-and-approval`: > "All state-changing requests require `Idempotency-Key`. This includes proposals, context, sync and timeline captures, not read-only POST searches/judgments/reviews. CLI generates a high-entropy operation ID before transmission when `--operation-id` is absent and includes it in success/failure data. Key scope is the single-user service identity, including rotated tokens for that identity. Canonical request fingerprint includes method, route, semantic JSON body, baseline ETags and approval digest; it excludes transport request IDs and secret tokens. > > Same key and fingerprint returns the recorded outcome or current pending state without another remote mutation. Different fingerprint returns `idempotency_conflict`. Keep operation identity/fingerprint for the retained journal lifetime; never expire it silently while clients can retry. Privacy erasure may remove it only with an explicit replay-risk warning." ### Error Identifiers & Exit Codes From `docs/technical/cli-api.md#error-identifiers`: | Code | Meaning / handling | |---|---| | idempotency_conflict | Same key, different semantic request; do not mutate | | journal_unavailable | No new remote write permitted | From `docs/technical/cli-api.md#transport-and-common-contract`: - Exit 4: Conflict (HTTP 409 / 412). - Exit 5: Dependency unavailable (HTTP 503). ### Verification Scenarios (R01, R02, R04) From `docs/technical/verification.md#recovery-scenarios`: | ID | Boundary / injected failure | Observable acceptance | |---|---|---| | R01 | Same Idempotency-Key and semantic payload replay | Recorded outcome/current pending state; exactly no additional remote dispatch | | R02 | Same key, changed body/route/ETag | idempotency_conflict, no mutation | | R04 | Journal fails before prepared commit | No remote write | ### Formal Section Citations - `docs/technical/architecture.md#mutation-sequence`: Illustrates the two-phase commit boundary and sequence of operations. - `docs/technical/timeline-consistency.md#local-durability`: Establishes the rule that prepared intent must commit before remote mutation. - `docs/technical/timeline-consistency.md#operation-state-machine`: Defines the lifecycle states `prepared`, `applied`, `failed`, `unknown`, `needs_reconciliation`. - `docs/technical/cli-api.md#idempotency-and-approval`: Specifies the canonical fingerprinting algorithm and replay behavior. - `docs/technical/cli-api.md#error-identifiers`: Defines `idempotency_conflict` and `journal_unavailable`. - `docs/technical/verification.md#recovery-scenarios`: Formulates normative test scenarios R01, R02, and R04. ## What to Build (Step-by-Step) 1. **Implement Canonical Request Fingerprinting (`src/history/fingerprint.rs`)**: - Construct a canonical representation of mutation requests: - HTTP method (e.g. `POST`, `PATCH`). - Normalized path (e.g. `/v1/tasks`). - Canonicalized JSON body (keys sorted recursively, insignificant whitespace stripped). - Baseline ETag (if present). - Approval digest (if present). - Compute SHA-256 hash formatted as lowercase hex string: `sha256:<hex>`. 2. **Implement Journal Service (`src/history/journal.rs`)**: - `prepare_operation(op_id, fingerprint, actor, intent_json, baseline_json) -> Result<PrepareResult, JournalError>`: - Check `operations` table for existing `operation_id`. - If existing row found: - If `request_hash == fingerprint`: return `PrepareResult::Replay(existing_row)`. - If `request_hash != fingerprint`: return `Err(JournalError::IdempotencyConflict)`. - If not found: insert row with `state = 'prepared'`, `prepared_at = NOW()`, `updated_at = NOW()`. - `commit_operation_result(op_id, state, result_json, events: Vec<NewLifeEvent>) -> Result<(), JournalError>`: - In one local SQLite transaction: - Update `operations` row: `state = ?`, `result_json = ?`, `updated_at = NOW()`. - Insert each event into `life_events`. 3. **Wire Request Execution Pipeline in API Handlers (`src/api.rs`)**: - Enforce mandatory `Idempotency-Key` header on all state-changing endpoints; reject requests missing `Idempotency-Key` immediately with HTTP 400 `invalid_input` (exit code 2). The CLI client generates a high-entropy operation ID before transmission when `--operation-id` is omitted. - Calculate canonical fingerprint. - Call `prepare_operation`: - If `PrepareResult::Replay`: immediately return recorded result without invoking downstream backend. - If `Err(JournalError::IdempotencyConflict)`: return HTTP 409 with error code `idempotency_conflict`. - If SQLite error occurs: return HTTP 503 with error code `journal_unavailable` and DO NOT proceed to remote write. - Dispatch remote network request (e.g. to Radicale mock/adapter). - On response, call `commit_operation_result` to transition `operations` to `applied` or `failed`. - On network timeout, call `commit_operation_result` with state `unknown`. 4. **Integration Test Suite for R01, R02, R04 (`tests/journal_idempotency.rs`)**: - **Test Scenario R04**: Inject SQLite error during `prepare_operation`. Assert that zero HTTP requests reach downstream mock and client receives `journal_unavailable` (503). - **Test Scenario R01**: Issue write request. Assert remote mock called once. Issue identical request with same `Idempotency-Key` and payload. Assert client receives identical response and mock call count remains 1. - **Test Scenario R02**: Issue write request. Issue second request with same `Idempotency-Key` but modified title or ETag. Assert client receives HTTP 409 `idempotency_conflict`, exit code 4, and zero additional mutations occur. ## Acceptance Criteria - [x] Injected failure in SQLite during prepared intent commit prevents any remote HTTP request from executing (verifying Scenario R04). - [x] Submitting the same `Idempotency-Key` with identical semantic payload returns the recorded result without calling downstream backend (verifying Scenario R01). - [x] Submitting the same `Idempotency-Key` with a different payload or route returns `idempotency_conflict` (HTTP 409 / Exit code 4) without mutating state (verifying Scenario R02). - [x] Transport headers like `X-Request-ID` do not alter the canonical fingerprint calculation. - [x] `life_events` are appended atomically within the same local transaction that transitions `operations` to `applied`. - [x] Mandatory Idempotency-Key on State-Changing API Requests: Calling a state-changing API endpoint without the `Idempotency-Key` header returns HTTP 400 `invalid_input` (exit code 2); the CLI client automatically generates a high-entropy operation ID before transmission when `--operation-id` is omitted. - [x] Conforms to all normative requirements in `docs/technical/verification.md` for R01, R02, and R04. ## Blocked by - #004: SQLite WAL Connection Pool, Worker & Baseline Schema
yuris closed this issue 2026-09-22 19:19:04 +00:00
yuris changed title from Issue #005: Two-Phase Commit Journaling & Idempotency Fingerprinting to Two-Phase Commit Journaling & Idempotency Fingerprinting 2026-09-22 20:02:40 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
v0.2.4
v0.2.3
v0.2.2
v0.2.1
v0.2.0
v0.1.0
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
yuris/life-service#5
No description provided.