Durable Context Outbox Queue & Background Delivery Worker #17

New issue

Closed

opened 2026-09-22 17:58:56 +00:00 by yuris · 0 comments

yuris commented 2026-09-22 17:58:56 +00:00

Owner

Copy link

Summary

Implement the durable SQLite outbox table (outbox), Tokio background delivery worker routed through the dedicated single database worker, operation status polling, asynchronous lifecycle state machine, and the foundational context & capture API routes (POST /v1/context and POST /v1/captures for task/event/context and mixed action+rationale, Scenario W02).

Background & Technical Rationale

Life records semantic context, rationales, and user preferences in Hindsight. However, network services can experience outages, latency spikes, or intermittent failures. A core architectural principle of Life is strict decoupling between transactional calendar execution and semantic context delivery:

1. Decoupled Consistency Invariant (Scenario I04):
   Calendar mutations (creating, updating, and completing tasks or events in Radicale) MUST be committed independently of Hindsight availability. If Hindsight is unreachable, down for maintenance, or returning HTTP 500/503 errors, the calendar mutation MUST succeed, commit to SQLite WAL and Radicale, and return success to the caller with context_delivery: "pending" (or "unavailable"). Under NO circumstances may an unavailable Hindsight backend roll back, fail, or delay a confirmed calendar write.
2. Three Distinct Delivery Milestones (Scenario I03):
   Life's outbox and API contracts strictly separate three distinct stages of context ingestion:
   • Upstream Accepted: The retain request was received and accepted into Hindsight's ingestion queue (HTTP 202 Accepted with an operation_id).
   • Ingestion Completed: Hindsight's background ingestion worker has processed the document and settled the operation status to completed.
   • Refreshed Derived Observations: Hindsight's higher-level mental models and semantic embeddings have updated based on the new memories. Ingestion completion does NOT imply that all derived observations or mental models have immediately refreshed. Life must never make premature claims of completed delivery.
3. Durable Local Durability:
   To survive crashes and power loss without dropping context, outbound memory payloads are committed to a durable SQLite outbox table within the same local ACID transaction that records the calendar change in life_events.
4. Operation Polling vs Resubmission:
   When Hindsight returns HTTP 202 Accepted with an operation_id, the worker MUST persist the upstream_operation_id and poll GET /v1/default/banks/{bank_id}/operations/{operation_id} rather than repeatedly resubmitting the memory. Resubmitting an in-flight operation wastes resources and can cause upstream lock contention.
5. Crash Recovery & Idempotent Resumption (Scenario R12):
   If the process crashes after Hindsight accepts a dispatch or during polling, the durable outbox record survives. Upon restart, the worker recovers the pending record, continues polling the known upstream_operation_id, or resubmits under the same deterministic document_id (life-event:<id>) with update_mode: "replace".

Exact Technical References & Specifications

Normative SQLite Outbox Schema

From docs/technical/timeline-consistency.md (lines 97–112):

CREATE TABLE outbox (
  delivery_id TEXT PRIMARY KEY,
  source_event_id TEXT NOT NULL REFERENCES life_events(event_id),
  destination TEXT NOT NULL,
  document_id TEXT NOT NULL,
  schema_version INTEGER NOT NULL,
  payload_json TEXT NOT NULL,
  state TEXT NOT NULL CHECK(state IN
    ('pending','accepted','completed','retry_wait','needs_reconciliation','failed')),
  attempts INTEGER NOT NULL DEFAULT 0,
  next_attempt_at TEXT,
  upstream_operation_id TEXT,
  upstream_ids_json TEXT,
  last_error_code TEXT,
  UNIQUE(destination, document_id)
);

Context Delivery & Outbox Contract

From docs/technical/timeline-consistency.md (lines 160–165):

"When meaningful context is selected, commit its source event and outbox entry with the calendar result. Hindsight failure never rolls back a confirmed calendar write. Poll known asynchronous operations; persist returned IDs/status. Retry with the stable document identity according to Hindsight. Possible duplicate upstream processing remains explicit when upstream idempotency/consolidation guarantees are insufficient."

From docs/technical/hindsight.md (lines 31–38):

"Life's outbox distinguishes pending, upstream accepted/queued, ingestion completed, retry_wait, failed and needs_reconciliation. Accepted retain, completed ingestion and refreshed derived observations are three different milestones. Ingestion completion does not imply that every mental model or observation has refreshed.

Persist a returned operation ID before polling. Poll the known operation rather than submitting again while it is active. Bounded backoff honors server retry signals; authentication/schema errors halt delivery and surface an operator error. If acceptance response is lost, keep uncertainty explicit, serialize retries per document and re-submit only the same document identity under verified replace semantics. This may reprocess content and does not promise exactly-once semantic consolidation. Do not retry with a new source event to hide uncertainty.

The calendar mutation is already committed independently. Its API result includes context delivery state; an unavailable Hindsight backend does not undo or relabel a confirmed calendar change. Context-only capture completes locally when queued, but returns pending/exit 6 until the requested upstream delivery is confirmed."

Verification Matrix Scenarios

From docs/technical/verification.md (lines 98, 117–118):

ID	Integration and input	Observable acceptance
I03	Async accepted, ingestion pending, derived refresh lagging	Three milestones remain distinct; no premature completed delivery claim
I04	Hindsight outage while live agenda/write succeeds	Agenda/write remains usable; context missing/pending visible and durable outbox retained
R12	Outbox dispatch accepted but response/poll result lost	Stable document identity and known operation polling; possible duplicate processing disclosed

Section Citations

• docs/technical/timeline-consistency.md: Section Illustrative schema (lines 97–112), Section Context delivery and correction (lines 160–165).
• docs/technical/hindsight.md: Section Delivery states (lines 31–38).
• docs/technical/verification.md: Section Recovery scenarios (line 98, Scenario R12), Section Integration scenarios (lines 117–118, Scenarios I03, I04).
• docs/technical/cli-api.md: Section Command payloads and results (lines 82, 87, 93, 148).

What to Build

1. Database Schema & Migration (src/storage/schema.rs)

• Add migration creating the outbox table as specified in timeline-consistency.md.
• Create index for fast polling:

  CREATE INDEX idx_outbox_queue ON outbox(state, next_attempt_at);
  

• Define OutboxState enum:

  #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
  #[serde(rename_all = "snake_case")]
  pub enum OutboxState {
      Pending,
      Accepted,
      Completed,
      RetryWait,
      NeedsReconciliation,
      Failed,
  }
  

2. Transactional Enqueue API (src/hindsight/outbox.rs)

• Implement enqueue_context_delivery routed through the dedicated database worker thread (DatabaseHandle):

  pub async fn enqueue_context_delivery(
      db: &DatabaseHandle,
      source_event_id: &str,
      destination: &str,
      document_id: &str,
      payload: &RetainRequest,
  ) -> Result<String, OutboxError>
  

• Generates a unique delivery_id (del_<nanoid>).
• Inserts row into outbox with state = 'pending', attempts = 0, schema_version = 1.
• Enforces the UNIQUE(destination, document_id) constraint via ON CONFLICT(destination, document_id) DO UPDATE to overwrite pending payloads if a newer update occurs before dispatch.

3. Asynchronous Outbox Delivery Worker (src/hindsight/worker.rs)

• Implement OutboxWorker running as a background Tokio task supervised by the Life daemon, performing all database reads and updates via DatabaseHandle:

  pub struct OutboxWorker {
      db: DatabaseHandle,
      client: Arc<HindsightClient>,
      cancel_token: tokio_util::sync::CancellationToken,
      config: OutboxConfig,
  }
  

• Worker loop logic:
  1. Fetch Ready Deliveries:
     Query up to N records with state IN ('pending', 'retry_wait') where next_attempt_at IS NULL OR next_attempt_at <= datetime('now').
  2. Dispatch Retain Request:
     Call client.retain(payload).
     • On HTTP 200 OK:
       Update outbox record to state = 'completed', record memory IDs in upstream_ids_json.
     • On HTTP 202 Accepted with operation ID:
       Update outbox record to state = 'accepted', record upstream_operation_id.
     • On 429 / 503 / Network Timeout:
       Increment attempts. Compute exponential backoff with full jitter:

       \Delta t = \min(2^{\text{attempts}} \times 1\text{s}, 300\text{s}) + \text{jitter}

       Parse Retry-After header if present. Update state = 'retry_wait', next_attempt_at = now + \Delta t.
     • On 401 / 403 / Schema Error:
       Update state = 'failed', record last_error_code, log structured error to stderr. Do not blindly retry fatal authentication/schema errors.
  3. Poll In-Flight Operations:
     Query records with state = 'accepted' and upstream_operation_id IS NOT NULL.
     Call client.poll_operation(upstream_operation_id).
     • If upstream status is "completed":
       Update state = 'completed', record settled memory IDs in upstream_ids_json.
     • If upstream status is "in_progress" or "pending":
       Leave in accepted, schedule next poll after interval (e.g. 2s).
     • If upstream status is "failed":
       Transition to retry_wait (if retryable) or failed.
  4. Graceful Shutdown:
     Listen on cancel_token.cancelled(). Complete current in-flight HTTP request, ensure transaction state is cleanly committed, and exit without leaving database locked.

4. API Endpoints & Unified Capture Foundation (src/api/context.rs, src/api/captures.rs, src/cli/capture.rs)

• Implement POST /v1/context (life context add --input -):
  • Input: { text: String, item_id: Option<String>, reason: Option<String>, supersedes_context_id: Option<String> }.
  • Appends context_added event to life_events and enqueues outbox record in one transaction.
  • Returns { operation_id, source_event_id, delivery_state }.
• Implement foundational POST /v1/captures (life capture --input -):
  • Input: { kind: "context" | "task" | "event", payload: serde_json::Value, context: Option<String> }.
  • For kind: "context" (Scenario W02): creates zero calendar items; enqueues context delivery; returns pending (exit 6) until upstream delivery confirms.
  • For mixed action+rationale (Scenario W02): calendar mutation commits independently; API response reports both confirmed calendar item/ETag and context delivery state (pending/accepted/completed).

5. Integration with API Mutation Responses

• Update mutation response builder in src/api.rs:
  • When a task or event is added/updated with context:
    • If Hindsight is configured and context queued: set context_delivery: "pending" in the JSON envelope.
    • If Hindsight worker already confirmed ingestion: set context_delivery: "delivered".
    • If Hindsight backend is unreachable: set context_delivery: "unavailable".
    • If no context was supplied: set context_delivery: "not_requested".

5. Integration Test Suite (tests/hindsight_outbox_tests.rs)

• Test Scenario I04 (Hindsight outage during calendar write):
  • Mock Hindsight server returns HTTP 503.
  • Execute life task add with context rationale.
  • Assert that calendar task is created successfully in Radicale and SQLite (HTTP 201).
  • Assert that API response returns context_delivery: "pending".
  • Assert that outbox table contains row with state = 'retry_wait'.
• Test Scenario I03 (Three milestones progression):
  • Mock Hindsight returns HTTP 202 Accepted on retain.
  • Run outbox worker tick; assert outbox state is accepted and upstream_operation_id is populated.
  • Verify that system does NOT report delivery as completed.
  • Mock operation endpoint returns "completed".
  • Run outbox worker poll tick; assert outbox state transitions to completed.
• Test Scenario R12 (Crash recovery and operation resume):
  • Seed outbox table with record in state = 'accepted' and known upstream_operation_id.
  • Launch worker; verify worker polls the operation endpoint rather than dispatching a duplicate retain request.

Acceptance Criteria

• Calendar task/event creation succeeds completely even when Hindsight is offline or returning 503, queuing context delivery into outbox as pending or retry_wait (Scenario I04).
• Outbox delivery distinguishes between HTTP 202 Accepted (accepted), settled ingestion (completed), and background derived observation refresh (Scenario I03).
• In-flight operations with an upstream_operation_id are polled via GET .../operations/{id} rather than repeatedly resubmitted while active.
• Context-only capture (life capture with kind: context) creates zero calendar items and returns pending delivery state (Scenario W02).
• Mixed action+rationale capture commits calendar mutation independently and reports both confirmed item and context delivery state (Scenario W02).
• All outbox state transitions and enqueues execute strictly through the single dedicated blocking SQLite database worker.
• Network failures and transient server errors trigger exponential backoff with jitter and honor Retry-After headers.
• Retries always resubmit the same deterministic document_id under update_mode: "replace" without generating new source events (Scenario R12).
• Process shutdown flushes active database state cleanly; queued outbox entries resume upon next daemon startup without data loss.

Notes

• Enqueue trigger decided in review: deliveries are requested only by explicit capture (POST /v1/captures, kind: context and task|event + context), never implicitly by a mutation carrying reason — docs/technical/hindsight.md forbids silently retaining every CRUD action. outcome/correction kinds are refused as not-yet-implemented (#28/#29).
• One review round: migration 002 now drops the superseded idx_outbox_state so fresh and pre-existing databases converge, with a test that fails without the drop.
• Routes left unenqueued (nothing faked): /v1/tasks, /v1/events, update and lifecycle. /v1/context and /v1/context/search are #18's.
• No dependency added, so the deferred Cargo.toml authors chore stays pending.

Blocked by

• #016: Hindsight Reqwest Client & Deterministic Document Identity

## Summary Implement the durable SQLite outbox table (`outbox`), Tokio background delivery worker routed through the dedicated single database worker, operation status polling, asynchronous lifecycle state machine, and the foundational context & capture API routes (`POST /v1/context` and `POST /v1/captures` for task/event/context and mixed action+rationale, Scenario W02). ## Background & Technical Rationale Life records semantic context, rationales, and user preferences in Hindsight. However, network services can experience outages, latency spikes, or intermittent failures. A core architectural principle of Life is strict decoupling between transactional calendar execution and semantic context delivery: 1. **Decoupled Consistency Invariant (Scenario I04):** Calendar mutations (creating, updating, and completing tasks or events in Radicale) MUST be committed independently of Hindsight availability. If Hindsight is unreachable, down for maintenance, or returning HTTP 500/503 errors, the calendar mutation MUST succeed, commit to SQLite WAL and Radicale, and return success to the caller with `context_delivery: "pending"` (or `"unavailable"`). Under NO circumstances may an unavailable Hindsight backend roll back, fail, or delay a confirmed calendar write. 2. **Three Distinct Delivery Milestones (Scenario I03):** Life's outbox and API contracts strictly separate three distinct stages of context ingestion: - **Upstream Accepted:** The retain request was received and accepted into Hindsight's ingestion queue (HTTP 202 Accepted with an `operation_id`). - **Ingestion Completed:** Hindsight's background ingestion worker has processed the document and settled the operation status to completed. - **Refreshed Derived Observations:** Hindsight's higher-level mental models and semantic embeddings have updated based on the new memories. Ingestion completion does NOT imply that all derived observations or mental models have immediately refreshed. Life must never make premature claims of completed delivery. 3. **Durable Local Durability:** To survive crashes and power loss without dropping context, outbound memory payloads are committed to a durable SQLite `outbox` table within the *same* local ACID transaction that records the calendar change in `life_events`. 4. **Operation Polling vs Resubmission:** When Hindsight returns HTTP 202 Accepted with an `operation_id`, the worker MUST persist the `upstream_operation_id` and poll `GET /v1/default/banks/{bank_id}/operations/{operation_id}` rather than repeatedly resubmitting the memory. Resubmitting an in-flight operation wastes resources and can cause upstream lock contention. 5. **Crash Recovery & Idempotent Resumption (Scenario R12):** If the process crashes after Hindsight accepts a dispatch or during polling, the durable outbox record survives. Upon restart, the worker recovers the pending record, continues polling the known `upstream_operation_id`, or resubmits under the same deterministic `document_id` (`life-event:<id>`) with `update_mode: "replace"`. ## Exact Technical References & Specifications ### Normative SQLite Outbox Schema From `docs/technical/timeline-consistency.md` (lines 97–112): ```sql CREATE TABLE outbox ( delivery_id TEXT PRIMARY KEY, source_event_id TEXT NOT NULL REFERENCES life_events(event_id), destination TEXT NOT NULL, document_id TEXT NOT NULL, schema_version INTEGER NOT NULL, payload_json TEXT NOT NULL, state TEXT NOT NULL CHECK(state IN ('pending','accepted','completed','retry_wait','needs_reconciliation','failed')), attempts INTEGER NOT NULL DEFAULT 0, next_attempt_at TEXT, upstream_operation_id TEXT, upstream_ids_json TEXT, last_error_code TEXT, UNIQUE(destination, document_id) ); ``` ### Context Delivery & Outbox Contract From `docs/technical/timeline-consistency.md` (lines 160–165): > "When meaningful context is selected, commit its source event and outbox entry with the calendar result. Hindsight failure never rolls back a confirmed calendar write. Poll known asynchronous operations; persist returned IDs/status. Retry with the stable document identity according to [Hindsight](hindsight.md). Possible duplicate upstream processing remains explicit when upstream idempotency/consolidation guarantees are insufficient." From `docs/technical/hindsight.md` (lines 31–38): > "Life's outbox distinguishes pending, upstream accepted/queued, ingestion completed, retry_wait, failed and needs_reconciliation. Accepted retain, completed ingestion and refreshed derived observations are three different milestones. Ingestion completion does not imply that every mental model or observation has refreshed. > > Persist a returned operation ID before polling. Poll the known operation rather than submitting again while it is active. Bounded backoff honors server retry signals; authentication/schema errors halt delivery and surface an operator error. If acceptance response is lost, keep uncertainty explicit, serialize retries per document and re-submit only the same document identity under verified replace semantics. This may reprocess content and does not promise exactly-once semantic consolidation. Do not retry with a new source event to hide uncertainty. > > The calendar mutation is already committed independently. Its API result includes context delivery state; an unavailable Hindsight backend does not undo or relabel a confirmed calendar change. Context-only capture completes locally when queued, but returns pending/exit 6 until the requested upstream delivery is confirmed." ### Verification Matrix Scenarios From `docs/technical/verification.md` (lines 98, 117–118): | ID | Integration and input | Observable acceptance | |---|---|---| | I03 | Async accepted, ingestion pending, derived refresh lagging | Three milestones remain distinct; no premature completed delivery claim | | I04 | Hindsight outage while live agenda/write succeeds | Agenda/write remains usable; context missing/pending visible and durable outbox retained | | R12 | Outbox dispatch accepted but response/poll result lost | Stable document identity and known operation polling; possible duplicate processing disclosed | ### Section Citations - `docs/technical/timeline-consistency.md`: Section *Illustrative schema* (lines 97–112), Section *Context delivery and correction* (lines 160–165). - `docs/technical/hindsight.md`: Section *Delivery states* (lines 31–38). - `docs/technical/verification.md`: Section *Recovery scenarios* (line 98, Scenario R12), Section *Integration scenarios* (lines 117–118, Scenarios I03, I04). - `docs/technical/cli-api.md`: Section *Command payloads and results* (lines 82, 87, 93, 148). ## What to Build ### 1. Database Schema & Migration (`src/storage/schema.rs`) - Add migration creating the `outbox` table as specified in `timeline-consistency.md`. - Create index for fast polling: ```sql CREATE INDEX idx_outbox_queue ON outbox(state, next_attempt_at); ``` - Define `OutboxState` enum: ```rust #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] pub enum OutboxState { Pending, Accepted, Completed, RetryWait, NeedsReconciliation, Failed, } ``` ### 2. Transactional Enqueue API (`src/hindsight/outbox.rs`) - Implement `enqueue_context_delivery` routed through the dedicated database worker thread (`DatabaseHandle`): ```rust pub async fn enqueue_context_delivery( db: &DatabaseHandle, source_event_id: &str, destination: &str, document_id: &str, payload: &RetainRequest, ) -> Result<String, OutboxError> ``` - Generates a unique `delivery_id` (`del_<nanoid>`). - Inserts row into `outbox` with `state = 'pending'`, `attempts = 0`, `schema_version = 1`. - Enforces the `UNIQUE(destination, document_id)` constraint via `ON CONFLICT(destination, document_id) DO UPDATE` to overwrite pending payloads if a newer update occurs before dispatch. ### 3. Asynchronous Outbox Delivery Worker (`src/hindsight/worker.rs`) - Implement `OutboxWorker` running as a background Tokio task supervised by the Life daemon, performing all database reads and updates via `DatabaseHandle`: ```rust pub struct OutboxWorker { db: DatabaseHandle, client: Arc<HindsightClient>, cancel_token: tokio_util::sync::CancellationToken, config: OutboxConfig, } ``` - Worker loop logic: 1. **Fetch Ready Deliveries:** Query up to $N$ records with `state IN ('pending', 'retry_wait')` where `next_attempt_at IS NULL OR next_attempt_at <= datetime('now')`. 2. **Dispatch Retain Request:** Call `client.retain(payload)`. - **On HTTP 200 OK:** Update `outbox` record to `state = 'completed'`, record memory IDs in `upstream_ids_json`. - **On HTTP 202 Accepted with operation ID:** Update `outbox` record to `state = 'accepted'`, record `upstream_operation_id`. - **On 429 / 503 / Network Timeout:** Increment `attempts`. Compute exponential backoff with full jitter: $$\Delta t = \min(2^{\text{attempts}} \times 1\text{s}, 300\text{s}) + \text{jitter}$$ Parse `Retry-After` header if present. Update `state = 'retry_wait'`, `next_attempt_at = now + \Delta t`. - **On 401 / 403 / Schema Error:** Update `state = 'failed'`, record `last_error_code`, log structured error to stderr. Do not blindly retry fatal authentication/schema errors. 3. **Poll In-Flight Operations:** Query records with `state = 'accepted'` and `upstream_operation_id IS NOT NULL`. Call `client.poll_operation(upstream_operation_id)`. - If upstream status is `"completed"`: Update `state = 'completed'`, record settled memory IDs in `upstream_ids_json`. - If upstream status is `"in_progress"` or `"pending"`: Leave in `accepted`, schedule next poll after interval (e.g. 2s). - If upstream status is `"failed"`: Transition to `retry_wait` (if retryable) or `failed`. 4. **Graceful Shutdown:** Listen on `cancel_token.cancelled()`. Complete current in-flight HTTP request, ensure transaction state is cleanly committed, and exit without leaving database locked. ### 4. API Endpoints & Unified Capture Foundation (`src/api/context.rs`, `src/api/captures.rs`, `src/cli/capture.rs`) - Implement `POST /v1/context` (`life context add --input -`): - Input: `{ text: String, item_id: Option<String>, reason: Option<String>, supersedes_context_id: Option<String> }`. - Appends `context_added` event to `life_events` and enqueues outbox record in one transaction. - Returns `{ operation_id, source_event_id, delivery_state }`. - Implement foundational `POST /v1/captures` (`life capture --input -`): - Input: `{ kind: "context" | "task" | "event", payload: serde_json::Value, context: Option<String> }`. - For `kind: "context"` (Scenario W02): creates zero calendar items; enqueues context delivery; returns pending (exit 6) until upstream delivery confirms. - For mixed action+rationale (Scenario W02): calendar mutation commits independently; API response reports both confirmed calendar item/ETag and context delivery state (`pending`/`accepted`/`completed`). ### 5. Integration with API Mutation Responses - Update mutation response builder in `src/api.rs`: - When a task or event is added/updated with context: - If Hindsight is configured and context queued: set `context_delivery: "pending"` in the JSON envelope. - If Hindsight worker already confirmed ingestion: set `context_delivery: "delivered"`. - If Hindsight backend is unreachable: set `context_delivery: "unavailable"`. - If no context was supplied: set `context_delivery: "not_requested"`. ### 5. Integration Test Suite (`tests/hindsight_outbox_tests.rs`) - Test Scenario I04 (Hindsight outage during calendar write): - Mock Hindsight server returns HTTP 503. - Execute `life task add` with context rationale. - Assert that calendar task is created successfully in Radicale and SQLite (HTTP 201). - Assert that API response returns `context_delivery: "pending"`. - Assert that `outbox` table contains row with `state = 'retry_wait'`. - Test Scenario I03 (Three milestones progression): - Mock Hindsight returns HTTP 202 Accepted on retain. - Run outbox worker tick; assert outbox state is `accepted` and `upstream_operation_id` is populated. - Verify that system does NOT report delivery as completed. - Mock operation endpoint returns `"completed"`. - Run outbox worker poll tick; assert outbox state transitions to `completed`. - Test Scenario R12 (Crash recovery and operation resume): - Seed `outbox` table with record in `state = 'accepted'` and known `upstream_operation_id`. - Launch worker; verify worker polls the operation endpoint rather than dispatching a duplicate retain request. ## Acceptance Criteria - [x] Calendar task/event creation succeeds completely even when Hindsight is offline or returning 503, queuing context delivery into `outbox` as `pending` or `retry_wait` (Scenario I04). - [x] Outbox delivery distinguishes between HTTP 202 Accepted (`accepted`), settled ingestion (`completed`), and background derived observation refresh (Scenario I03). - [x] In-flight operations with an `upstream_operation_id` are polled via `GET .../operations/{id}` rather than repeatedly resubmitted while active. - [x] Context-only capture (`life capture` with `kind: context`) creates zero calendar items and returns pending delivery state (Scenario W02). - [x] Mixed action+rationale capture commits calendar mutation independently and reports both confirmed item and context delivery state (Scenario W02). - [x] All outbox state transitions and enqueues execute strictly through the single dedicated blocking SQLite database worker. - [x] Network failures and transient server errors trigger exponential backoff with jitter and honor `Retry-After` headers. - [x] Retries always resubmit the same deterministic `document_id` under `update_mode: "replace"` without generating new source events (Scenario R12). - [x] Process shutdown flushes active database state cleanly; queued outbox entries resume upon next daemon startup without data loss. ## Notes - **Enqueue trigger decided in review:** deliveries are requested only by explicit capture (`POST /v1/captures`, `kind: context` and `task|event` + `context`), never implicitly by a mutation carrying `reason` — `docs/technical/hindsight.md` forbids silently retaining every CRUD action. `outcome`/`correction` kinds are refused as not-yet-implemented (#28/#29). - One review round: migration 002 now drops the superseded `idx_outbox_state` so fresh and pre-existing databases converge, with a test that fails without the drop. - Routes left unenqueued (nothing faked): `/v1/tasks`, `/v1/events`, update and lifecycle. `/v1/context` and `/v1/context/search` are #18's. - No dependency added, so the deferred `Cargo.toml` authors chore stays pending. ## Blocked by - #016: Hindsight Reqwest Client & Deterministic Document Identity

yuris referenced this issue 2026-09-22 17:58:56 +00:00

Strict Context Search & Supersession Provenance #18

yuris changed title from Issue #017: Durable Context Outbox Queue & Background Delivery Worker to Durable Context Outbox Queue & Background Delivery Worker 2026-09-22 20:02:37 +00:00

yuris referenced this issue from a commit 2026-09-22 22:55:54 +00:00

feat(hindsight): reqwest client and deterministic document identity

yuris referenced this issue 2026-09-22 22:56:35 +00:00

feat(hindsight): reqwest client and deterministic document identity #55

yuris referenced this issue 2026-09-22 22:58:10 +00:00

feat(hindsight): reqwest client and deterministic document identity #55

yuris referenced this issue 2026-09-22 22:58:10 +00:00

Hindsight Reqwest Client & Deterministic Document Identity #16

yuris referenced this issue 2026-09-22 23:28:58 +00:00

feat(jev): candidate reference resolution with deterministic first #58

yuris referenced this issue from a commit 2026-09-22 23:37:09 +00:00

feat(hindsight): durable context outbox and background delivery worker

yuris referenced this issue from a pull request that will close it, 2026-09-22 23:37:13 +00:00

feat(hindsight): durable context outbox and background delivery worker #59

yuris referenced this issue from a commit 2026-09-22 23:43:15 +00:00

feat(hindsight): durable context outbox and background delivery worker

yuris closed this issue 2026-09-22 23:44:20 +00:00

yuris referenced this issue from a commit 2026-09-22 23:44:22 +00:00

feat(hindsight): durable context outbox and background delivery worker (#59)

yuris referenced this issue 2026-09-22 23:48:17 +00:00

feat(ops): hardened production container and exclusive service directory lock #60

yuris referenced this issue 2026-09-23 00:01:48 +00:00

feat(hindsight): strict context search and supersession provenance #61

yuris referenced this issue 2026-09-23 00:03:23 +00:00

feat(hindsight): strict context search and supersession provenance #61

yuris referenced this issue 2026-09-23 03:23:37 +00:00

feat(api): work-block outcome capture as user-reported facts #72

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

v0.2.4

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1.0

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

yuris/life-service#17

No description provided.