yuris/life-service
1
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages 1 Wiki Activity Actions

SQLite Online Vacuum Snapshot & Disaster Recovery Rehearsal #36

New issue
Closed
opened 2026-09-22 17:59:00 +00:00 by yuris · 0 comments
yuris commented 2026-09-22 17:59:00 +00:00
Owner
Copy link

Summary

Implement the robust SQLite online snapshot mechanism (VACUUM INTO ? or SQLite Online Backup API) to an isolated staging directory, write atomic snapshot manifests with SHA-256 integrity checksums, and construct an automated disaster recovery rehearsal test suite verifying schema integrity, foreign key consistency, operation reconciliation, and temporal observation gap reporting when restored alongside differently-timed external calendar backups (Scenarios D10, D11, D12).

Background & Technical Rationale

The WAL Backup Pitfall (Scenario D10)

In SQLite Write-Ahead Logging (WAL) mode (PRAGMA journal_mode=WAL), transactions are written to the -wal sidecar file and flushed according to PRAGMA synchronous=FULL. If a backup tool or operator executes a standard file copy of the main database (cp life.sqlite3 /backups/), the resulting file will be inconsistent, corrupted, or missing recently committed transactions whose pages reside exclusively in the WAL.

Copying the live database file alone is an explicitly rejected operational procedure. Valid snapshots must be produced using SQLite's native Online Backup API (sqlite3_backup_*) or an atomic, consistent VACUUM INTO '<staging_path>' query executed over an active database connection. This produces a single, self-contained, clean SQLite database file containing all committed transactions without interrupting concurrent read queries.

Multi-System Asynchrony & Observation Gaps (Scenario D11)

The life system architecture intentionally couples an append-only local SQLite database (storing operations, proposals, and timeline events) with an external CalDAV store (Radicale) and a semantic memory bank (Hindsight).

Because atomic distributed transactions across disparate systems do not exist, SQLite and Radicale backups will inevitably have different capture timestamps:

  • Radicale is backed up via its native filesystem storage procedures.
  • SQLite is backed up via online snapshots.

When executing disaster recovery:

  1. Never roll back Radicale to match SQLite: Radicale remains the authoritative calendar store.
  2. Treat timestamp skew as an Observation Gap: If the SQLite snapshot was taken at 02:00 and Radicale at 02:15, the 15-minute discrepancy is flagged as an explicit observation gap (coverage: [{state: "incomplete", ...}]). History is not assumed to be zero.
  3. Rebaseline without fake events: Invalidate stale sync tokens and perform an external rebaseline. Do not synthesize fake deletion or creation events for discrepancies spanning the gap.
  4. Operation Reconciliation: Scan the restored SQLite database for prepared or unknown operations, inspect Radicale state using known operation IDs, and retain uncertain attribution where conclusive evidence is missing.

Integrity Verification & Privacy Erasure (Scenario D12)

Restoration must never assume a snapshot file is valid. Before any service process is permitted to serve traffic or append new operations:

  • The snapshot file checksum must match the cryptographic manifest.
  • The restored database must pass PRAGMA integrity_check and PRAGMA foreign_key_check.
  • The schema version must be validated against the binary's supported migrations.
  • If a corrupted snapshot or missing manifest is detected, restore must immediately abort and require human operator intervention.
  • If an exceptional privacy erasure was executed after the snapshot date, the erasure manifest must be reapplied to the restored database before access is enabled, preventing deleted PII from being resurrected.

Exact Technical References & Specifications

1. Operations Security: Backup & Restore Contract

From docs/technical/operations-security.md (lines 51–66):

"Use SQLite's online backup API or a verified VACUUM INTO snapshot to a new staging path, then validate and encrypt before off-host transfer through existing deployment backup facilities. Do not copy only the live main database: WAL may contain committed transactions. A consistent SQLite snapshot covers its own database, not Radicale/Hindsight simultaneously.

Backup Radicale separately, following its deployment's storage procedure, and record each system's capture time, application/schema/backend versions and checksums in a backup manifest. Hindsight has its own backup lifecycle. Exact destination, frequency, retention, encryption-key custody and recovery objectives are deployment inputs; this document does not invent existing bucket configuration. Monitor snapshot age/failures and rehearse restore before claiming recoverability.

Restore procedure:

  1. Stop all Life writers and preserve the failed/current data directory for evidence; select a trusted matching snapshot/binary/schema version.
  2. Restore into an isolated node-local directory with correct ownership, never overwrite a running database or mix unrelated WAL/SHM files with the snapshot.
  3. Verify checksums, PRAGMA integrity_check, PRAGMA foreign_key_check, expected schema and manifest capture times. Failure aborts startup.
  4. Restore Radicale only if separately required; otherwise current Radicale remains authoritative. Do not roll it back merely to match SQLite. Treat different backup times as an observation gap.
  5. Start recovery with ordinary writes disabled. Scan prepared/unknown operations, inspect remote state and retain uncertain attribution. Invalidate stale sync cursors and perform a backend rebaseline without fake deletions/creation history.
  6. Reconcile outbox known operation IDs before retrying stable document identities. Hindsight restore may have different coverage; report it and repair with deliberate retained-source replay where authorized.
  7. Verify live agenda/ETags, journal appendability, absence of overlapping service instances and explicit unresolved operations. Re-enable writes only where recovery permits. Record lost-history intervals; do not claim exact activity counts through them."

2. Privacy Erasure & Zombie Data Prevention

From docs/technical/operations-security.md (lines 68–72):

"Immutable/offline backups may require expiration rather than immediate deletion. Document the retention window, restrict restoration and reapply erasure manifests before restored data becomes accessible. Erasure can destroy idempotency/history evidence and reduce recovery certainty; disclose that tradeoff and never call the resulting history complete."

3. Verification Scenarios D10, D11, D12

From docs/technical/verification.md (lines 143–145, 155):

ID Input / operation Observable acceptance
D10 Online SQLite snapshot while WAL has committed data Snapshot includes committed state; copying live main file alone is rejected procedure
D11 Encrypted off-host backup restored with differently timed Radicale backup Integrity/foreign-key checks pass before use; operation recovery/rebaseline; capture-time gaps disclosed
D12 Corrupt snapshot, missing manifest or erased-data restore Abort/require explicit resolution; erasure reapplied before access; no false complete history

Normative Verification Principle (lines 155):

"Process-level fault injection around each journal/remote boundary establishes crash behavior. Backup/restore rehearsal runs against disposable state with known markers and validates observable recovery, not only SQL success."

Formal Citations

  • docs/technical/operations-security.md:
    • Section "Backup and restore" (lines 51–66)
    • Section "Privacy and retention" (lines 67–72)
  • docs/technical/timeline-consistency.md:
    • Section "Local durability" (lines 5–10)
    • Section "External reconciliation" (lines 155–159)
  • docs/technical/verification.md:
    • Table "Deployment scenarios", Scenarios D10, D11, D12 (lines 132, 143–145)
    • Section "Test layers" (lines 155)
    • Section "Acceptance sequencing", Gate 5 (lines 160, 166)

What to Build

1. SQLite Online Snapshot Generator (src/storage/backup.rs)

Implement the snapshot engine:

  • create_snapshot(db_conn: &mut SqliteConnection, staging_dir: &Path) -> Result<SnapshotManifest, BackupError>:
    • Generate unique target file name: snapshot_<TIMESTAMP>_<UUID>.sqlite3 in the designated staging directory (outside source and active data directories).
    • Execute SQL: 
      VACUUM INTO ?;
      passing the staging file path as parameter.
    • Open the generated snapshot file independently and execute validation queries: 
      PRAGMA integrity_check;
PRAGMA foreign_key_check;
      Confirm that integrity_check returns ok and foreign_key_check returns zero constraint violation rows.
    • Calculate SHA-256 digest of the snapshot file.
    • Query current schema migration version from schema_migrations.
    • Write snapshot_manifest.json containing: 
      {
  "schema_version": 1,
  "snapshot_file": "snapshot_2030-04-08T02:00:00Z_uuid.sqlite3",
  "sha256": "4b9e28f0...",
  "capture_time": "2030-04-08T02:00:00Z",
  "db_schema_version": 4,
  "binary_version": "0.1.0"
}

2. Snapshot Verification & Restoration Validator (src/storage/restore.rs)

Implement the pre-restore validation pipeline:

  • validate_snapshot(manifest_path: &Path, snapshot_path: &Path) -> Result<(), RestoreError>:
    • Read and parse snapshot_manifest.json. If missing or malformed, return RestoreError::MissingManifest (Scenario D12).
    • Compute SHA-256 of snapshot file and verify exact match with manifest. If mismatch, return RestoreError::ChecksumMismatch (Scenario D12).
    • Open read-only connection to snapshot file and run PRAGMA integrity_check and PRAGMA foreign_key_check. If failed, return RestoreError::CorruptedDatabase.
    • Verify db_schema_version is supported by current binary migrations.
  • restore_into(snapshot_path: &Path, target_data_dir: &Path) -> Result<(), RestoreError>:
    • Ensure target directory contains no active lock or running database.
    • Copy validated snapshot to /var/lib/life-service/life.sqlite3.
    • Ensure no orphaned -wal or -shm sidecar files exist in the target directory.

3. Disaster Recovery Rebaseline & Gap Reporter (src/recovery/disaster.rs)

Implement post-restore reconciliation:

  • Scan operations table in restored database for operations with state IN ('prepared', 'unknown').
  • Connect to Radicale:
    • Query items by uid and inspect remote ETags.
    • Reconcile operation outcomes where evidence exists; mark as needs_reconciliation where uncertain.
  • Invalidate all cached WebDAV sync tokens (sync_token = NULL).
  • Perform full collection enumeration to re-establish the baseline without synthesizing fake deletion events for missing items.
  • Insert a temporal gap marker into life_events covering [snapshot_capture_time, restore_start_time] with kind: "observation_gap" and evidence_class: "confirmed_service", recording the gap interval and collection coverage in after_json (docs/technical/timeline-consistency.md:43–44).

4. Automated Disaster Recovery Rehearsal Suite (tests/disaster_recovery_rehearsal.rs)

Build the integration test suite validating Scenarios D10, D11, and D12:

  • Test Case D10 (Snapshot with Active WAL):
    • Launch service instance against a test database.
    • Insert 50 operations and timeline events within active transactions so that data exists solely in life.sqlite3-wal.
    • Execute create_snapshot(...).
    • Verify that the snapshot file is fully self-contained and querying it returns all 50 operations even though the source database never executed PRAGMA wal_checkpoint(TRUNCATE).
  • Test Case D11 (Asynchronous Multi-System Restore & Gap Reporting):
    • Set up SQLite and mock Radicale.
    • Take SQLite snapshot at T_1.
    • Advance time to T_2: add 2 events directly to Radicale, simulate host crash.
    • Restore SQLite snapshot from T_1 while Radicale remains at T_2.
    • Run disaster recovery coordinator. Assert:
      • Integrity checks pass.
      • Radicale events created between T_1 and T_2 are discovered and recorded as external observations.
      • Zero fake deletion events are emitted.
      • An observation gap event covering [T_1, T_2] is recorded in the timeline.
  • Test Case D12 (Corrupted Snapshot & Missing Manifest):
    • Test Case A: Corrupt 100 bytes in the middle of a snapshot file. Assert restoration validator aborts immediately with ChecksumMismatch / CorruptedDatabase.
    • Test Case B: Attempt restore without snapshot_manifest.json. Assert restoration validator aborts immediately with MissingManifest.
    • Test Case C: Seed an erasure manifest into recovery directory; verify that target PII is wiped from restored database before serving begins.

Acceptance Criteria

Observable Assertions & Verification

  • Database snapshots created via VACUUM INTO ? while WAL contains un-checkpointed transactions produce a single, self-contained SQLite file containing all committed transactions (Scenario D10).
  • Attempting to copy the raw life.sqlite3 file directly during write operations is documented as an explicitly rejected, unsafe procedure (Scenario D10).
  • Restoring an online snapshot alongside a differently-timed Radicale calendar passes PRAGMA integrity_check and PRAGMA foreign_key_check before traffic is admitted (Scenario D11).
  • Post-restore recovery reconciles prepared/unknown operations, invalidates sync tokens, and records an explicit temporal observation gap without fabricating fake creation or deletion events (Scenario D11).
  • Corrupted snapshot files or missing manifests abort restoration immediately with fatal diagnostic errors (Scenario D12).
  • Automated disaster recovery rehearsal suite (tests/disaster_recovery_rehearsal.rs) passes and demonstrates complete recovery of all state and gap reporting.

Notes

  • Snapshot is VACUUM INTO through the single DatabaseWorker connection; the naive main-file-only copy is rejected with evidence (the test shows the copy loses the WAL-resident commit the snapshot keeps).
  • Post-restore reconciliation reuses recover_pending_operations, then invalidates sync cursors and records one observation_gap; fabricated deletion is asserted absent by query and by an exact event-kind set.
  • Deployment inputs deliberately not implemented: encryption, off-host destination, frequency, retention, key custody (documented in the PR).
  • No dependency added.

Blocked by

  • #035: Scoped Bearer Token Auth & PII Diagnostic Redaction
## Summary Implement the robust SQLite online snapshot mechanism (`VACUUM INTO ?` or SQLite Online Backup API) to an isolated staging directory, write atomic snapshot manifests with SHA-256 integrity checksums, and construct an automated disaster recovery rehearsal test suite verifying schema integrity, foreign key consistency, operation reconciliation, and temporal observation gap reporting when restored alongside differently-timed external calendar backups (Scenarios D10, D11, D12). --- ## Background & Technical Rationale ### The WAL Backup Pitfall (Scenario D10) In SQLite Write-Ahead Logging (`WAL`) mode (`PRAGMA journal_mode=WAL`), transactions are written to the `-wal` sidecar file and flushed according to `PRAGMA synchronous=FULL`. If a backup tool or operator executes a standard file copy of the main database (`cp life.sqlite3 /backups/`), the resulting file will be **inconsistent, corrupted, or missing recently committed transactions** whose pages reside exclusively in the WAL. Copying the live database file alone is an **explicitly rejected operational procedure**. Valid snapshots must be produced using SQLite's native Online Backup API (`sqlite3_backup_*`) or an atomic, consistent `VACUUM INTO '<staging_path>'` query executed over an active database connection. This produces a single, self-contained, clean SQLite database file containing all committed transactions without interrupting concurrent read queries. ### Multi-System Asynchrony & Observation Gaps (Scenario D11) The `life` system architecture intentionally couples an append-only local SQLite database (storing operations, proposals, and timeline events) with an external CalDAV store (Radicale) and a semantic memory bank (Hindsight). Because atomic distributed transactions across disparate systems do not exist, SQLite and Radicale backups will **inevitably have different capture timestamps**: - Radicale is backed up via its native filesystem storage procedures. - SQLite is backed up via online snapshots. When executing disaster recovery: 1. **Never roll back Radicale to match SQLite:** Radicale remains the authoritative calendar store. 2. **Treat timestamp skew as an Observation Gap:** If the SQLite snapshot was taken at 02:00 and Radicale at 02:15, the 15-minute discrepancy is flagged as an explicit observation gap (`coverage: [{state: "incomplete", ...}]`). History is not assumed to be zero. 3. **Rebaseline without fake events:** Invalidate stale sync tokens and perform an external rebaseline. Do not synthesize fake deletion or creation events for discrepancies spanning the gap. 4. **Operation Reconciliation:** Scan the restored SQLite database for `prepared` or `unknown` operations, inspect Radicale state using known operation IDs, and retain uncertain attribution where conclusive evidence is missing. ### Integrity Verification & Privacy Erasure (Scenario D12) Restoration must never assume a snapshot file is valid. Before any service process is permitted to serve traffic or append new operations: - The snapshot file checksum must match the cryptographic manifest. - The restored database must pass `PRAGMA integrity_check` and `PRAGMA foreign_key_check`. - The schema version must be validated against the binary's supported migrations. - If a corrupted snapshot or missing manifest is detected, restore must immediately abort and require human operator intervention. - If an exceptional privacy erasure was executed after the snapshot date, the erasure manifest must be reapplied to the restored database before access is enabled, preventing deleted PII from being resurrected. --- ## Exact Technical References & Specifications ### 1. Operations Security: Backup & Restore Contract From `docs/technical/operations-security.md` (lines 51–66): > "Use SQLite's online backup API or a verified `VACUUM INTO` snapshot to a new staging path, then validate and encrypt before off-host transfer through existing deployment backup facilities. Do not copy only the live main database: WAL may contain committed transactions. A consistent SQLite snapshot covers its own database, not Radicale/Hindsight simultaneously. > > Backup Radicale separately, following its deployment's storage procedure, and record each system's capture time, application/schema/backend versions and checksums in a backup manifest. Hindsight has its own backup lifecycle. Exact destination, frequency, retention, encryption-key custody and recovery objectives are deployment inputs; this document does not invent existing bucket configuration. Monitor snapshot age/failures and rehearse restore before claiming recoverability. > > Restore procedure: > 1. Stop all Life writers and preserve the failed/current data directory for evidence; select a trusted matching snapshot/binary/schema version. > 2. Restore into an isolated node-local directory with correct ownership, never overwrite a running database or mix unrelated WAL/SHM files with the snapshot. > 3. Verify checksums, `PRAGMA integrity_check`, `PRAGMA foreign_key_check`, expected schema and manifest capture times. Failure aborts startup. > 4. Restore Radicale only if separately required; otherwise current Radicale remains authoritative. Do not roll it back merely to match SQLite. Treat different backup times as an observation gap. > 5. Start recovery with ordinary writes disabled. Scan prepared/unknown operations, inspect remote state and retain uncertain attribution. Invalidate stale sync cursors and perform a backend rebaseline without fake deletions/creation history. > 6. Reconcile outbox known operation IDs before retrying stable document identities. Hindsight restore may have different coverage; report it and repair with deliberate retained-source replay where authorized. > 7. Verify live agenda/ETags, journal appendability, absence of overlapping service instances and explicit unresolved operations. Re-enable writes only where recovery permits. Record lost-history intervals; do not claim exact activity counts through them." ### 2. Privacy Erasure & Zombie Data Prevention From `docs/technical/operations-security.md` (lines 68–72): > "Immutable/offline backups may require expiration rather than immediate deletion. Document the retention window, restrict restoration and reapply erasure manifests before restored data becomes accessible. Erasure can destroy idempotency/history evidence and reduce recovery certainty; disclose that tradeoff and never call the resulting history complete." ### 3. Verification Scenarios D10, D11, D12 From `docs/technical/verification.md` (lines 143–145, 155): | ID | Input / operation | Observable acceptance | |---|---|---| | D10 | Online SQLite snapshot while WAL has committed data | Snapshot includes committed state; copying live main file alone is rejected procedure | | D11 | Encrypted off-host backup restored with differently timed Radicale backup | Integrity/foreign-key checks pass before use; operation recovery/rebaseline; capture-time gaps disclosed | | D12 | Corrupt snapshot, missing manifest or erased-data restore | Abort/require explicit resolution; erasure reapplied before access; no false complete history | *Normative Verification Principle (lines 155):* > "Process-level fault injection around each journal/remote boundary establishes crash behavior. Backup/restore rehearsal runs against disposable state with known markers and validates observable recovery, not only SQL success." --- ## Formal Citations - `docs/technical/operations-security.md`: - Section "Backup and restore" (lines 51–66) - Section "Privacy and retention" (lines 67–72) - `docs/technical/timeline-consistency.md`: - Section "Local durability" (lines 5–10) - Section "External reconciliation" (lines 155–159) - `docs/technical/verification.md`: - Table "Deployment scenarios", Scenarios D10, D11, D12 (lines 132, 143–145) - Section "Test layers" (lines 155) - Section "Acceptance sequencing", Gate 5 (lines 160, 166) --- ## What to Build ### 1. SQLite Online Snapshot Generator (`src/storage/backup.rs`) Implement the snapshot engine: - `create_snapshot(db_conn: &mut SqliteConnection, staging_dir: &Path) -> Result<SnapshotManifest, BackupError>`: - Generate unique target file name: `snapshot_<TIMESTAMP>_<UUID>.sqlite3` in the designated staging directory (outside source and active data directories). - Execute SQL: ```sql VACUUM INTO ?; ``` passing the staging file path as parameter. - Open the generated snapshot file independently and execute validation queries: ```sql PRAGMA integrity_check; PRAGMA foreign_key_check; ``` Confirm that `integrity_check` returns `ok` and `foreign_key_check` returns zero constraint violation rows. - Calculate SHA-256 digest of the snapshot file. - Query current schema migration version from `schema_migrations`. - Write `snapshot_manifest.json` containing: ```json { "schema_version": 1, "snapshot_file": "snapshot_2030-04-08T02:00:00Z_uuid.sqlite3", "sha256": "4b9e28f0...", "capture_time": "2030-04-08T02:00:00Z", "db_schema_version": 4, "binary_version": "0.1.0" } ``` ### 2. Snapshot Verification & Restoration Validator (`src/storage/restore.rs`) Implement the pre-restore validation pipeline: - `validate_snapshot(manifest_path: &Path, snapshot_path: &Path) -> Result<(), RestoreError>`: - Read and parse `snapshot_manifest.json`. If missing or malformed, return `RestoreError::MissingManifest` (Scenario D12). - Compute SHA-256 of snapshot file and verify exact match with manifest. If mismatch, return `RestoreError::ChecksumMismatch` (Scenario D12). - Open read-only connection to snapshot file and run `PRAGMA integrity_check` and `PRAGMA foreign_key_check`. If failed, return `RestoreError::CorruptedDatabase`. - Verify `db_schema_version` is supported by current binary migrations. - `restore_into(snapshot_path: &Path, target_data_dir: &Path) -> Result<(), RestoreError>`: - Ensure target directory contains no active lock or running database. - Copy validated snapshot to `/var/lib/life-service/life.sqlite3`. - Ensure no orphaned `-wal` or `-shm` sidecar files exist in the target directory. ### 3. Disaster Recovery Rebaseline & Gap Reporter (`src/recovery/disaster.rs`) Implement post-restore reconciliation: - Scan `operations` table in restored database for operations with `state IN ('prepared', 'unknown')`. - Connect to Radicale: - Query items by `uid` and inspect remote ETags. - Reconcile operation outcomes where evidence exists; mark as `needs_reconciliation` where uncertain. - Invalidate all cached WebDAV sync tokens (`sync_token = NULL`). - Perform full collection enumeration to re-establish the baseline without synthesizing fake deletion events for missing items. - Insert a temporal gap marker into `life_events` covering `[snapshot_capture_time, restore_start_time]` with `kind: "observation_gap"` and `evidence_class: "confirmed_service"`, recording the gap interval and collection coverage in `after_json` (`docs/technical/timeline-consistency.md:43–44`). ### 4. Automated Disaster Recovery Rehearsal Suite (`tests/disaster_recovery_rehearsal.rs`) Build the integration test suite validating Scenarios D10, D11, and D12: - **Test Case D10 (Snapshot with Active WAL):** - Launch service instance against a test database. - Insert 50 operations and timeline events within active transactions so that data exists solely in `life.sqlite3-wal`. - Execute `create_snapshot(...)`. - Verify that the snapshot file is fully self-contained and querying it returns all 50 operations even though the source database never executed `PRAGMA wal_checkpoint(TRUNCATE)`. - **Test Case D11 (Asynchronous Multi-System Restore & Gap Reporting):** - Set up SQLite and mock Radicale. - Take SQLite snapshot at $T_1$. - Advance time to $T_2$: add 2 events directly to Radicale, simulate host crash. - Restore SQLite snapshot from $T_1$ while Radicale remains at $T_2$. - Run disaster recovery coordinator. Assert: - Integrity checks pass. - Radicale events created between $T_1$ and $T_2$ are discovered and recorded as external observations. - Zero fake deletion events are emitted. - An observation gap event covering $[T_1, T_2]$ is recorded in the timeline. - **Test Case D12 (Corrupted Snapshot & Missing Manifest):** - Test Case A: Corrupt 100 bytes in the middle of a snapshot file. Assert restoration validator aborts immediately with `ChecksumMismatch` / `CorruptedDatabase`. - Test Case B: Attempt restore without `snapshot_manifest.json`. Assert restoration validator aborts immediately with `MissingManifest`. - Test Case C: Seed an erasure manifest into recovery directory; verify that target PII is wiped from restored database before serving begins. --- ## Acceptance Criteria ### Observable Assertions & Verification - [x] Database snapshots created via `VACUUM INTO ?` while WAL contains un-checkpointed transactions produce a single, self-contained SQLite file containing all committed transactions (Scenario D10). - [x] Attempting to copy the raw `life.sqlite3` file directly during write operations is documented as an explicitly rejected, unsafe procedure (Scenario D10). - [x] Restoring an online snapshot alongside a differently-timed Radicale calendar passes `PRAGMA integrity_check` and `PRAGMA foreign_key_check` before traffic is admitted (Scenario D11). - [x] Post-restore recovery reconciles prepared/unknown operations, invalidates sync tokens, and records an explicit temporal observation gap without fabricating fake creation or deletion events (Scenario D11). - [x] Corrupted snapshot files or missing manifests abort restoration immediately with fatal diagnostic errors (Scenario D12). - [x] Automated disaster recovery rehearsal suite (`tests/disaster_recovery_rehearsal.rs`) passes and demonstrates complete recovery of all state and gap reporting. --- ## Notes - Snapshot is `VACUUM INTO` through the single `DatabaseWorker` connection; the naive main-file-only copy is rejected with evidence (the test shows the copy loses the WAL-resident commit the snapshot keeps). - Post-restore reconciliation reuses `recover_pending_operations`, then invalidates sync cursors and records one `observation_gap`; fabricated deletion is asserted absent by query and by an exact event-kind set. - Deployment inputs deliberately not implemented: encryption, off-host destination, frequency, retention, key custody (documented in the PR). - No dependency added. ## Blocked by - #035: Scoped Bearer Token Auth & PII Diagnostic Redaction
yuris changed title from Issue #036: SQLite Online Vacuum Snapshot & Disaster Recovery Rehearsal to SQLite Online Vacuum Snapshot & Disaster Recovery Rehearsal 2026-09-22 20:02:32 +00:00
yuris closed this issue 2026-09-23 00:58:12 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
v0.2.4
v0.2.3
v0.2.2
v0.2.1
v0.2.0
v0.1.0
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
yuris/life-service#36
No description provided.