ci(ingestion): scheduled CI/CD workflow to compile and sync PMTiles archives to Cloudflare R2 #102

New issue

Closed

opened 2026-10-15 00:35:21 +00:00 by yuris · 0 comments

yuris commented

2026-10-15 00:35:21 +00:00

Owner

This was generated by AI during triage.
State: ready-for-human / hold (Requires R2 infrastructure credentials from Issue #101: Do NOT trigger autonomous implementers)

Context & Purpose

To keep Git repository history lean and ensure production vector tiles are fresh, the spatial compilation pipeline must run in an ephemeral CI/CD environment or scheduled workflow rather than burning persistent worker daemon memory.

Because cadastral boundaries and peatland depth maps change slowly (monthly / quarterly), a scheduled monthly workflow will re-scrape government GIS layers and push fresh .pmtiles archives to Cloudflare R2.

Technical Specification

Workflow File (.forgejo/workflows/sync-tiles-r2.yml):
- Triggers:
  - schedule: - cron: "0 2 1 * *" (2:00 AM UTC on the 1st of every month).
  - workflow_dispatch (manual trigger).
- Steps:
  1. Checkout repository.
  2. Set up Python 3.12 and Nix / Tippecanoe.
  3. Run python3 scripts/ingest-real-concessions.py (fetches 654 authentic boundaries from BIG Satu Peta to .cache/).
  4. Run bash scripts/build-concessions-pmtiles.sh (emits public/tiles/concessions.pmtiles).
  5. Upload to Cloudflare R2 bucket using scripts/upload-tiles-r2.mjs or rclone with repository secrets.
  6. Trigger Cloudflare Cache Purge for updated tiles.

Acceptance Criteria

.forgejo/workflows/sync-tiles-r2.yml authored and committed.
Manual workflow_dispatch run successfully compiles and uploads tiles to R2.
No large binary or GeoJSON artifacts are committed to Git.

> *This was generated by AI during triage.* > **State:** `ready-for-human` / `hold` (Requires R2 infrastructure credentials from Issue #101: Do NOT trigger autonomous implementers) ## Context & Purpose To keep Git repository history lean and ensure production vector tiles are fresh, the spatial compilation pipeline must run in an ephemeral CI/CD environment or scheduled workflow rather than burning persistent worker daemon memory. Because cadastral boundaries and peatland depth maps change slowly (monthly / quarterly), a scheduled monthly workflow will re-scrape government GIS layers and push fresh `.pmtiles` archives to Cloudflare R2. ## Technical Specification 1. **Workflow File (`.forgejo/workflows/sync-tiles-r2.yml`)**: - Triggers: - `schedule: - cron: "0 2 1 * *"` (2:00 AM UTC on the 1st of every month). - `workflow_dispatch` (manual trigger). - Steps: 1. Checkout repository. 2. Set up Python 3.12 and Nix / Tippecanoe. 3. Run `python3 scripts/ingest-real-concessions.py` (fetches 654 authentic boundaries from BIG Satu Peta to `.cache/`). 4. Run `bash scripts/build-concessions-pmtiles.sh` (emits `public/tiles/concessions.pmtiles`). 5. Upload to Cloudflare R2 bucket using `scripts/upload-tiles-r2.mjs` or `rclone` with repository secrets. 6. Trigger Cloudflare Cache Purge for updated tiles. ## Acceptance Criteria - [x] `.forgejo/workflows/sync-tiles-r2.yml` authored and committed. - [x] Manual `workflow_dispatch` run successfully compiles and uploads tiles to R2. - [x] No large binary or GeoJSON artifacts are committed to Git.

yuris referenced this issue from a pull request that will close it,

2026-10-15 00:57:07 +00:00

ci(ingestion): scheduled CI/CD workflow to compile and sync PMTiles archives to Cloudflare R2 #104

yuris closed this issue

2026-10-15 01:21:53 +00:00