Files
GrandPlan/.claude/memory/topology.md
T
wall 45f4dc733d Add CLAUDE.md, project memory notes, and token_count
CLAUDE.md points future sessions at .claude/memory/ for accumulated
project context (purpose, SA/MR topology, packaging/signing decisions,
open external dependencies) not fully captured elsewhere in the repo.
2026-07-28 14:27:05 +10:00

1.3 KiB

SA/MR Deployment Topology

SA is the name of the whole system pictured in grand_plan.jpg: two chains of independent, single-function NiFi hosts (each host linked to the next via Site-to-Site) bridged in the middle by MR.

MR is the one-way relay (diode-style boundary) between the two chains — forward error correction via an HTTP service, ~1GB target chunk size, no return channel for retransmission requests.

Content flows one direction through SA: external S3/S2S in -> first NiFi chain (node 1 -> 2 -> 3) -> MR -> second NiFi chain (node 1 -> 2 -> 3) -> external S3/S2S out. The Replay DB branches off before MR, via a "copy" from node 1 of the first chain — i.e. at the true ingest point into SA, not after delivery. This was a real correction during design review — an earlier draft had the Replay DB positioned after MR, which was wrong.

Chunking and hashing both happen at that same ingest point (node 1 of the first chain), in a single streaming pass, so the same chunked representation transits both the intra-chain S2S hops and the MR relay uniformly — no separate chunking scheme per transport, and no size threshold decides whether to chunk (a small file is just the one-chunk case).

Source: whiteboard sketch grand_plan.jpg, reconciled into docs/superpowers/specs/2026-07-28-flowfile-packaging-signing-design.md.