wall
45f4dc733d
CLAUDE.md points future sessions at .claude/memory/ for accumulated project context (purpose, SA/MR topology, packaging/signing decisions, open external dependencies) not fully captured elsewhere in the repo.
46 lines
1.9 KiB
Markdown
46 lines
1.9 KiB
Markdown
# CLAUDE.md
|
|
|
|
Project notes for the NiFi prototyping repo — improving NiFi workflow
|
|
performance, packaging/signing of FlowFiles, and large-file (TB-scale)
|
|
transfer across a one-way network boundary.
|
|
|
|
## Before starting work
|
|
|
|
Read everything in `.claude/memory/` first — it holds accumulated notes,
|
|
terminology, and context (some of it non-obvious from the code/docs alone)
|
|
that this file only summarizes. Treat it as living project memory: update
|
|
it as significant new decisions or context emerge, don't just accumulate.
|
|
|
|
## Key documents
|
|
|
|
- `2026-07-23-phat-files-design.md` — approved spec for chunked TB-scale
|
|
S3-to-S3 transfer across a one-way boundary. Source of proven concepts
|
|
(chunking, resend/duplicate handling, server-side assembly) reused by the
|
|
packaging/signing design below — not a dependency of it.
|
|
- `docs/superpowers/specs/2026-07-28-flowfile-packaging-signing-design.md`
|
|
— design for FlowFile packaging (content + JSON trailer), in-process
|
|
OpenPGP signing, uniform chunking, and S3/XFS storage backend handling.
|
|
Status: draft, pending user review.
|
|
- `grand_plan.jpg` — whiteboard sketch of the SA/MR deployment topology the
|
|
design above was reconciled against.
|
|
|
|
## Terminology
|
|
|
|
- **SA** — the overall system: two chains of independent, single-function
|
|
NiFi hosts (linked via Site-to-Site within each chain) bridged by MR.
|
|
- **MR** — the one-way relay/diode boundary between the two NiFi host
|
|
chains. No return channel; forward error correction via an HTTP service,
|
|
~1GB target chunk size.
|
|
|
|
## Environment
|
|
|
|
- On-prem VMs, vSAN-backed storage — shared IOPS across the vSphere
|
|
cluster, so contention (not per-VM disk speed) is the bottleneck.
|
|
- NiFi runs as independent single-node instances (no clustering), each
|
|
with a distinct logical function, scaled horizontally.
|
|
|
|
## Remote
|
|
|
|
- `https://gitea-5001.taild45a1b.ts.net/Hackathon/GrandPlan` (branch
|
|
`main`) — credentials supplied out-of-band, not stored in this repo.
|