wall
45f4dc733d
CLAUDE.md points future sessions at .claude/memory/ for accumulated project context (purpose, SA/MR topology, packaging/signing decisions, open external dependencies) not fully captured elsewhere in the repo.
1.2 KiB
1.2 KiB
Project Purpose
This repo is about improving NiFi workflows in two related ways:
- Performance — the current pack/unpack path (
MergeContent/UnpackContent) is disk-I/O bound on shared vSAN storage, due to redundant read/write passes: an ingest write, a separate read to hash content, a read+write to merge/pack, a read+write to unpack. vSAN is shared, virtualized IOPS across the vSphere cluster, so the pain is contention, not any single slow disk. - Integrity & non-repudiation — FlowFiles packed and exported outside NiFi's space need signed attributes (the attributes already carry a content hash, so content isn't re-hashed at pack time) to prove both integrity and authorship, for S3 storage/retrieval, HTTP/filesystem delivery, and selective date/category replay from a catalog database.
A future requirement extends this to TB-scale files that must be chunked, tracked, and transited through NiFi (for routing/content-assurance controls — all content must pass through NiFi) — including across a one-way network boundary (MR) with no return channel for retransmission.
See topology and packaging-signing-decisions.