wall
45f4dc733d
CLAUDE.md points future sessions at .claude/memory/ for accumulated project context (purpose, SA/MR topology, packaging/signing decisions, open external dependencies) not fully captured elsewhere in the repo.
23 lines
1.2 KiB
Markdown
23 lines
1.2 KiB
Markdown
# Project Purpose
|
|
|
|
This repo is about improving NiFi workflows in two related ways:
|
|
|
|
1. **Performance** — the current pack/unpack path (`MergeContent` /
|
|
`UnpackContent`) is disk-I/O bound on shared vSAN storage, due to
|
|
redundant read/write passes: an ingest write, a separate read to hash
|
|
content, a read+write to merge/pack, a read+write to unpack. vSAN is
|
|
shared, virtualized IOPS across the vSphere cluster, so the pain is
|
|
*contention*, not any single slow disk.
|
|
2. **Integrity & non-repudiation** — FlowFiles packed and exported outside
|
|
NiFi's space need signed attributes (the attributes already carry a
|
|
content hash, so content isn't re-hashed at pack time) to prove both
|
|
integrity and authorship, for S3 storage/retrieval, HTTP/filesystem
|
|
delivery, and selective date/category replay from a catalog database.
|
|
|
|
A future requirement extends this to TB-scale files that must be chunked,
|
|
tracked, and transited through NiFi (for routing/content-assurance
|
|
controls — all content must pass through NiFi) — including across a
|
|
one-way network boundary (MR) with no return channel for retransmission.
|
|
|
|
See [[topology]] and [[packaging-signing-decisions]].
|