Ubuntu
0bae727a26
Defines the SA-V3-Ext/Int placeholder attribute scheme (external list owned by the S3 handover spec, internal list for SA routing + internal S3 usage), the per-egress Regex property for packing/signing selection, the inverse-regex delete step at true external egress, optional ingest-time signing with mandatory validation when already signed, and the S3 JSON-fallback size attribute for suffix-range trailer reads.
46 lines
2.6 KiB
Markdown
46 lines
2.6 KiB
Markdown
# Packaging & Signing — Key Decisions
|
|
|
|
Full detail lives in
|
|
`docs/superpowers/specs/2026-07-28-flowfile-packaging-signing-design.md`.
|
|
These are the non-obvious calls worth remembering, several of which
|
|
reversed an earlier direction during design review:
|
|
|
|
- **No sidecar fronting ingestion.** A claim-check sidecar to bypass
|
|
NiFi's content repo for large payloads was considered and rejected —
|
|
all content must transit NiFi for routing/content-assurance controls.
|
|
The external service's role is post-NiFi only: assembly, stall
|
|
monitoring, cleanup, catalog DB maintenance.
|
|
- **No external signing service.** Vault Transit / a network-isolated
|
|
signer was considered and rejected in favor of in-process signing
|
|
(Bouncy Castle OpenPGP, key loaded via a Controller Service) to keep the
|
|
architecture self-contained within NiFi. This is a deliberate
|
|
non-repudiation trade-off, stated explicitly in the spec — key custody
|
|
now depends on host-level protection (sensitive-property encryption,
|
|
restricting which instances have the Controller Service) rather than
|
|
network isolation.
|
|
- **Clear-sign (OpenPGP), not a bespoke signature format.** Chosen
|
|
specifically because an operator on the far side of MR can verify an
|
|
artifact by hand with stock `gpg --verify` and a public key — no custom
|
|
SDK needed. Matters because MR already has a manual-operator recovery
|
|
model for missing chunks (adopted from the Phat Files design).
|
|
- **HMAC was considered and rejected** as the integrity mechanism — it
|
|
detects tampering but can't give non-repudiation, since every verifier
|
|
necessarily holds the same shared secret and could in principle have
|
|
forged the tag themselves.
|
|
- **Content + JSON trailer as one object, not two.** Chosen for
|
|
end-to-end loss protection: two separate objects (content, manifest)
|
|
have independent failure/lifecycle/replication paths that can diverge.
|
|
A single completed write can't end up in a split state.
|
|
- **S3 attribute handover defers to an existing, established handover
|
|
interface spec**, not this design's own invention — see
|
|
[[open-external-dependencies]].
|
|
- **S3-vs-XFS storage backend split by file size is unresolved** — open
|
|
question whether S3 can serve small files efficiently enough to drop
|
|
the XFS tier entirely.
|
|
- **Attribute list is a controlled allowlist, not "all FlowFile
|
|
attributes."** ~15 controlled external attributes carry through
|
|
ingest-to-delivery (these are what get signed/packed); a separate
|
|
controlled internal set is SA-only and stripped before export. Import
|
|
drops uncontrolled external attributes; export drops internal ones. See
|
|
[[attribute-governance]].
|