NCL Pipe Bootstrap — Config Validation and Secret Injection via Unix Pipeline
Accepted
Context
Ontoref-daemon and any process that receives structured config faces two problems: (1) Nickel NCL requires a subprocess to evaluate, introducing a system-call injection surface if the daemon itself calls `nickel export` at runtime; (2) credentials and secrets embedded in config files (TOML, JSON) persist on disk after the process starts, creating a forensic artifact. The existing registry.toml approach (NCL → TOML file → daemon reads file) partially addresses the first problem but not the second — the TOML file remains on disk with hashed credentials. SOPS and Vault are standard secret management tools that produce decrypted output on stdout.
Decision
All config delivery to long-running processes follows a three-stage Unix pipeline: Stage 1 — structural validation: `nickel export --format json config.ncl` produces JSON with schema-validated structure but no secret values; Stage 2 — secret injection (optional): SOPS decrypt or Vault lookup merges credentials into the JSON stream; Stage 3 — process bootstrap: the target process reads the composed JSON from stdin via `--config-stdin`. No intermediate file is written to disk. If any stage fails, the pipeline breaks and the process does not start. A bash wrapper script (not Nu — Nu may not be available at service boot time) orchestrates the pipeline. A Nu helper `ncl-bootstrap` provides the same interface for interactive/development use.
Constraints
- Hard The bootstrap pipeline must not write an intermediate config file to disk at any stage
- Hard The bash wrapper must depend only on bash, nickel, and the target binary — no Nu, no jq unless SOPS/Vault stage is active
- Hard The target process must redirect stdin to /dev/null after reading the config JSON
- Hard NCL config files used with ncl-bootstrap must not contain plaintext secret values — only SecretRef placeholders or empty fields
Alternatives considered
- TOML file on disk (current registry.toml approach) — rejected: File persists on disk with credentials. Stale file may be read after config changes. Requires explicit cleanup logic. Forensic artifact risk.
- Environment variables for secrets — rejected: Environment variables are visible in /proc/PID/environ on Linux and via `ps eww` on some systems. They persist for the lifetime of the process and are inherited by child processes. Worse attack surface than stdin pipe.
- Encrypted TOML file (AES256 at rest) — rejected: Decryption key must be available at runtime — the problem is deferred, not solved. The decrypted form still passes through disk (tmpfs or swap). Adds a custom encryption layer instead of using standard tools (SOPS, Vault) that the ecosystem already supports.
- Daemon reads NCL directly at runtime via nickel-lang-core — rejected: nickel-lang-core has an unstable Rust API. More critically, it means the daemon can evaluate arbitrary Nickel — including NCL files with system calls via builtins. The pipeline approach ensures the daemon only ever sees validated JSON, never executable Nickel.