12. Configuration: The Codex

Context and Problem Statement

Configuration fragmentation creates structural blindness. When intent is scattered across hardcoded paths, environment variables, and implicit runtime state, the system loses determinism. In a Sepulcher that bridges Host and Container, this fragmentation produces port collisions, permission mismatches, and non-reproducible infrastructure states.

A sovereign system requires a single source of truth, strict validation before manifestation, and a deterministic mapping between filesystem topology and runtime behavior.

---

Requirements

• Single Source of Truth: All user intent must reside within a bounded configuration domain.
• Type Authority: Configuration must be validated through explicit schemas before any infrastructure is generated.
• Deterministic Discovery: Filesystem hierarchy must uniquely determine schema ownership and instance identity.
• Fail-Fast Validation: Port conflicts, duplicate singleton declarations, and schema violations must abort loading before Quadlets are written.
• Secret Discipline: Sensitive values must be protected from accidental exposure and validated for permission correctness.
• Privatization Policy: Context egress thresholds and anonymization requirements must be configurable as first-class policy.
• Extensibility Contract: Extensions must integrate into the configuration system without custom parsers or ad-hoc loading logic.
• Infrastructure Integrity: Configuration must be fully validated before container units are manifested.

---

Considered Options

Option 1: Monolithic Configuration

Placing all configuration — global settings and multi-instance infrastructure — into a single lychd.toml.

• Pros: Easy to locate and edit.
• Cons: Structural Degeneration. Instance identity becomes implicit. Ordering matters. Silent overwrites become possible. Extension grafting becomes fragile.

Option 2: Environment-Driven Configuration

Relying primarily on environment variables or distributed .env files.

• Pros: Familiar pattern.
• Cons: Opaque State. Environment variables are invisible to version control and cannot express multi-instance topology. Structural validation cannot occur before process startup.

Option 3: Layered Codex Architecture

Separating configuration into a typed Schema Layer and anchored Rune Schemas.

• Pros:
  • Deterministic Topology: Directory structure defines schema ownership.
  • Type Enforcement: All configuration validated through Pydantic models.
  • Extension Compatibility: Any extension inheriting RuneConfig participates in the same loading model.
  • Fail-Fast Infrastructure: Quadlets are generated only after full validation succeeds.
  • Clear Secret Model: Explicit at-rest protection and runtime boundary definition.

---

Decision Outcome

A layered configuration system is adopted, structured around the Codex. This ADR defines the Codex contract (global settings, rune schemas, ownership, discovery, validation, and loading order). Layout (ADR 13) defines the filesystem geography and mount topology where that contract resides.

The Codex resides at:

~/.config/lychd/

XDG path resolution and Host/Container symmetry for this domain are specified in Layout (ADR 13).

It contains two distinct domains:

1. The Application Settings
2. Rune Schemas

---

1. The Application Settings (lychd.toml)

Contains global configuration only.

Examples include:

• Application-level settings
• Core service configuration
• Coven alliances
• Global defaults

Schema authority for global settings resides in src/lychd/config/settings.py. src/lychd/config/components.py consumes the validated settings object into framework component configuration.

It carries no multi-instance infrastructure definitions.

If secret references are declared in Codex:

• The file must be owned by the Magus.
• File permissions must be 0600.
• Startup validation emits a structured warning if permissions are broader.

The Global config defines global truth.

---

2. The Schema Layer (src/lychd/config/settings.py)

The Schema Layer provides type authority and deterministic loading.

It is implemented using Pydantic and defines:

• Required fields
• Strict typing
• Secret-reference enforcement for credentials (*_secret fields)
• Deterministic source precedence (Settings.settings_customise_sources()):

Init kwargs → Explicit Environment Overrides → Codex `.env` → `lychd.toml` → File Secrets → Model Defaults

Environment variables enter through explicit override channels in the schema loader.
If .env files are used:

• They must reside within the Codex boundary.
• They must be 0600.
• Permission violations produce warnings.

The Schema Layer validates global state before any infrastructure intent is processed.

Exact source precedence and reserved-port validation are implemented in src/lychd/config/settings.py:321:

Live snippet: src/lychd/config/settings.py:321

        codex_issues = codex_permission_issues(PATH_LYCHD_TOML)
        if codex_issues:
            import warnings
            warnings.warn(
                f"codex_permissions_policy_violation: path={PATH_LYCHD_TOML} issues={codex_issues}",
                UserWarning,
                stacklevel=2,
            )

        return self


def ensure_internal_secret_fallbacks(settings: Settings) -> list[str]:
    """Ensure app/db runtime secrets exist even before Podman bind.

    This is a startup safety-net for direct process execution (development,
    tests, or any flow that boots without mounted Podman secrets). It does not
    replace bind-time Podman secret provisioning.
    """
    created: list[str] = []

    if needs_generated_secret_fallback(
        value_env_keys=("APP__SECRET_KEY", "APP_SECRET_KEY"),
        file_env_keys=("APP__SECRET_KEY_FILE", "APP_SECRET_KEY_FILE"),
        default_file=Path("/run/secrets") / settings.app.secret_key_secret,
    ):
        settings.app.set_runtime_secret_key_override(secrets.token_hex(32))
        created.append(settings.app.secret_key_secret)

    if needs_generated_secret_fallback(
        value_env_keys=("DB__PASSWORD", "DB_PASSWORD"),
        file_env_keys=("DB__PASSWORD_FILE", "DB_PASSWORD_FILE"),
        default_file=Path("/run/secrets") / settings.db.password_secret,
    ):
        settings.db.set_runtime_password_override(secrets.token_urlsafe(16))
        created.append(settings.db.password_secret)

    return created


@lru_cache(maxsize=1, typed=True)
def get_settings() -> Settings:
    settings = Settings()
    ensure_internal_secret_fallbacks(settings)
    return settings

---

3. Rune Schemas (Instance Scrolls)

Infrastructure intent is declared through Rune Schemas.

Each schema:

• Inherits from RuneConfig
• Declares relative_path rooted at ~/.config/lychd/runes/ (or None for runes root)
• Uses singleton: bool | None where None enables auto topology inference

Rune instances are stored as TOML files under their Anchor directory.

Example structure:

~/.config/lychd/
lychd.toml
runes/
  animator/
    animator.toml
    soulstones/
      vision.toml
      ocr.toml
    portals/
      openai.toml

---

The Anchor Doctrine

Each RuneConfig schema owns exactly one anchor territory.

• Folder location determines schema type.
• Anchors may not overlap.
• No internal type= switching is permitted.
• The filesystem hierarchy is authoritative.

Schema ownership is structural, not dynamic.

Loader enforcement for anchor scanning, singleton exclusivity, top-level TOML payload, and duplicate identity rejection lives in src/lychd/config/runes/loader.py:19:

Live snippet: src/lychd/config/runes/loader.py:19

class ConfigLoader:
    """Validation loader for explicit RuneConfig schemas."""

    def __init__(self, runes_dir: Path | None = None) -> None:
        """Create a loader for runes under a specific root."""
        self._runes_dir = runes_dir or RuneConfig.config_root

    def load_all(self, schemas: Sequence[type[RuneConfig]]) -> list[RuneConfig]:
        """Load and validate all rune instances for explicit schema classes."""
        loaded: list[RuneConfig] = []

        for cls in schemas:
            loaded.extend(self._load_class_instances(cls))

        logger.debug("runes_loaded", count=len(loaded), classes=[c.__name__ for c in schemas])
        return loaded

    def _load_class_instances(self, cls: type[RuneConfig]) -> list[RuneConfig]:
        files = self._candidate_files(cls)
        instances: list[RuneConfig] = []

        for file_path in files:
            payload = self._read_payload(file_path, cls)
            instance = cls.model_validate(payload).with_file_name(file_path)
            instances.append(instance)

        if cls.effective_singleton() and len(instances) > 1:
            msg = f"Schema '{cls.__name__}' is singleton but found {len(instances)} files in '{cls.anchor_dir()}'."
            raise RuneConfigError(msg)

        self._assert_unique_identity(cls, files)
        return instances

    def _candidate_files(self, cls: type[RuneConfig]) -> list[Path]:
        if cls.relative_path is None:
            singleton_path = self._runes_dir / cls.default_file_name()
            return [singleton_path] if singleton_path.exists() else []

        anchor = self._runes_dir / cls.relative_path
        if not anchor.exists():
            return []

        files = sorted(anchor.rglob("*.toml"))
        if not files:
            return []

        # Parent schemas do not sweep child anchors.
        child_anchors = [self._runes_dir / rel for rel in self._descendant_relative_paths(cls)]
        if not child_anchors:
            return files

        return [path for path in files if not any(path.is_relative_to(child) for child in child_anchors)]

    def _descendant_relative_paths(self, cls: type[RuneConfig]) -> set[Path]:
        paths: set[Path] = set()
        for sub in cls.__subclasses__():
            if sub.relative_path is not None:
                paths.add(sub.relative_path)
            paths.update(self._descendant_relative_paths(sub))
        return paths

    def _read_payload(self, file_path: Path, cls: type[RuneConfig]) -> dict[str, Any]:
        try:
            parsed = tomllib.loads(file_path.read_text(encoding="utf-8"))
        except tomllib.TOMLDecodeError as exc:
            msg = f"Malformed TOML in '{file_path}'."
            raise RuneConfigError(msg) from exc
        except OSError as exc:
            msg = f"Could not read '{file_path}'."
            raise RuneConfigError(msg) from exc

        content: dict[str, Any] = {str(k): v for k, v in parsed.items()}
        if "model" in content and isinstance(content["model"], dict) and "model" not in cls.model_fields:
            msg = (
                f"File '{file_path}' uses legacy '[model]' envelope syntax. "
                "Rune payload must be written at TOML top level."
            )
            raise RuneConfigError(msg)

        return content

    def _assert_unique_identity(self, cls: type[RuneConfig], files: list[Path]) -> None:
        seen: set[str] = set()

---

The Instance Doctrine

Within an Anchor:

• One TOML file equals one instance.
• Instance payload resides at TOML top level (arrays-of-tables are forbidden for instance encoding).
• Instance identity is derived from relative path.
• Duplicate identity across files is forbidden.

If a schema resolves as singleton, only one file may exist.

Violations abort configuration loading.

---

The Leaf Principle

Only leaf schemas (those without subclasses) may define multiple instances by default.

Non-leaf schemas are singleton by default.

Explicit singleton=True/False overrides auto behavior.

This prevents ambiguous discovery and implicit polymorphic loading.

4. The Configurable Contract (Extension Integration)

Configuration extensibility is governed by a single structural contract: RuneConfig.

Any extension that wishes to declare configuration must:

• Inherit from RuneConfig
• Declare relative_path (or None for top-level singleton)
• Optionally declare singleton override

Example contract and helper methods are defined directly in src/lychd/config/runes/base.py (see RuneConfig snippet above).

Registration Doctrine

Configuration schemas are registered structurally, not procedurally.

• The subclass itself is the registration signal.
• Runtime import plus subclass discovery determines ownership.
• No extension may implement custom configuration loaders.
• No dynamic type= dispatch is permitted.

Inheritance automatically binds the schema to:

~/.config/lychd/runes/<relative_path>/

If an extension is installed:

1. Its RuneConfig subclasses are discovered.
2. Their Anchors become valid Codex territories.
3. One TOML file equals one instance and payload lives at TOML top level.
4. Instances located in those directories are validated and loaded.
5. Validated instances become infrastructure intent.

Extension import + inheritance registers schema ownership under the shared loader.

The Codex loader remains singular and authoritative.

---

Structural Guarantees

This model ensures:

• Extensions cannot fragment configuration loading.
• Configuration remains globally validated.
• Infrastructure manifestation remains downstream of schema authority.
• Removal of an extension invalidates only its anchor territory.

Configuration extensibility is therefore achieved without sacrificing determinism.

5. Port Arbitration

Port ownership is validated before Quadlet generation.

The validator aggregates:

• Reserved core ports
• Ports declared by Rune Schemas

If any collision is detected:

• Configuration loading fails immediately.
• No Quadlets are written.

Infrastructure is never generated from invalid state.

---

6. Runtime Realization (lychd init)

At runtime, initialization follows a deterministic inscription path:

1. lychd init calls CodexService.inscribe().
2. RuneSchemaDiscovery.discover_classes() imports built-in extensions and discovers all RuneConfig subclasses.
3. ConfigWriter.initialize_anchors() materializes all anchor directories.
4. ConfigWriter.inscribe_samples() writes one sample TOML per schema only when no instance file exists yet.

This keeps extension configuration registration structural (inheritance/import), not procedural. Animation follows the same path: AnimatorLoader consumes the same RuneConfig runes under runes/animator/.

Discovery import + subclass traversal (RuneSchemaDiscovery) is implemented in src/lychd/config/runes/discovery.py:10:

Live snippet: src/lychd/config/runes/discovery.py:10

class RuneSchemaDiscovery:
    """Discover RuneConfig schemas after runtime import/bootstrap."""

    def __init__(
        self,
        *,
        include_builtin_extensions: bool = True,
        external_packages: Iterable[str] = (),
    ) -> None:
        """Create a schema discoverer.

        Args:
            include_builtin_extensions: Import built-in extension modules.
            external_packages: Additional extension package trees to import.

        """
        self._include_builtin_extensions = include_builtin_extensions
        self._external_packages = tuple(external_packages)

    def discover_classes(self) -> list[type[RuneConfig]]:
        """Return all currently imported RuneConfig subclasses."""
        # Import core schema module before subclass traversal.
        importlib.import_module("lychd.domain.animation.schemas")

        discovery = ExtensionDiscovery()
        if self._include_builtin_extensions:
            discovery.import_builtin_modules()
        if self._external_packages:
            discovery.import_packages(self._external_packages)

        discovered: set[type[RuneConfig]] = set()
        self._collect_subclasses(RuneConfig, discovered)
        filtered = [cls for cls in discovered if self._is_allowed_schema_module(cls)]
        return sorted(filtered, key=lambda cls: (cls.__module__, cls.__qualname__))

    def _collect_subclasses(self, parent: type[RuneConfig], discovered: set[type[RuneConfig]]) -> None:
        for subclass in parent.__subclasses__():
            self._collect_subclasses(subclass, discovered)
            discovered.add(subclass)

    def _is_allowed_schema_module(self, cls: type[RuneConfig]) -> bool:
        module = cls.__module__
        allowed_packages = ("lychd", *self._external_packages)
        return any(module == package or module.startswith(f"{package}.") for package in allowed_packages)

Anchor creation and sample inscription (ConfigWriter) are implemented in src/lychd/config/runes/writer.py:16:

Live snippet: src/lychd/config/runes/writer.py:16

    def __init__(self, runes_dir: Path | None = None) -> None:
        """Create a writer for runes under a specific root."""
        self._runes_dir = runes_dir or RuneConfig.config_root

    def initialize_anchors(self, schemas: list[type[RuneConfig]]) -> None:
        """Ensure all schema anchor directories exist."""
        for schema in schemas:
            anchor = self._runes_dir if schema.relative_path is None else self._runes_dir / schema.relative_path
            anchor.mkdir(parents=True, exist_ok=True)
            logger.debug("anchor_initialized", schema=schema.__name__, anchor=str(anchor))

    def inscribe_samples(self, schemas: list[type[RuneConfig]]) -> list[Path]:
        """Write one sample TOML per schema when no files exist yet."""
        created: list[Path] = []

        for schema in schemas:
            target = self._target_sample_file(schema)
            if target is None:
                continue

            target.parent.mkdir(parents=True, exist_ok=True)
            target.write_text(self._render_sample(schema), encoding="utf-8")
            created.append(target)
            logger.info("rune_sample_inscribed", schema=schema.__name__, path=str(target))

        return created

    def _target_sample_file(self, schema: type[RuneConfig]) -> Path | None:
        """Return sample target path if schema has no existing TOML instances."""
        if schema.relative_path is None:
            target = self._runes_dir / schema.default_file_name()
            return None if target.exists() else target

        anchor = self._runes_dir / schema.relative_path
        existing = list(anchor.rglob("*.toml")) if anchor.exists() else []
        child_anchors = [self._runes_dir / rel for rel in self._descendant_relative_paths(schema)]
        if child_anchors:
            existing = [path for path in existing if not any(path.is_relative_to(child) for child in child_anchors)]
        if existing:
            return None

        return anchor / schema.default_file_name()

    def _descendant_relative_paths(self, schema: type[RuneConfig]) -> set[Path]:
        paths: set[Path] = set()
        for sub in schema.__subclasses__():
            if sub.relative_path is not None:
                paths.add(sub.relative_path)
            paths.update(self._descendant_relative_paths(sub))
        return paths

    def _render_sample(self, schema: type[RuneConfig]) -> str:
        lines: list[str] = []

---

7. Assembly Pipeline

The configuration lifecycle proceeds in strict order:

1. Load Global config.
2. Validate Schema Layer.
3. Discover Anchored Rune Schemas.
4. Validate each instance.
5. Enforce:
6. Singleton exclusivity
7. Duplicate identity rejection
8. Port arbitration
9. Only after full validation:
10. Generate Quadlets.

Infrastructure is a manifestation of validated intent.

---

8. Secret Covenant

Secrets are declared by reference:

• Codex stores secret names (*_secret) only.
• Soulstones may map runtime env vars to secret names via secret_env_files.
• Values live in rootless Podman secret storage.
• Generated Quadlets bind them through Secret= directives.

Secrets:

• Are not stored inline in lychd.toml or rune TOMLs
• Are mounted only into units that require them
• Are accessible to the process boundary that consumes them

Example lifecycle:

printf '%s' "$OPENAI_API_KEY" | podman secret create --replace portal_openai_main -
podman secret ls

Filesystem permissions protect secrets from other host users.
They do not protect secrets from code executing within the same Quadlet unit.

If isolation from agent-level execution is required, the secret must reside in a separate service boundary.

---

9. Context Privatization Policy

Privatization policy is configured in the Codex and enforced by runtime dispatch:

• portal_threshold: minimum weight requiring anonymization before portal egress.
• forbidden_threshold: minimum weight forbidding raw portal egress.
• require_anonymization_workflow: fail-closed if no sanitization path exists.

Canonical source is Codex (lychd.toml). Phylactery-backed policy records, when enabled for adaptive tuning, must remain equal or stricter than the Codex baseline.

Conceptual shape:

[security.privatization]
portal_threshold = 0.40
forbidden_threshold = 0.70
require_anonymization_workflow = true

---

10. Dual-Plane Trust Delta

Configuration is now split by trust boundary:

• Vessel config is the only source of truth for secrets, persistence, and policy.
• The Tomb config is a generated runtime envelope with only task-safe fields.
• The Tomb config is derived data, never an alternate source of truth.
• The Tomb schema forbids secret fields and infrastructure authority fields.
• Provider/API keys are never serialized into Tomb payloads.
• The Tomb cannot override queue, network, or authority policy.

5. Authority Matrix

Dimension	Vessel (Trusted Control Plane)	The Tomb (Untrusted Execution Plane)
Secrets	Loaded via Podman secret references and mounted into trusted units.	Secret fields are forbidden by schema.
Mounts	Codex-backed config and durable state mounts.	Sanitized task-scoped config artifact only.
Network	Resolves provider and broker routes per policy.	No direct secret-bearing provider routes.
Queue Ownership	Owns queue workflow configuration.	No queue configuration ownership.
Context Privatization	Defines thresholds and anonymization policy for portal egress.	Cannot lower thresholds or bypass sanitization gates.
Authority Boundaries	Defines and signs runtime envelopes.	Consumes envelope; cannot redefine authority.

Consequences

Positive

• Deterministic Topology: Filesystem structure defines schema ownership and instance identity.
• Fail-Fast Guarantees: Invalid configuration aborts before infrastructure generation.
• Extension Uniformity: Any extension inheriting RuneConfig integrates without special handling.
• Clear Trust Boundaries: Secret visibility is explicit and aligned with process boundaries.

Negative

• Strict Structural Discipline: Incorrect directory placement or duplicate identity causes immediate failure.
• Shared Process Trust Domain: Secrets available to a Quadlet unit are accessible to code within that unit.
• Operational Responsibility: File permissions must be maintained to preserve at-rest protection.