33. Training

Decision

Soulforge owns corpus admission, dataset compilation, training-job contracts, candidate lineage, and handoff to independent evaluation/promotion. It never harvests runtime traces by default, equates Karma/consent/repetition/a positive Riddle verdict with corpus admission, mutates serving weights in place, treats trainer telemetry as promotion evidence, selects production routing, or turns Trial Suite feedback into gradients/automatic repair.

nominated evidence → admitted immutable corpus → compiled dataset + sealed holdout
→ isolated Training Run → frozen Candidate Bundle → independent Riddle evaluation
→ explicit promotion decision → versioned serving observation

Every arrow is attributable and may refuse; there is no ambient online self-training.

Records

Training Intent pins Principal, purpose, target capability, base digest, allowed data classes, resources, and promotion authority. Corpus Admission freezes members/exclusions, provenance, transforms, privacy, consent/license, splits, objective, and approval. Dataset Manifest pins examples, lineage groups, split digests, compiler/schema/statistics, and contamination checks. Recipe pins trainer/dependencies, method/hyperparameters/randomness/precision, budgets/stops, and expected change. Training Run pins exact inputs/environment/resources/logs/checkpoints, terminal state, and artifact digests. Candidate Bundle binds base, candidate bytes, tokenizer/config changes, corpus/Recipe lineage, receipt, and compatibility. Promotion Decision binds independent evidence, capability envelope, serving constraints, approver, rollout, rollback, and retirement. Names never replace digests; a re-run creates a new Run/candidate unless its execution contract proves reproducibility.

Corpus admission

Riddle Cases/Outcomes, Mirror attribution, HitL decisions, Archive rows, success, refusal, repetition, storage consent, and database access may nominate material; none grants training rights. Before trainer access, admission proves source/Run/artifact/producer/time/transform lineage; exact purpose, subject, derivative/retention use and approving Principal; training-specific consent, license, or authority; privacy minimization and retained redaction limits; target relevance and label quality; lineage-aware grouping of duplicates, revisions, sibling trajectories, and generated variants; immutable train/development/sealed-holdout splits; base target, learning signal, unacceptable change, expected lift, regression limits, and stops.

The sealed holdout is fixed before training-facing generation and withheld from trainer, augmentation, selection, and model judges used to tune. A negative ledger retains rejected, redacted, and duplicate ids so retry/compiler cannot restore them. Missing provenance, authority, privacy clearance/minimization, relevance, safe split, or uncontaminated holdout blocks admission. Runtime data is opt-in: completion is not quality, and a blocked/refused Run can be useful if objective and evidence support it. Redaction never erases source influence; privatization labels and lineage remain until Security admits narrower disclosure.

Dataset compilation

Only an admitted snapshot may yield an immutable Manifest. Fields trace to source or versioned deterministic transform; it preserves role/tool/workflow/authority context, observations versus human labels/evaluator judgments/augmentations, truthful non-completion, split/group membership, filtering/redaction/truncation/normalization/sampling, and audit statistics for balance, concentration, length, duplication, and exclusion. Schema-valid generation proves no truth.

Holdout answers, benchmark solutions, evaluator rationales, and downstream targets may not leak via prompts, retrieval, augmentation, preference construction, trainer metadata, or selection. A benchmark used to choose the corpus is disclosed and cannot independently promote without an untouched control. The Manifest alone crosses to training.

Training execution

An admitted execution boundary receives one immutable Manifest, base-model/tokenizer/config digest set, Intent/Recipe, admitted secrets/network, and fresh Run-owned output. Orchestrator owns exact accelerator/memory/storage/network/duration/conflicting-Coven request, readiness, affected lease drain, conflict resolution, and restoration—not corpus meaning or trainer success. Training has no universal priority; unused devices stay undisturbed and policy may postpone/refuse.

Isolation requires enforceable filesystem, credential, network, process, and resource policy with receipts; container/Coven labels do not prove it. Local has no privacy exemption; remote records egress, provider custody/retention, secrets/network, returned artifacts, and receipts. The trainer cannot expand corpus with live traces, replace serving, widen resources, or self-promote. LoRA, QLoRA, full tuning, preference optimization, distillation, and later methods are replaceable Recipe ports; each adapter pins its base digest and runtime compatibility. Failure/cancellation preserves terminal truth, logs/checkpoints, and diagnostic quarantined partials. A checkpoint may be evaluated only as pinned candidate material through the same independent handoff; it never becomes serving state. A retry is a new Run, and a partial is never a Candidate Bundle.

Independent evaluation

A frozen Bundle enters Riddle as an exact, immutable subject; Riddle cannot alter candidate, corpus, Recipe, or lineage. Evidence includes sealed target holdout, matched base or currently promoted baseline Outcomes, named regression Trial Suites, adversarial/authority-boundary Cases, and quality/latency/memory/cost in compatible serving Environment, with uncertainty, errors, exclusions, and evaluator calibration. Loss, development improvement, trainer-authored examples, and self-grading are diagnostics. One visible Trial Suite establishes neither general capability nor safety outside its Environment.

Promotion, serving, and rollback

Passing Riddle means eligible, not promoted. Owning policy and Magus/HitL choose whether to promote and bind exact Bundle/Outcomes, admitted capabilities and denied uses, compatible engine and base relation, rollout/observation/stops, fallback/rollback, and retention/quarantine/retirement. Dispatcher consumes a registered capability; files do not create one. Revisions are immutable; supersession is a new Bundle/decision. Rollback routes new work to an earlier digest and quarantines the suspect candidate, never erasing completed effects.

Later privacy, authority, license, contamination, or leakage failure traces exact descendants: Dataset, Run, Bundle, claims, and promotion. Policy may block serving/reuse, quarantine/delete controlled artifacts, invalidate claims, and require clean admission/rebuild. Deletion cannot untrain produced weights. Retrieval remains proper for current, mutable attributable knowledge; exporting all history to a provider or letting trainer select, evaluate, and deploy itself is refused as a collapse of distinct authority.

Consequences

Acceptance evidence

Soulforge remains Designed until one bounded recipe proves admissions/exclusions, lineage-safe splits, sealed holdout, isolated immutable-input execution, candidate custody, independent Riddle evaluation, explicit promotion, compatible serving, and rollback to a prior revision. State of Work records the transition.