vLLM

vLLM is a resident model-serving engine behind an OpenAI-compatible connector. The Soulstone adapter binds one declared model service to a Rune, derives its model capability, probes the live inventory, and exposes it only when the exact declared model is ready.

A server pinned to one model produces is_dynamic=False capabilities. Reachability establishes link liveness; a capability becomes WARM only when the validated live /models inventory also contains its exact declared model id. Malformed inventory or a missing id becomes ERROR. Focused tests cover current planning and connector behavior.

vLLM framework arguments come from the operator-owned exec list, which the adapter passes through verbatim. Image, devices, mounts, ports, and environment remain typed Soulstone and Quadlet fields. Keep the command's served model id and endpoint consistent with the Rune; these declarations do not synthesize vLLM flags.

The exec arguments follow the selected image's entrypoint. The generated example expects ["vllm", "serve"] and begins with the model path; adding another serve would repeat the subcommand. Inspect and pin the actual image entrypoint before adapting a recipe. A different entrypoint needs its matching argument vector; the adapter does not rewrite it. The example's environment uses an ordinary TOML table and remains inactive until deliberately edited.

Bring one service to a first reply

For an already running local service, the standalone acceptance probe exercises explicit built-in registration, exact Dispatcher grants, AgentForge text output, and one harmless in-memory tool round. It reads no installation Settings or Runes and cannot start, stop, or load a runtime:

timeout --signal=TERM --kill-after=5s 40s \
  uv run --locked --no-sync python -m scripts.local_model_probe \
  --base-url http://127.0.0.1:20000/v1 \
  --model-id YOUR-SERVED-MODEL-ID \
  --source-revision "$(git rev-parse HEAD)"

Supply the endpoint and exact model id you intend to test. The probe accepts only literal loopback HTTP endpoints with an explicit port. It sends synthetic inputs, runs the two cases sequentially with finite request/output limits and a cooperative deadline, and writes one JSON result to stdout; diagnostics use stderr. Success requires the model to return a fresh marker obtained from the actual tool call and leave no live capability lease. Each response must identify the declared model and carry an explicit provider finish reason matching stop or tool_call before the Agent may process it. The SDK's default for an absent wire reason does not qualify. Filtered, truncated, errored, or unknown completion states fail even if their text contains the marker. A failed case retains any response metadata already observed by the probe. A failed or cancelled call does not prove that the externally operated server has finished its work. Shielded client cleanup can outlast the cooperative deadline; the outer process timeout bounds the command. A missing final JSON receipt after forced termination leaves cleanup and server outcome unknown and blocks a pass.

Retain the JSON outside the checkout with the independently observed image/model/host identities. The supplied source revision is a declaration, not a clean-checkout or artifact attestation. This probe qualifies neither an installed Vessel nor Bridge, streaming, persistence, managed activation, runtime-contributed tools, capacity, shutdown, or recovery.

Choose the boundary being tested before adapting an existing serving recipe:

A working external server is useful input to the first boundary. It does not transfer ownership of that container or establish the other two. dedicated = false withholds lifecycle authority; it does not adopt an existing container, supply capacity admission, or make a cold service startable. Retain the existing recipe until its replacement has its own acceptance and recovery evidence, and do not run competing model copies on the same devices.

Bind the image digest, model artifact and served id, command, environment, mounts, devices, context/output limits, and tested concurrency to the receipt. The generated Rune sample is an editable example, not a benchmark-qualified recipe. Check the endpoint from the Vessel's network namespace: host loopback and the loopback inside its pod are different destinations. An endpoint override alone does not prove reachability or admit a new network route.

For a repeatable Bridge trial, use the exact capability selector with revision 2 and a capability declaring chat and tools. Verify the Run's pinned Pattern revision in Loom and its recorded capability dispatch in Orb. Capability-key pinning does not pin model bytes or the runtime generation; keep those identities separately in the receipt. Start with one admitted request at a time. Before increasing concurrency, measure waiting, timeouts, cancellation and useful completed work on the exact serving profile; engine sequence count and LychD lease count are different bounds.

Use Summoning's host and first-light checks for the surrounding body. Its worked model recipe is llama.cpp; a vLLM trial must substitute its own qualified declaration and record the actual runtime names and shutdown path. A connector smoke test cannot stand in for that composed receipt.

The exact model, quantization, GPU topology, image digest, arguments, readiness, inference, and shutdown still require an operator receipt. The current delivery boundary is recorded under vLLM integration.

Join one reply to durable Bridge

The former warm-model TestClient, sequential-spawn and source-native TCP selections are frozen historical observations, retired on 2026-10-20 at base 7057210a. They recorded one actual structured Bridge reply and identical retained turns/model history after a second lifespan or process boot, with explicit test Settings/Runes and refusing host actuation. Their raw receipts and historical validators remain; no old source hash is refreshed to claim current qualification.

The former source-native TCP launcher and host Core test deployment are removed. There is no host pytest fallback. A new proof must execute Core in the explicit OCI test image or, for installed runtime acceptance, the actual rebuilt Vessel image with its declared PostgreSQL and security boundary. External warm-model use, daemon/socket authority, profiles, resource bounds and terminal cleanup need fresh admission; the ordinary wrapper does not forward arbitrary external profile files or proof environment. No replacement container runtime receipt is claimed.

The frozen trusted browser and installed-hybrid observations retain their original private manager, plain Docker, inert model-unit and borrowed warm-vLLM limits. They do not qualify the current fixture, production image, managed-model lifecycle or active container stop/start. State of Work owns the exact historical envelope and current proof gaps.