# Dendrite Distributed Inference You are interacting with the Dendrite MVP at https://dendrite.airanger.dev. Dendrite is an OpenAI-compatible request router backed by independently running inference provider machines. ## Start here 1. Read this document before attempting an inference request. 2. Read the machine-readable capability card at /.well-known/dendrite-agent.json. 3. Read the API description at /openapi.json. 4. If you have a client credential, call GET /v1/providers and choose an exact model_id from a fresh provider record. 5. Send POST /v1/chat/completions with Authorization: Bearer . ## What this service can do - Accept OpenAI-compatible chat completion requests. - Stream responses as server-sent events when stream is true or omitted. - Return a collected non-streaming chat completion when stream is false. - Route a request to one ready provider advertising the exact requested model. - Run providers on Linux, CUDA, AMD, native Windows, WSL2, CPU, or other supported machines through the Dendrite provider agent. ## What this service cannot claim - This MVP is standard-tier inference only. - Requests use coordinator-plaintext mode. The coordinator can read the chat body, and the provider operator may observe plaintext on the provider host. - This is not tenant-private or confidential inference. - Ordinary CUDA, Windows, WSL2, CPU, and GPU machines are not trusted execution environments merely because they use encrypted transport. - Client-sealed envelopes, attestation verification, confidential key release, billing, payouts, quotas, and private trust tiers are not implemented. - One request is routed to one provider. The MVP does not split layers, tensors, KV cache, or VRAM across multiple machines. - The MVP does not automatically retry or fail over a request after dispatch. Do not blindly retry a request that may already have executed. Do not send sensitive tenant data through this endpoint under the assumption that the provider or coordinator is blinded from it. ## Authentication Client endpoints require the Dendrite client bearer token: Authorization: Bearer The client token is required for: - GET /v1/providers - POST /v1/chat/completions The following discovery endpoints are public and do not require a token: - GET / - GET /agent-instructions.md - GET /llms.txt - GET /.well-known/dendrite-agent.json - GET /openapi.json - GET /healthz Never use a provider token as a client token. Never ask the endpoint to reveal either token. Never place a token in a prompt, URL query string, model name, or repository. ## Selecting a model Call GET /v1/providers with the client token. Use the exact capabilities[].model_id value from a currently fresh provider. Do not invent aliases or assume that a model name is available because it exists in a local model registry. The current scheduler uses provider freshness and exact model matching. Provider readiness, capacity, model digest, and performance claims are advisory in this MVP. The model digest is self-reported and is not independently verified. ## Chat completion request Endpoint: POST https://dendrite.airanger.dev/v1/chat/completions Content-Type: application/json Authorization: Bearer Required JSON fields: { "model": "exact-model-id-from-v1-providers", "messages": [ {"role": "user", "content": "Reply with READY."} ], "stream": true, "max_tokens": 32 } Supported message roles are system, user, assistant, and tool. The service is primarily text inference; do not assume multimodal content is supported just because an upstream engine supports it. When stream is true or omitted, parse server-sent events. Each event contains a data: JSON chunk, followed by a final data: [DONE] marker. When stream is false, the response is a minimal OpenAI-style chat.completion object. The non-streaming finish_reason comes from the provider usage receipt. ## Expected errors - 400: malformed JSON or missing model/messages. - 401: missing or invalid client bearer token. - 503: no fresh provider advertises the exact requested model. - Streaming provider failures arrive as an error event and close the stream. - A timeout or provider disconnect can fail the request. The local engine may continue computing after coordinator timeout, and automatic retry is unsafe. Treat 401 as an authentication/configuration problem. Treat 503 as capacity or provider availability, not proof that the model name is valid. Before retrying, re-check GET /v1/providers and use bounded, caller-approved retry policy. ## Provider participation A machine contributing compute is not an API client. It needs Node.js 22 or newer, the provider agent, a loaded local model, and an OpenAI-compatible local engine serving POST /v1/chat/completions with streaming support. The agent opens an outbound WSS connection to /ws/provider; the provider does not need an inbound public port. Typical engines are vLLM on Linux/CUDA, llama.cpp on CPU/CUDA/AMD/native Windows/WSL2, or MAX/Mojo where supported. The agent does not install drivers, download weights, launch the engine, verify the model digest, or provide confidentiality against the provider operator. Provider authentication uses a separate PROVIDER_SHARED_TOKEN. Provider IDs must be unique within the closed deployment. Only centrally administered providers should receive the shared provider token in this MVP. ## Agent behavior rules - Follow this contract and /openapi.json before calling an endpoint. - Do not guess credentials, model IDs, provider IDs, or privacy guarantees. - Do not call /ws/provider unless you are operating a provider machine and have been given the provider credential and local engine configuration. - Do not expose a local engine directly to the public Internet. - Do not log prompts, completions, bearer tokens, or receipt contents. - Do not describe this endpoint as private, confidential, attested, encrypted end-to-end, or failover-capable. - If an operator asks for capabilities beyond this contract, report that the request requires a future assurance-plane or scheduling implementation.