Your own model servers

On the Enterprise plan an organization can connect the AI servers it runs itself — Ollama, vLLM, LM Studio, llama.cpp, TGI, or a gateway in front of them. Anything that speaks the /v1/chat/completions standard works.

Vanguard stays the layer around your model: the chat, Agent Builder, /v1/responses, tools, skills, project knowledge and guardrails all work as before. The answers come from your server.

A message on your own model costs no credits. Tools that run on Vanguard (making a document, OCR, speech) keep their usual price.
A conversation on your model stays on it. If the model is switched off or its server is removed, the conversation stops answering — it never moves to a Vanguard model unless someone confirms that move.
By default nothing from those conversations goes to Vanguard’s cloud AI. An owner or admin can allow individual features (below).

Connect a server

In Settings → Models, choose Add a server.
Pick the kind, give it a name, and enter its address, for example https://gpu.example.com/v1.
Add the API key if your server needs one, and any extra headers (one Name: value per line).
Press Test connection. The Test lists the models your server offers.

The address must be HTTPS and reachable from the internet. A server inside a private network can use the Connector (below), or be published through Cloudflare Tunnel with Cloudflare Access: put the service token in the extra headers as CF-Access-Client-Id and CF-Access-Client-Secret.

Keys and headers are stored encrypted and are never shown again; leave a field blank to keep what is stored.

A server on a private network: the Connector

If your model server is not reachable from the internet, run the Vanguard Connector on a machine that can reach it. The connector dials out to Vanguard over HTTPS (port 443) and carries model requests back, so you open no inbound port.

In Settings → Models → Connectors, create a connector and copy the command. Its token is shown only once.
Run it next to your model server:

docker run -d --name vanguard-connector --restart unless-stopped \
  -e VANGUARD_TOKEN=vc_… \
  -e UPSTREAM_URL=http://ollama:11434/v1 \
  ghcr.io/vanguardinitiative/vanguard-connector:latest

Add a server, choose Through a connector, and pick it.

What it does and does not do:

It only ever makes four requests to your server: list models, a chat completion, embeddings, and Ollama’s own model list. Anything else is refused by the connector itself.
If your server needs a key, give it to the connector as UPSTREAM_API_KEY. That key stays on your network.
It reconnects by itself after a network blip or a Vanguard update. Revoking it in Settings → Models disconnects it at once.

Add a model

Choose Add a model under a server, pick the model from the list, and set:

Setting	What it means
Context window	How many tokens the server lets the model hold, prompt and answer together. Vanguard never sends more than this.
Can call tools	Offer tools (documents, sheets, pages, skills…) to this model.
Can read pictures	Send attached pictures to it. Otherwise it is told a picture is there.
Tools offered	None, Basic, Standard or Full. Small models answer better with fewer tools.

A new model starts switched off. Press Test: Vanguard checks an answer, tool calling, pictures (if claimed), and that a long prompt reaches the model whole. When the Test passes, switch the model on. Changing how a model runs later needs a new passing Test.

Server tips

Ollama cuts a prompt longer than its context setting from the front, silently. Set OLLAMA_CONTEXT_LENGTH to the context window you enter here — the Test catches a mismatch.
vLLM needs --enable-auto-tool-choice and a --tool-call-parser for tool calling.
llama.cpp needs --jinja for tool calling.
Models that write their thinking inside <think> tags: set Thinking → Inside think tags on the server.

What may leave your model

Each switch lets one feature send content from a conversation on your own model to Vanguard’s cloud AI. All are off until you turn them on.

Switch	When it is off
Search project knowledge with Vanguard’s cloud	Documents are not indexed by our cloud and scans are not read; chats read documents whole. Applies to the whole organization.
Web search	Not offered to your models.
Pictures	Image mode and picture making are off on your models.
OCR, speech and document services	Not offered to your models as tools.
Helper tasks	Memory, picture briefs and page checks run on your utility model, or are skipped.

You can also hide Vanguard’s models from your members, choose the default model for new chats, and choose the utility model.

Using your models from the API

Each model you add gets an id like pm_x3k9…. Use it wherever the API takes a model — /v1/chat, /v1/agents, /v1/responses. It works only with keys of your organization. GET /v1/chat/private-models lists them for the signed-in organization.

Errors from your server come back with codes that start with PRIVATE_MODEL_ — see Errors.