I generated a documentation site with the Doccupine CLI, gave it no API key of any kind, and went looking for the chat button. It wasn't there. Search worked, the pages worked, and the only trace of an assistant was an endpoint that answered "The AI assistant is not configured on this site" if you went out of your way to call it.
I like that. AI on a docs site should be off until someone decides to turn it on, and turning it on is a decision, not a checkbox.
Adding AI chat to a documentation site comes down to four things: something that finds the right pages for a question, a model that answers only from those pages and shows which ones it used, a way to keep strangers from running up your bill, and a way to switch the whole thing off. With the Doccupine CLI the first two are already in the generated site, and switching them on is two lines in an env file. The last two are the part most guides skip, so that is where I will spend most of this post.
What the assistant actually does
It is not "trained on your docs." Every time a reader asks something, the site looks up the handful of pages that best match the question and hands them to a model with one instruction: answer from these, and nothing else.
That is the whole trick, and it has a consequence people underestimate. The assistant knows exactly what your documentation says, and not one thing more. If a page is right, the answer is right and comes with a link. If a page is wrong, the answer is wrong and still comes with a link. The chat is a mirror for your docs, and it doesn't flatter.
Turning it on
Pick a provider and put its key in the generated app's .env file:
LLM_PROVIDER=openai
OPENAI_API_KEY=sk-...Restart, and the chat button appears in the header. Cmd+I opens it, and the search box grows an "Ask AI" button that hands your search straight to the assistant. That is genuinely the whole setup for a self-hosted site.
Two things to know before you deploy. Anthropic does not offer an embeddings API, so a site running on Anthropic also needs an OpenAI key for the search index. And when you build for production, build with the key present. The site can index itself on the first question if it has to, but it refuses to do that for a large site, and readers would be the first to notice.
The chat button shows up when a provider is set, not when a key is present. Set the provider and forget the key and readers get a button that fails on every message. It is the one mistake here that your readers find before you do.
The one test worth running
Ask the assistant a question your docs don't answer.
The right response is some version of "that isn't covered here, but these pages might help." The prompt inside the generated site opens with "Answer ONLY from the provided context. Never fabricate information," and every answer arrives with the source pages listed underneath it, so you can check where it looked. But the prompt is only a rule, and the test is how you find out whether the rule held.
Then ask a question the docs do answer, and check that the source under the reply is the page you would have sent someone to.
I will be honest about the limit. Grounding a model in your pages cuts made-up answers down a lot. It does not get them to zero. A model handed six plausible paragraphs can still stitch them into a sentence none of them contains. The fix is boring and it works: read the answers to the questions readers actually ask, and fix the page when the answer is wrong.
Don't let strangers spend your money
On a public docs site the chat endpoint is public. That is correct, and it also means anyone with a terminal can send it questions all day on your key.
The generated site has a rate limit, ten requests a minute per visitor, and hard caps on how long a question can be. Treat those as a speed bump, not a lock. Then do two things:
- Put a spending limit on the key at your provider. Every provider lets you. A key with no ceiling behind a public endpoint is an invoice you haven't received yet.
- If the docs are private, gate the whole site. One
SITE_PASSWORDvariable puts a login in front of the pages, and the chat and search endpoints refuse anyone who hasn't logged in. That is the setup from how to password protect a documentation site, and the chat keeps working for the people who belong there.
The combination that goes wrong is always the same one. A public site, a key with no limit, and the belief that a rate limiter is a budget.
Picking a model
Chat needs a fast, cheap model. It writes a short paragraph from a few pages while someone waits, and that is a very different job from writing a documentation page, so don't reach for the biggest model you can find. The generated site defaults to a small model for each provider, and you can override it with LLM_CHAT_MODEL.
The harder question is whether to name a specific model at all. Pin one and it stays put, which also means it never gets better and eventually gets retired. Follow the newest one and it changes under you. The hosted platform handles this with a "Recommended" setting that follows the newest fast model from your provider, refreshed daily, while picking a specific model pins it until you change it. Either choice is fine. Just make it on purpose.
One gotcha worth knowing. The generated site sends a temperature of 0, the usual advice for grounded answers. As of 2026-09-07, Anthropic's API rejects anything but the default temperature on models newer than Claude Opus 4.6, so if you pin one of those and every message fails, set LLM_TEMPERATURE=1. The platform checks this for you and disables the setting for models that don't support it.
Who pays
If you self-host, the key is yours, the bill is yours, and the off switch is removing LLM_PROVIDER and redeploying. For a lot of teams that is the complete answer.
On the hosted platform, one setting per project decides what every AI feature spends, from the chat on your published site to the assistant in the editor. It has three positions, and only the project owner can move it:
- Included credit. Every plan comes with a monthly AI budget instead of per-message metering. As I write this on 2026-09-07 the pricing page lists $20 a month on Pro and $50 on Enterprise, and you can top it up if a month runs hot.
- Your own key. Point the project at OpenAI, Anthropic, or Google and the spend goes to your provider. Doccupine checks the key against the provider when you save it, so a bad key is caught in settings rather than in front of a reader.
- Off. Every AI feature refuses, and after the next deploy the chat button is gone from your site. Not disabled. Gone.
When to leave it off
Sometimes the hidden button is the right button:
- Your docs are thin or out of date. The assistant will repeat them confidently. Fix the pages first, and keep them current, or the chat becomes a faster way to be wrong.
- Search already answers everything. Search runs with no key and no model. If your site is twenty well-named pages, chat is a recurring bill for very little.
- Reader questions can't leave your infrastructure. Every question goes to the model provider. No setting changes that.
The chat that wasn't there when I started is the version I trust most, because nobody had answered those questions yet. Answer them, turn it on, and run the test. If the assistant admits what it doesn't know, you have something readers will use. If it doesn't, you have found the page to fix.
If your docs assistant ever answered with something no page says, email [email protected] with the question and the pages it cited. Those are the ones I want to see.