Every documentation site starts the same way: a folder of Markdown files. A README that outgrew itself, a docs/ directory someone added to the repo, a wiki export nobody wants to maintain anymore. The question is always the same too — how do I turn these files into an actual documentation site, with navigation and search and a real URL, without signing up for a six-week tooling project?
I've built this pipeline more times than I'd like to admit, and the honest answer is that the writing was never the hard part. The hard part is everything wrapped around the writing: navigation that builds itself, search that works, a theme that doesn't look like a default template, hosting, and — new as of the last couple of years — making the whole thing readable by the AI tools your users actually develop with. This guide walks through each of those decisions, and ends with the shortest path I know from a folder of Markdown to a live site.
Why Markdown is still the right source format
Before the how, a quick word on the what, because the source format decides everything downstream.
Markdown (and MDX, its component-capable extension) has won documentation for three reasons that have nothing to do with fashion:
- It's diffable. Plain text files live in Git, get reviewed in pull requests, and travel through the same workflow as code. That's the entire foundation of docs-as-code, and it's why docs written this way stay current while wiki pages rot.
- It's portable. A folder of
.mdfiles moves between any generator, any host, any platform. If a tool stores your content in a proprietary block format instead, you don't own your docs — you rent access to them. - It's what machines read best. LLMs parse Markdown natively. Headings become structure, code fences stay intact, and the whole page survives the trip into an AI assistant's context window. The era where AI agents read your docs more often than humans do is already here, and Markdown is the format they're trained on.
If your content is currently trapped somewhere else — a wiki, a word processor, a CMS — exporting to Markdown is step zero, and it's worth doing even if you stop reading here.
What a documentation site actually needs in 2026
The checklist used to be short: render the pages, generate a sidebar, add search. The bar has moved. Here's the full list I'd hold any approach against today:
- Automatic navigation. The sidebar should build itself from the file structure and frontmatter. Hand-maintained navigation config is the first thing to drift out of sync with the actual content.
- Full-text search. Non-negotiable, and it should work out of the box rather than requiring a third-party search service and an indexing pipeline.
- Dark mode. Developers live in dark editors and dark terminals. A docs site that only ships a light theme — or that blinds dark-mode readers with hardcoded light screenshots — reads as unmaintained.
- A real preview workflow. Save a file, see the change. If the feedback loop involves a manual build step, edits stop happening.
- AI readability. Per-page Markdown mirrors, an
llms.txtindex, clean heading hierarchy. AI assistants are a primary audience now, not an edge case. - An MCP server. This is the newest line on the list and the one most generators miss entirely. Exposing your docs over MCP lets Claude Code, Cursor, and Windsurf query your live documentation instead of hallucinating your API from training data.
- Standard, deployable output. The build should produce something you can host anywhere — your own server, a static host, a managed platform — without the generator's permission.
Hold onto that list. It's the difference between a docs site that's an asset and one that's a liability you revisit in a year.
Step 1: Structure your Markdown files
However you generate the site, the content layer looks the same. Each page is one file, and each file carries frontmatter that describes where it lives:
---
title: "Authentication"
description: "How to authenticate requests against the API"
category: "Getting Started"
order: 2
---A few structural rules that pay off regardless of tooling:
- One topic per file. Pages that try to cover everything rank for nothing and retrieve badly in both search indexes and RAG pipelines.
- Frontmatter on every page. Title and description at minimum — they become your HTML titles and meta descriptions, which is where your organic search traffic starts.
- Categories over deep nesting. A flat directory with category frontmatter beats a five-level folder hierarchy. Readers scan sidebars; they don't spelunk them.
- Real headings, in order.
h1thenh2thenh3, no skips. Heading hierarchy is how both search engines and LLMs reconstruct your page's structure.
Step 2: Generate the site
This is where most guides hand you a week of configuration. Here's the version I actually use. With Node.js v22+ installed, run this in the directory where your docs live (or where you want them to live):
npx doccupineThat's the whole step. The CLI asks two questions — where your MDX files are (default: docs/) and where to put the generated app (default: nextjs-app/) — then scaffolds a complete Next.js documentation site, installs its dependencies, and starts a dev server at localhost:3000. If the docs directory is empty, it seeds starter pages so you're never staring at a blank screen.
From that point it watches your files. Save a Markdown file, the page regenerates. Add frontmatter categories, the sidebar reorders itself. Drop images into public/, they sync into the app. The feedback loop is the one you already know from modern web development: edit, save, see it.
Out of the box, the generated site covers the entire checklist from above — auto-built navigation, search, dark/light themes via a theme.json, tabbed sections for splitting docs from API reference, per-page Markdown mirrors, an llms.txt index, a built-in RAG-powered AI chat assistant, and an MCP server exposing search_docs, get_doc, and list_docs to any AI coding tool your users point at it.
It's open source and free, and the output is a standard Next.js app. Which matters for the next step.
Step 3: Deploy it
When you're ready to ship, build once instead of watching:
npx doccupine buildWhat comes out is a normal Next.js application — not a proprietary bundle, not a hosted-only artifact. That means every deployment path that works for Next.js works for your docs:
- Self-host it. Run it on your own infrastructure, behind your own proxy, inside your own VPC. Some teams need docs behind a VPN; this is how.
- Push it to any Node-capable host or static platform. The same app deploys anywhere Next.js deploys.
- Let a platform run it for you. This is what we built the Doccupine platform for — it takes the same generated site and handles the GitHub repo, the deploys, the custom domain and SSL, and the pending-changes preview workflow, so shipping a docs update is a pull request and nothing else.
The point of the architecture is that this is a choice, not a commitment. Free self-hosters and paying platform customers run the same engine, and moving between the two is a deploy decision, not a migration project.
Step 4: Make it findable — by people and by agents
A live site is the halfway point. The traffic comes from two audiences now, and they find you differently.
For search engines, the fundamentals are unchanged: unique titles and meta descriptions from your frontmatter, one h1 per page, descriptive URLs, and content that actually answers the query. Docs pages are some of the best-ranking content a product company has, because they target the exact phrases developers search.
For AI assistants, the surface area is newer: the llms.txt index tells crawlers what exists, per-page .md mirrors give them clean content without HTML soup, and the MCP server turns your docs into a live tool that coding agents query mid-task. This audience is largely invisible in standard analytics — tracking the bot traffic on your docs is its own discipline — but it's the audience that increasingly decides whether the integration code your users ship is right or wrong. Your documentation has quietly become the primary interface between your product and the developers building on it; building the site is step one of treating it that way.
The short version
A folder of Markdown files, one npx doccupine command, and a deploy target you choose — that's the entire distance between "we should really have a docs site" and a live, searchable, AI-ready documentation site on your own domain. The structural work — clear pages, honest frontmatter, headings in order — is yours. The scaffolding shouldn't be.
If you're migrating docs out of somewhere painful and hit a snag turning them into Markdown, write to [email protected] — I've seen most of the export formats by now, and I'm happy to point you at the least bad path out.