How to automate a documentation pipeline that fails when it should

I put a page with an unclosed tag into a docs site and ran the whole pipeline. Generation exited 0. The production build exited 0. Every step was green, the page was prerendered as static content, and it deployed. The only trace of the problem was a single line buried in the build log, and the reader got an error panel where the documentation should have been.

That is not a bug. It is a deliberate design decision, and it is the thing most docs pipelines get wrong.

To automate a documentation pipeline, run four stages on every pull request: generate the site from source, install and static-check the generated app, run the production build, then gate on the build log for the content errors the build intentionally tolerates. Deploy from that same verified commit on merge. The fourth stage is the one almost nobody adds, and without it a green pipeline is not evidence that your documentation is intact.

Why a documentation build refuses to fail

A code build should fail loudly. A docs build usually should not, and the reasoning is sound: one author's typo in one page should not block a release or take down the other four hundred pages.

Doccupine's generated site implements exactly that. Each page body compiles inside a try/catch, and the comment in the source says why:

Compiles the MDX body inside a try/catch so an authoring mistake (an
orphaned closing tag, a stray {expression}) renders an inline error panel
instead of throwing during static prerender - a broken page must never
fail `next build` for the rest of the site.

So the page renders a red error panel, the build continues, and the pipeline reports success. Here is what that looked like when I ran it against a page containing <UnclosedThing> on CLI 0.0.147:

  Generating static pages using 13 workers (3/12)
[doccupine] MDX error in broken.mdx: [next-mdx-remote] error compiling MDX:
Expected a closing tag for `<UnclosedThing>` (4:44-4:59) before the end of `paragraph`

✓ Generating static pages using 13 workers (12/12) in 639ms

Route (app)
┌ ○ /
├ ○ /broken

Exit code 0. Route /broken shipped. This is the gap your pipeline has to close, because resilience at render time and correctness at merge time are different goals, and only one of them is the build's job.

The four stages of an automated docs pipeline

Treat these as separate stages even if they run in one job. Each one catches a different class of failure.

1. Generate. Turn your MDX source into the site. This is fast and has no network dependency.
2. Static-check. Install the generated app's dependencies and run its type checker and linter. This catches structural and component problems.
3. Build. Run the real production build. This catches anything that only fails during prerender.
4. Gate. Inspect the build output for tolerated errors and fail the job yourself.

Stage 4 exists because stages 1 through 3 are all designed to succeed.

Stage 1: generate, non-interactively

The build command generates the Next.js app once, without installing dependencies and without starting a server, which is exactly what you want in CI:

npx doccupine build

On the site I tested this finished in about 0.2 seconds and produced no node_modules. It also writes the machine-readable outputs on every run, so your sitemap.ts, llms.txt, and llms-full.txt regenerate from the current page set rather than drifting from it.

The config file is designed to be committed: paths are stored project-relative specifically so the same file works across machines, CI, and containers.

Stage 2 and 3: install, check, build

The generated app ships its own scripts. Run them in order, cheapest first:

cd nextjs-app
pnpm install
pnpm run type-check   # tsgo --noEmit
pnpm run lint         # eslint .
pnpm run build        # precompute embeddings, then next build

This is the same sequence the CLI runs against a freshly generated site in its own smoke test, which is a useful signal: it is the path the tool is actually tested on.

Two details worth setting deliberately rather than discovering later:

The embedding precompute fails soft. The build script runs an embedding step before next build. With no LLM API key it prints a warning, exits 0, and lets the build proceed; the chat then embeds documents on demand at runtime instead. That is the right default for a pipeline, but it means a missing key in CI is silent. If you want precomputed vectors in the deployed artifact, set the key in CI on purpose.

Set the site URL before the build, not after. Without a configured public URL, generation still succeeds but the sitemap and llms.txt are written with relative URLs. With config.json carrying "url": "https://docs.example.com", the same run emits absolute ones. You can override it per environment with NEXT_PUBLIC_SITE_URL, which is useful for staging, but the value is baked in at build time, so changing it requires a rebuild.

Stage 4: the gate that actually fails

The error message uses a stable, greppable prefix. Capture the build output and check it:

set -o pipefail
pnpm run build 2>&1 | tee build.log
grep -q '\[doccupine\] MDX error' build.log && exit 1

Put together, a complete workflow looks like this. Action versions are current as of 2026-08-02; check for newer majors before you copy this.

name: Docs

on:
  pull_request:
  push:
    branches: [main]

jobs:
  docs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: pnpm/action-setup@v6
      - uses: actions/setup-node@v7
        with:
          node-version: "22.12.0"

      - name: Generate the site
        run: npx doccupine build

      - name: Install generated app dependencies
        working-directory: nextjs-app
        run: pnpm install

      - name: Static checks
        working-directory: nextjs-app
        run: |
          pnpm run type-check
          pnpm run lint

      - name: Build
        working-directory: nextjs-app
        shell: bash
        env:
          NEXT_PUBLIC_SITE_URL: https://docs.example.com
        run: pnpm run build 2>&1 | tee build.log

      - name: Fail on tolerated MDX errors
        working-directory: nextjs-app
        shell: bash
        run: |
          if grep -q '\[doccupine\] MDX error' build.log; then
            echo "::error::A docs page failed to compile and would ship broken."
            grep '\[doccupine\] MDX error' build.log
            exit 1
          fi

shell: bash matters on the build step. GitHub's default Linux shell runs with -e but without pipefail, so piping the build into tee would otherwise hand you tee's exit code and mask a genuine build failure. Declaring shell: bash turns on pipefail, which is why the pipe is safe here.

Node 22.12.0 is the CLI's declared minimum, so pin at or above it.

What this gate does not catch

This is the honest limit, and it is worth knowing precisely rather than assuming the gate covers everything.

A page that references a component the renderer does not know about does not produce a logged error. I tested this too: a page using <TotallyMadeUpComponent /> generated cleanly, built cleanly, shipped as route /ghost, and logged nothing at all. The reader sees a "Missing component" panel; your pipeline sees a flawless run.

So the log gate catches MDX that fails to compile. It does not catch MDX that compiles into something wrong. If unknown components are a real risk for your team, the check has to happen on the rendered output, not the build log.

And no build-time check of any kind tells you whether a page is accurate. Compiling and being true are unrelated properties. That problem is documentation drift, and it needs source-linked detection rather than a build step.

What to commit, and why it decides your pipeline shape

Regeneration is deterministic. I generated the same site twice from scratch and diffed the results: byte-identical, apart from pnpm-lock.yaml and next-env.d.ts, which are produced by pnpm install and next build rather than by the generator.

That gives you a real choice:

• Commit only the source (docs/, doccupine.json, config.json) and regenerate in CI. Clean diffs, nothing redundant in review. The cost is that without a committed lockfile your installs are not pinned, so pin them yourself.
• Commit the generated app as well. You get a lockfile, reproducible installs, and a host that can build the directory directly with no generate step. The cost is a large initial diff.

Determinism is what makes the second option tolerable. Because the generator emits formatter-canonical output, a diff in the generated app means something changed, not that a formatter reshuffled it.

If you are still choosing a source layout, start from the Markdown structure before you automate anything around it. Automating a structure you are about to change is wasted work.

What not to automate

Some things belong in a pipeline. Others only look like they do.

• Do automate: generation, static checks, the production build, the error gate, deploys from a verified commit, and regenerating machine-readable files like sitemap.xml and llms.txt so they cannot fall behind your page set.
• Do not automate: merging documentation changes without a human reading them. A pipeline can prove a page compiles. It cannot tell you the page explains the right thing, and auto-merging on green converts a build signal into an editorial decision it was never qualified to make.

If your problem is that nobody updates the docs, a pipeline will not fix it. That is a workflow problem, and it needs the practices that keep documentation current rather than another CI job.

When you should not build this at all

If you self-host, this pipeline is yours to own, and the CLI is built for that: it generates a standard Next.js app you can build and deploy anywhere, with no account required.

But be honest about whether you want that job. If you have thirty pages and one writer, connecting the repository to a host with Git-based deploys is enough, and a four-stage workflow is overhead you will maintain for no return. And if you do not want to run docs infrastructure at all, the hosted Doccupine platform runs the builds and deployments for you, which means this entire article describes work you would not be doing. That is the actual trade: the CLI gives you the pipeline and the control, and the platform gives you the outcome without the pipeline. Neither one is a crippled version of the other.

The unclosed tag I started with is still the useful test. Run it through your own pipeline before you trust it: add a deliberately broken page, push it, and watch what happens.