feat: add root llms.txt#520
Merged
Merged
Conversation
Serve a product-level llms.txt at docs.page/llms.txt via an app-router route handler, mirroring the existing schema.json handler. Points agents at the docs and the per-repository agent surfaces (llms.txt, llms-full.txt, MCP, search.json, raw .md/.mdx). A route handler is used rather than a public/ file because the pages catch-all returns 404 for single-segment paths; the app-router handler reliably takes precedence.
|
|
|
Bugbot is not enabled for this team, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Elliot Hesp · Slack thread
Summary
Adds a product-level
llms.txtserved at the root domaindocs.page/llms.txt.Before:
docs.page/llms.txtreturned a 404. The root domain had no machine-readable index — agents landing ondocs.pagehad no llms.txt entry point, even though every hosted docs site already exposes one atdocs.page/{owner}/{repo}/llms.txt.After:
docs.page/llms.txtserves a concise, spec-compliant llms.txt that summarises the product and points agents at the documentation (use.docs.page, quickstart, CLI, GitHub) and at the per-repository agent surfaces (per-repollms.txt/llms-full.txt, the MCP server,search.json, and raw.md/.mdx).How
The file lives in an app-router route handler at
app/src/app/llms.txt/route.ts, mirroring the existingapp/src/app/schema.json/route.tshandler. It returns the text withContent-Type: text/plain; charset=utf-8.Routing note: a
public/llms.txtwas considered (simplest) but a route handler is more reliable here. The pages-router catch-all (app/src/pages/[[...path]].tsx) returns a 404 for any single-segment path, so serving a bare/llms.txtdepends on static-vs-dynamic precedence. An app-router route handler is guaranteed to take precedence over the pages catch-all (asschema.jsonalready demonstrates) and matches the repo's established convention for root-level generated files. The Next 16 middleware (app/src/proxy.ts) does not intercept/llms.txton the canonical domain — it falls through untouched — and the/:owner/:repo/llms.txtrewrite inapp/next.config.tsonly matches the three-segment per-repo pattern.Scope
app/(hosted site, MCP, Ask AI)packages/cli/packages/mdx-bundler/docs/(product documentation)Type of change
Test plan
bun devinapp/, thencurl -sS localhost:3000/llms.txtreturns HTTP 200 withContent-Type: text/plain; charset=utf-8and the expected body.bunx biome checkclean on the new file.bun run check(full) not run.Notes for reviewers
All links in the file were verified against the repo (
use.docs.page,use.docs.page/quickstart,use.docs.page/reference/cli, the GitHub repo, and the MCP tool namesread_doc_page/list_doc_files). The blockquote summary and endpoint list can be tweaked to taste.Addresses point 1 of #519.
🤖 Generated with Claude Code
https://claude.ai/code/session_01DHjPAWXa8wkqtsVzJVyBne
Generated by Claude Code