Skip to content

chore: update llms.txt #460

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
moshloop merged 1 commit into from
Jan 9, 2026
Merged

chore: update llms.txt #460

moshloop merged 1 commit into from
Jan 9, 2026

Conversation

@moshloop
Copy link
Member

@moshloop moshloop commented Jan 9, 2026 •
edited by coderabbitai bot
Loading

Summary by CodeRabbit

  • Chores
    • Removed LLMs plugin and associated auto-generated content blocks (guides, references, integrations, installation docs, etc.)
    • Updated build process tooling and dependencies to streamline documentation generation

✏️ Tip: You can customize this high-level summary in your review settings.

@vercel
Copy link

vercel bot commented Jan 9, 2026 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
docs Ready Ready Preview Jan 9, 2026 11:36am

@netlify
Copy link

netlify bot commented Jan 9, 2026 •
edited
Loading

✅ Deploy Preview for canarychecker canceled.

Name Link
🔨 Latest commit f8ad6b8
🔍 Latest deploy log https://app.netlify.com/projects/canarychecker/deploys/6960e78f6dd9e7000839a14e

@coderabbitai
Copy link

coderabbitai bot commented Jan 9, 2026 •
edited
Loading

Caution

Review failed

The pull request is closed.

Walkthrough

The pull request removes the docusaurus-plugin-llms dependency and its configuration from the Docusaurus setup, replacing it with a custom Node.js script. The new move-llms.mjs script generates consolidated llms.txt files from documentation sources during the build process, handling MDX/MD file processing, section organization, and sitemap-based root documentation generation.

Changes

Cohort / File(s) Summary
Configuration and Build System
mission-control/docusaurus.config.ts, mission-control/package.json		 Removed entire docusaurus-plugin-llms configuration block (133 lines), plugin dependency, and updated build scripts to use new custom script. Added remark-frontmatter dependency for MDX processing.
Script Migration
mission-control/scripts/move-llms.js		 Completely removed (269 lines). This file implemented the previous llms.txt file relocation and transformation workflow via the Docusaurus plugin.
Custom LLMS Generation Script
mission-control/scripts/move-llms.mjs		 New Node.js script (406 lines) that replaces plugin functionality. Processes MDX/MD documentation files, generates per-section llms.txt files with metadata and cleaned content, builds a root llms.txt with navigation index from sitemap.xml, and handles errors per-file with graceful continuation.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch pr/llms
📜 Recent review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c45d7be and f8ad6b8.

📒 Files selected for processing (4)
  • mission-control/docusaurus.config.ts
  • mission-control/package.json
  • mission-control/scripts/move-llms.js
  • mission-control/scripts/move-llms.mjs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

github-advanced-security[bot]
github-advanced-security bot found potential problems Jan 9, 2026
View reviewed changes
mission-control/scripts/move-llms.mjs
}

function stripHtmlComments(content) {
return content.replace(/<!--[\s\S]*?-->/g, '');

Check failure

Code scanning / CodeQL

Incomplete multi-character sanitization High

This string may still contain
<!--
Loading
, which may cause an HTML element injection vulnerability.

Copilot Autofix

AI 2 days ago

In general, to fix incomplete multi-character sanitization, either (a) apply the replacement repeatedly until no further matches occur, or (b) switch to a well-tested sanitization library that correctly handles HTML comments and edge cases. Here, the simplest, behavior-preserving approach is to keep the existing regex but iterate until no more replacements happen. This ensures that any new <!--...--> sequences that might be formed after a replacement are also removed.

Concretely, in mission-control/scripts/move-llms.mjs, we should modify stripHtmlComments (lines 174–176) so it no longer performs a single replace call. Instead, we introduce a small loop: keep a previous string, repeatedly call replace(/<!--[\s\S]*?-->/g, '') until the result is identical to the previous value, then return the final string. This preserves all existing calling code and behavior for normal inputs while closing the edge case highlighted by CodeQL. No additional imports or external libraries are required.

Suggested changeset 1
mission-control/scripts/move-llms.mjs

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/mission-control/scripts/move-llms.mjs b/mission-control/scripts/move-llms.mjs
--- a/mission-control/scripts/move-llms.mjs
+++ b/mission-control/scripts/move-llms.mjs
@@ -172,7 +172,13 @@
 }
 
 function stripHtmlComments(content) {
-  return content.replace(/<!--[\s\S]*?-->/g, '');
+  let previous;
+  let current = content;
+  do {
+    previous = current;
+    current = current.replace(/<!--[\s\S]*?-->/g, '');
+  } while (current !== previous);
+  return current;
 }
 
 function cleanupOutput(content) {
EOF
@@ -172,7 +172,13 @@
}

function stripHtmlComments(content) {
return content.replace(/<!--[\s\S]*?-->/g, '');
let previous;
let current = content;
do {
previous = current;
current = current.replace(/<!--[\s\S]*?-->/g, '');
} while (current !== previous);
return current;
}

function cleanupOutput(content) {
Copilot is powered by AI and may make mistakes. Always verify output.
@moshloop moshloop merged commit c61ba4c into main Jan 9, 2026
12 of 15 checks passed
@moshloop moshloop deleted the pr/llms branch January 9, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@moshloop