https://github.com/kozmozio/kirby-llms
A Kirby CMS plugin that generates an llms.txt file for Large Language Models
https://github.com/kozmozio/kirby-llms
kirby4 kirby5 kirbycms llms llmstxt robots robotstxt
Last synced: 3 months ago
JSON representation
A Kirby CMS plugin that generates an llms.txt file for Large Language Models
- Host: GitHub
- URL: https://github.com/kozmozio/kirby-llms
- Owner: kozmozio
- License: mit
- Created: 2025-03-07T12:21:46.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2025-08-29T21:41:01.000Z (10 months ago)
- Last Synced: 2025-10-03T05:25:35.826Z (8 months ago)
- Topics: kirby4, kirby5, kirbycms, llms, llmstxt, robots, robotstxt
- Language: PHP
- Homepage:
- Size: 409 KB
- Stars: 7
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# Kozmoz Kirby LLMs Plugin
A Kirby CMS plugin that generates an `llms.txt` file in the root of your website or responds to the `llms.txt` route with necessary information for Large Language Models (LLMs)
## Overview
This plugin creates an `llms.txt` file that provides structured information about your website to help Large Language Models better understand and interact with your content. Similar to how `robots.txt` works for search engines, `llms.txt` provides guidance for LLMs.
## Features
- Generates an `llms.txt` file in the root of your website in Markdown format
- Generates an XML sitemap at `sitemap.xml` using the same page filtering as LLMs.txt
- Provides dedicated routes to access both LLMs information and sitemap
- Creates proper Markdown links for pages with trailing slashes (configurable)
- Strips HTML tags from descriptions and metadata for clean output
- Configurable exclusion of pages and templates
- Configurable inclusion of pages that override exclusions
- Groups pages by parent page name or template type with section headings (e.g. `## Blog`, `## Products`)
- Automatic cache clearing when content changes
- Supports configuration via your site's main config.php file
- Panel integration for easy configuration through the Kirby admin interface
## Installation
### Manual Installation
1. Download or clone this repository
2. Place the folder `kirby-llms` in your `site/plugins` directory
3. Rename the folder to `kirby-llms` if needed
### Composer Installation
```bash
composer require kozmozio/kirby-llms
```
## Configuration
### Via Config File
You can configure the plugin by adding options to your `config.php` file:
```php
return [
'kozmozio.llms' => [
'enabled' => true,
'cache' => true,
'cache.duration' => 60, // minutes
'add_trailing_slash' => true, // Whether to add trailing slashes to URLs
'group_by' => 'parent', // Group pages by: 'parent', 'template', or 'none'
'sitemap_enabled' => true, // Enable XML sitemap generation
'exclude' => [
'templates' => ['error', 'faq', 'faqs', 'faqpage', 'faq-page'],
'pages' => ['faqs', 'private-page', 'another-page']
],
'include' => [
'pages' => ['important-page', 'always-visible'] // Pages to always include despite exclusions
]
]
];
```
### Via Kirby Panel
The plugin also provides a panel interface for configuring the settings. To enable this, you need to add the LLMs tab to your site blueprint.
1. Add the LLMs tab to your `site.yml` blueprint:
```yaml
# site.yml
title: Site
unlisted: true
tabs:
# Your existing tabs...
# LLMs settings tab
llms: kozmoz/llms
```
This will add a new "LLMs" tab to your site settings in the panel, where you can configure:
- Enable/disable the LLMs.txt generation
- Enable/disable the XML sitemap generation
- Enable/disable caching
- Set cache duration
- Enable/disable adding trailing slashes to URLs
- Choose page grouping strategy (by parent, by template, or no grouping)
- Exclude templates and pages
- Include pages that should always be visible despite exclusions
### Blueprint Structure
The plugin uses the following blueprint structure:
```
blueprints/
├── kozmoz/
│ └── llms-settings.yml # The main settings section
├── tabs/
│ └── llms.yml # The tab that extends the settings section
```
You can customize these blueprints by copying them to your site's blueprint directory and modifying them as needed:
```
site/
└── blueprints/
├── sections/
│ └── llms-settings.yml # Your customized settings
└── tabs/
└── llms.yml # Your customized tab
```
### Configuration Options
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `enabled` | boolean | `true` | Enable or disable serving llms.txt |
| `sitemap_enabled` | boolean | `true` | Enable or disable XML sitemap generation |
| `cache` | boolean | `false` | Enable or disable caching |
| `cache.duration` | integer | `60` | Cache duration in minutes |
| `add_trailing_slash` | boolean | `true` | Whether to add trailing slashes to URLs in the output |
| `group_by` | string | `'parent'` | Group pages by: `'parent'` (parent page name), `'template'` (template type), or `'none'` (flat list) |
| `exclude.templates` | array | `['error']` | Templates to exclude from the output |
| `exclude.pages` | array | `[]` | Pages to exclude from the output |
| `include.pages` | array | `[]` | Pages to always include despite exclusions |
### Excluding Pages
You can exclude specific pages from the llms.txt output by adding their slugs to the `exclude.pages` array in your configuration. The plugin performs matching on:
- Exact page ID or URI match
- Pages with the same name in different locations (e.g., if you exclude 'inan-olcer', both 'inan-olcer' and 'team/inan-olcer' will be excluded)
- Child pages of excluded parents
- Pages with URIs containing specific strings (e.g., if you exclude 'faq', all pages with 'faq' in their URI will be excluded)
For example:
```php
'exclude' => [
'pages' => ['about', 'blog/private-post', 'team-member', 'faqs']
]
```
### Including Pages (Override Exclusions)
You can specify pages that should always be included in the output, even if they would normally be excluded by template or page exclusion rules. This is useful when you want to exclude a template globally but include specific pages that use that template.
The `include.pages` option overrides all exclusion rules and uses the same matching logic as exclusions:
- Exact page ID or URI match
- Child pages of included parents
- Pages with URIs containing the included string
For example:
```php
'include' => [
'pages' => ['faqs', 'important-faq', 'contact']
],
'exclude' => [
'templates' => ['faqs', 'contact'], // These templates are excluded
'pages' => ['private-page']
]
```
In this example:
- All `faqs` and `contact` template pages are excluded by default
- But pages under `faqs/` (like `faqs/question-one`) will be included because `faqs` is in the include list
- The `contact` page will be included despite using the excluded `contact` template
- Any page with `important-faq` in its URI will be included
**Include rules always take precedence over exclude rules.**
## Usage
Once installed, the plugin automatically sets up two routes:
- `yourdomain.com/llms.txt` - Generates the LLMs information in Markdown format
- `yourdomain.com/sitemap.xml` - Generates an XML sitemap using the same page filtering
The LLMs content will be in Markdown format with proper Markdown links for pages and their descriptions. The sitemap will be in standard XML format following the sitemap protocol. All HTML tags are automatically stripped from descriptions and metadata to ensure clean, plain text output.
### Automatic Cache Clearing
The plugin automatically clears its cache when:
- Pages are created, updated, or deleted
- Page properties change (status, slug, title, template)
- Site information is updated
This ensures that both the `llms.txt` content and `sitemap.xml` are always up-to-date with your website's content.
### Example Output
#### LLMs.txt Output
With default `group_by: parent`:
```markdown
# Your Website Title
> Your Website Title is your website description
Generated on: 2023-06-15 12:34:56
## Blog
- [Article One](https://example.com/blog/article-one/) - Our first post
- [Article Two](https://example.com/blog/article-two/) - Our second post
## Products
- [Widget A](https://example.com/products/widget-a/) - Our flagship widget
- [Widget B](https://example.com/products/widget-b/) - A compact alternative
## Pages
- [Home](https://example.com/) - Welcome to our website
- [About Us](https://example.com/about/) - Learn more about our company
- [Contact](https://example.com/contact/) - Get in touch with us
```
With `group_by: none` (flat list):
```markdown
# Your Website Title
> Your Website Title is your website description
Generated on: 2023-06-15 12:34:56
## Docs
- [Home](https://example.com/) - Welcome to our website
- [About Us](https://example.com/about/) - Learn more about our company and our mission
- [Products](https://example.com/products/) - Explore our range of products
- [Blog](https://example.com/blog/) - Read our latest articles
- [Contact](https://example.com/contact/) - Get in touch with us
```
#### Sitemap.xml Output
```xml
https://example.com/
2023-06-15T12:34:56+00:00
daily
1.0
https://example.com/about/
2023-06-10T08:20:15+00:00
weekly
0.8
https://example.com/products/
2023-06-12T14:45:30+00:00
weekly
0.8
```
## Static Site Generation
If you're using a static site generator with Kirby, make sure to include both routes in your static routes:
```php
array_push($staticRoutes, [
'path' => 'llms.txt',
'route' => 'llms.txt'
]);
array_push($staticRoutes, [
'path' => 'sitemap.xml',
'route' => 'sitemap.xml'
]);
```
## Development Status
### Phase 1: Basic Structure and Setup ✓
1. Create plugin folder structure ✓
2. Set up plugin initialization ✓
3. Register the route for `llms.txt` ✓
4. Implement basic configuration options ✓
### Phase 2: Core Functionality ✓
1. Develop the content generator for `llms.txt` ✓
2. Implement page collection and filtering ✓
3. Create the response formatter ✓
4. Add caching mechanism ✓
### Phase 3: Advanced Features ✓
1. Add customization options for content exclusion ✓
2. Implement metadata extraction ✓
3. Add automatic cache clearing when content changes ✓
4. Strip HTML tags from descriptions and metadata ✓
5. Ensure URLs have trailing slashes ✓
### Phase 4: Testing and Documentation ✓
1. Test with different Kirby setups ✓
2. Create comprehensive documentation ✓
3. Add examples and use cases ✓
4. Prepare for release ✓
## Author
[Inan Olcer Kozmoz](https://kozmoz.io)
## License
MIT