Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/bnomei/kirby3-robots-txt
Manage the robots.txt from the Kirby config file
https://github.com/bnomei/kirby3-robots-txt
kirby-cms kirby-plugin robots-exclusion-protocol robots-txt
Last synced: 15 days ago
JSON representation
Manage the robots.txt from the Kirby config file
- Host: GitHub
- URL: https://github.com/bnomei/kirby3-robots-txt
- Owner: bnomei
- License: mit
- Created: 2018-10-05T14:14:31.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2024-11-26T20:09:33.000Z (about 1 month ago)
- Last Synced: 2024-12-09T15:40:51.508Z (24 days ago)
- Topics: kirby-cms, kirby-plugin, robots-exclusion-protocol, robots-txt
- Language: PHP
- Homepage: https://forum.getkirby.com/t/kirby3-robots-txt-manage-the-robots-txt-from-the-kirby-config-file/23587
- Size: 189 KB
- Stars: 20
- Watchers: 2
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# Kirby Robots.txt
[](https://getkirby.com)
[](https://codeclimate.com/github/bnomei/kirby3-robots-txt)
[](https://codeclimate.com/github/bnomei/kirby3-robots-txt/issues)
[](https://discordapp.com/users/bnomei)
[](https://www.buymeacoffee.com/bnomei)
Installing the plugin is enough to get a virtual [robots.txt](https://developers.google.com/search/reference/robots_txt) file.
## Installation
- unzip [master.zip](https://github.com/bnomei/kirby3-robots-txt/archive/master.zip) as folder `site/plugins/kirby3-robots-txt` or
- `git submodule add https://github.com/bnomei/kirby3-robots-txt.git site/plugins/kirby3-robots-txt` or
- `composer require bnomei/kirby3-robots-txt`
## Zero-Configuration
The plugin will work out of the box without the need to configure anything. It will generate a virtual `robots.txt` file with the following content.
```txt
User-agent: *
Disallow: /kirby/
Disallow: /site/
Disallow: /cdn-cgi/
Allow: /media/
```
## Active debug mode will block indexing by search engines
> [!WARNING]
> Setting the global Kirby `debug` configuration to `true` will prevent all indexing for every user agent (all Search Engines).
## Link to Sitemap.xml
This plugin will add the link to your `sitemap.xml` **automatically** for most available SEO plugins. In that case, you can skip setting the `bnomei.robots-txt.sitemap` config value to `sitemap.xml`.
## Optional Manual Configuration
The plugin generates automatic defaults optimized for the official [Kirby Starterkit](https://github.com/getkirby/starterkit). You do not have to enter any of them in the config file. But if you would, they would look like this.
**defaults for Kirby Starterkit**
```php
null, // string or callback
'bnomei.robots-txt.sitemap' => null, // null (aka. AUTOMATIC) or string or callback
'bnomei.robots-txt.groups' => [ // array or callback
'*' => [ // user-agent
'disallow' => [
'/kirby/',
'/site/',
'/cdn-cgi/',
],
'allow' => [
'/media/',
]
]
]
];
```
**using a plain string**
```php
'user-agent: *
disallow: /kirby/
disallow: /site/
disallow: /cdn-cgi/
allow: /media/',
];
```
**using a callback**
```php
function() {
return site()->myRobotsTxtContentField()->value();
},
];
```
**sitemap and multiple user-agents**
```php
'sitemap.xml',
'bnomei.robots-txt.groups' => [
'*' => [
'disallow' => [
'/',
],
],
'googlebot-images' => [
'allow' => [
'/media/',
]
]
]
];
```
## Disclaimer
This plugin is provided "as is" with no guarantee. Use it at your own risk and always test it yourself before using it in a production environment. If you find any issues, please [create a new issue](https://github.com/bnomei/kirby3-robots-txt/issues/new).
## License
[MIT](https://opensource.org/licenses/MIT)
It is discouraged to use this plugin in any project that promotes racism, sexism, homophobia, animal abuse, violence or any other form of hate speech.