Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/bnomei/kirby3-robots-txt
Manage the robots.txt from the Kirby config file
https://github.com/bnomei/kirby3-robots-txt
kirby3 kirby3-cms kirby3-plugin robots-exclusion-protocol robots-txt
Last synced: 16 days ago
JSON representation
Manage the robots.txt from the Kirby config file
- Host: GitHub
- URL: https://github.com/bnomei/kirby3-robots-txt
- Owner: bnomei
- License: mit
- Created: 2018-10-05T14:14:31.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2024-05-28T14:47:17.000Z (6 months ago)
- Last Synced: 2024-10-03T13:22:11.819Z (about 1 month ago)
- Topics: kirby3, kirby3-cms, kirby3-plugin, robots-exclusion-protocol, robots-txt
- Language: PHP
- Homepage: https://forum.getkirby.com/t/kirby3-robots-txt-manage-the-robots-txt-from-the-kirby-config-file/23587
- Size: 169 KB
- Stars: 20
- Watchers: 2
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# Kirby 3/4 Robots.txt
[](https://travis-ci.com/bnomei/kirby3-robots-txt)
[](https://coveralls.io/github/bnomei/kirby3-robots-txt)
[](https://codeclimate.com/github/bnomei/kirby3-robots-txt)
[](https://twitter.com/bnomei)
Manage the [robots.txt](https://developers.google.com/search/reference/robots_txt) from the Kirby config file.
## Similar Robots.txt Plugins
- [kirby3-helpers](https://github.com/johannschopplich/kirby-helpers)
- [kirby3-wellknown](https://github.com/omz13/kirby3-wellknown)
## Related SEO Plugins
- [kirby3-seo](https://github.com/tobimori/kirby-seo)
- [kirby3-helpers](https://github.com/johannschopplich/kirby-helpers)
- [kirby-meta-knight](https://github.com/diesdasdigital/kirby-meta-knight)
## Commercial Usage
>
> Support open source!
> This plugin is free but if you use it in a commercial project please consider to sponsor me or make a donation.
> If my work helped you to make some cash it seems fair to me that I might get a little reward as well, right?
> Be kind. Share a little. Thanks.
> ‐ Bruno
>
| M | O | N | E | Y |
|---|----|---|---|---|
| [Github sponsor](https://github.com/sponsors/bnomei) | [Patreon](https://patreon.com/bnomei) | [Buy Me a Coffee](https://buymeacoff.ee/bnomei) | [Paypal dontation](https://www.paypal.me/bnomei/15) | [Hire me](mailto:[email protected]?subject=Kirby) |
## Installation
- unzip [master.zip](https://github.com/bnomei/kirby3-robots-txt/archive/master.zip) as folder `site/plugins/kirby3-robots-txt` or
- `git submodule add https://github.com/bnomei/kirby3-robots-txt.git site/plugins/kirby3-robots-txt` or
- `composer require bnomei/kirby3-robots-txt`
## Staging Server? Debug Mode = Disallow all
When you set the global Kirby `debug` config to `true` the plugin will disallow all indexing for all user-agents. This is especially useful on a staging server but you could consider xml-sitemap and rss-feed among other things as well.
> ⚠️⚠️⚠️ THIS MEANS IF YOU HAVE KIRBY'S DEBUG MODE ENABLED IN PRODUCTION ALL SEARCH ENGINES WILL BE BLOCKED FROM INDEXING YOUR SITE!
## Adding Sitemap Link to Robots.txt
This plugin will add the sitemap link **automatically** to the `robots.txt` file for most available SEO plugins. You can skip setting the `bnomei.robots-txt.sitemap` config value to `sitemap.xml` in that case.
## Setup
The plugin generates automatic defaults for the starterkit. You do not have to enter them in the config file. But if you would it would look like this.
**defaults for starterkit**
```php
null, // string or callback
'bnomei.robots-txt.sitemap' => null, // null (aka. AUTOMATIC) or string or callback
'bnomei.robots-txt.groups' => [ // array or callback
'*' => [ // user-agent
'disallow' => [
'/kirby/',
'/site/',
'/cdn-cgi/',
],
'allow' => [
'/media/',
]
]
]
];
```
**using a plain string**
```php
'user-agent: *
disallow: /kirby/
disallow: /site/
disallow: /cdn-cgi/
allow: /media/',
];
```
**using a callback**
```php
function() {
return site()->myRobotsTxtContentField()->value();
},
];
```
**sitemap and multiple user-agents**
```php
'sitemap.xml',
'bnomei.robots-txt.groups' => [
'*' => [
'disallow' => [
'/',
],
],
'googlebot-images' => [
'allow' => [
'/media/',
]
]
]
];
```
## Disclaimer
This plugin is provided "as is" with no guarantee. Use it at your own risk and always test it yourself before using it in a production environment. If you find any issues, please [create a new issue](https://github.com/bnomei/kirby3-robots-txt/issues/new).
## License
[MIT](https://opensource.org/licenses/MIT)
It is discouraged to use this plugin in any project that promotes racism, sexism, homophobia, animal abuse, violence or any other form of hate speech.