https://github.com/antonioribeiro/twill-robots-txt
A Twill Capsule to handle robots.txt
https://github.com/antonioribeiro/twill-robots-txt
Last synced: 7 months ago
JSON representation
A Twill Capsule to handle robots.txt
- Host: GitHub
- URL: https://github.com/antonioribeiro/twill-robots-txt
- Owner: antonioribeiro
- License: apache-2.0
- Created: 2022-11-29T15:22:25.000Z (almost 3 years ago)
- Default Branch: 1.x
- Last Pushed: 2023-03-15T16:37:10.000Z (over 2 years ago)
- Last Synced: 2025-01-06T01:43:48.087Z (9 months ago)
- Language: PHP
- Size: 2.33 MB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: .github/CONTRIBUTING.md
- License: LICENSE
- Security: .github/SECURITY.md
Awesome Lists containing this project
README
# robots.txt Twill Capsule
This Twill Capsule is intended to enable developers to configure Robots TXT on their applications.
## How to use it on the CMS
Text areas are supposed to contain a full `robots.txt` file content and we have these two states:
- **Protected**: when the record is published/enabled, your website is hidden (protected) from all robots. Usually this will be a deny all rule.
- **Unprotected**: when the record is unpublished/disabled, your website is open for robots, add here all the [allowed/denied paths and exceptions](https://developers.google.com/search/docs/crawling-indexing/robots/robots_txt#syntax).
## Domains
You add as many domains as you need and configure different `robots.txt` values for each. If you enable `.env` confifuration, an `all domains (*)` entry will appear, the same configuration will be used for all domains, and all other domains will be hidden.
## Installing
### Require the Composer package:
``` bash
composer require area17/twill-robots-txt
```
### Publish the configuration
``` bash
php artisan vendor:publish --provider="A17\TwillRobotsTxt\ServiceProvider"
```
### Load Capsule helpers by adding calling the loader to your AppServiceProvider:
``` php
/**
* Register any application services.
*
* @return void
*/
public function register()
{
\A17\TwillRobotsTxt\Services\Helpers::load();
}
```
#### .env
The configuration works both on `.env` or in the CMS settings. If you set them on `.env` the CMS settings will be disabled and overloded by `.env`.
```dotenv
TWILL_ROBOTS_TXT_PROTECTED=true
TWILL_ROBOTS_TXT_RATE_LIMITING_ATTEMPTS=10
TWILL_ROBOTS_TXT_CONTENTS_PROTECTED="User-agent: *\nDisallow: /"
TWILL_ROBOTS_TXT_CONTENTS_UNPROTECTED="User-agent: *\nAllow: /"
```
## Contribute
Please contribute to this project by submitting pull requests.