{"id":17344018,"url":"https://github.com/tractorcow/silverstripe-robots","last_synced_at":"2025-08-02T01:36:54.527Z","repository":{"id":4428589,"uuid":"5566575","full_name":"tractorcow/silverstripe-robots","owner":"tractorcow","description":"Simple robots generation module for Silverstripe (SS 4 and above)","archived":false,"fork":false,"pushed_at":"2023-08-29T23:10:04.000Z","size":25,"stargazers_count":13,"open_issues_count":0,"forks_count":6,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-14T20:13:29.294Z","etag":null,"topics":["robots","robots-txt","silverstripe","silverstripe-4"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tractorcow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2012-08-27T04:18:17.000Z","updated_at":"2024-09-22T08:29:32.000Z","dependencies_parsed_at":"2024-06-18T19:51:35.645Z","dependency_job_id":"2d89991f-a92d-4a6e-9a65-174171b1037a","html_url":"https://github.com/tractorcow/silverstripe-robots","commit_stats":{"total_commits":22,"total_committers":7,"mean_commits":3.142857142857143,"dds":0.5454545454545454,"last_synced_commit":"a5ae868cea20780b3b8bc500dc9316e954779686"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tractorcow%2Fsilverstripe-robots","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tractorcow%2Fsilverstripe-robots/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tractorcow%2Fsilverstripe-robots/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tractorcow%2Fsilverstripe-robots/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tractorcow","download_url":"https://codeload.github.com/tractorcow/silverstripe-robots/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248952352,"owners_count":21188426,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["robots","robots-txt","silverstripe","silverstripe-4"],"created_at":"2024-10-15T16:10:54.747Z","updated_at":"2025-08-02T01:36:54.511Z","avatar_url":"https://github.com/tractorcow.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Robots.txt generation module for Silverstripe\n\nThis module provides simple robots.txt generation for Silverstripe, with various configuration\noptions available.\n\nWhen a site is not in live mode (such as on a testing domain) it will respectively\nblock the entire domain, ensuring that (at least respectful) search engines will\nrefrain from indexing your test site.\n\nThis doesn't support complex rules (such as multiple rules for different user agents),\nas it's designed to cater to the 99.9% of cases which won't require manual configuration.\n\n## Credits and Authors\n\n * Damian Mooyman - \u003chttps://github.com/tractorcow/silverstripe-robots\u003e\n\n## Requirements\n\n * Silverstripe 6\n\n## Installation Instructions\n\n * Extract all files into the 'robots' folder under your Silverstripe root, or install using composer\n\n```bash\ncomposer require \"tractorcow/silverstripe-robots\": \"^5\"\n```\n\n * Make sure you are correctly setting your environment for it to work properly\n * It's also advisable to either install the googlesitemaps module, or to create a physical `sitemap.xml` in your site root.\n\n## Configuration\n\nYou can add a page or pattern to be blocked by adding it to the disallowedUrls configuration\n\n```yaml\n---\nName: myrobotsconfiguration\n---\nTractorCow\\Robots\\Robots:\n  disallowed_urls:\n    - 'mysecretpage.html'\n    - '_private'\n    - 'Documents-and-Settings/Ricky/My-Documents/faxes/sent-faxes'\n```\n\nAlso by default, any page with 'ShowInSearch' set to false will also be excluded. This\ncan be useful for hiding auxilary pages like \"thanks for signing up\", or error pages.\n\nYou can turn this off (if you really absolutely think you need to) using the below\n\n```yaml\n---\nName: myrobotsconfiguration\n---\nTractorCow\\Robots\\Robots:\n  disallow_unsearchable: false\n```\n\nBy default the module will check for a sitemap file in `/sitemap.xml`, or will assume\none is there if the googlesitemap module is installed. You can set a custom file location\nusing the below configuration.\n\n```yaml\n---\nName: myrobotsconfiguration\n---\nTractorCow\\Robots\\Robots:\n  sitemap: '/sitemap_index.xml'\n```\n\n## Need more help?\n\nMessage or email me at damian.mooyman@gmail.com or, well, read the code!\n\n## License\n\nCopyright (c) 2013, Damian Mooyman\nAll rights reserved.\n\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are met:\n\n * Redistributions of source code must retain the above copyright\n   notice, this list of conditions and the following disclaimer.\n * Redistributions in binary form must reproduce the above copyright\n   notice, this list of conditions and the following disclaimer in the\n   documentation and/or other materials provided with the distribution.\n * The name of Damian Mooyman may not be used to endorse or promote products\n   derived from this software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND\nANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED\nWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL \u003cCOPYRIGHT HOLDER\u003e BE LIABLE FOR ANY\nDIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES\n(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;\nLOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND\nON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT\n(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\nSOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftractorcow%2Fsilverstripe-robots","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftractorcow%2Fsilverstripe-robots","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftractorcow%2Fsilverstripe-robots/lists"}