{"id":23440785,"url":"https://github.com/dabapps/django-simple-robots","last_synced_at":"2025-10-06T00:12:57.505Z","repository":{"id":25710182,"uuid":"106307738","full_name":"dabapps/django-simple-robots","owner":"dabapps","description":"Simple robots.txt file serving for web applications","archived":false,"fork":false,"pushed_at":"2024-03-20T16:42:37.000Z","size":27,"stargazers_count":2,"open_issues_count":2,"forks_count":2,"subscribers_count":19,"default_branch":"master","last_synced_at":"2025-04-13T09:12:55.800Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dabapps.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-10-09T16:28:04.000Z","updated_at":"2021-12-27T15:58:45.000Z","dependencies_parsed_at":"2025-04-13T09:12:57.200Z","dependency_job_id":"028423a9-4fe4-4ed1-a14d-3894bc13b8ae","html_url":"https://github.com/dabapps/django-simple-robots","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dabapps/django-simple-robots","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dabapps%2Fdjango-simple-robots","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dabapps%2Fdjango-simple-robots/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dabapps%2Fdjango-simple-robots/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dabapps%2Fdjango-simple-robots/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dabapps","download_url":"https://codeload.github.com/dabapps/django-simple-robots/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dabapps%2Fdjango-simple-robots/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278538332,"owners_count":26003362,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-05T02:00:06.059Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-23T16:18:39.491Z","updated_at":"2025-10-06T00:12:57.462Z","avatar_url":"https://github.com/dabapps.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"django-simple-robots\n====================\n\n[![Build Status](https://travis-ci.org/dabapps/django-simple-robots.svg)](https://travis-ci.org/dabapps/django-simple-robots)\n[![pypi release](https://img.shields.io/pypi/v/django-simple-robots.svg)](https://pypi.python.org/pypi/django-simple-robots)\n\nMost web applications shouldn't be indexed by Google. This app just provides a view that serves a \"deny all\" robots.txt.\n\nIn some cases, you do want your app to be indexed - but only in your production environment (not any staging environments). For this case, you can set `ROBOTS_ALLOW_HOST`. If the incoming hostname matches this setting, an \"allow all\" robots.txt will be served. Otherwise, the \"deny all\" will be served.\n\nTested against Django 3.2, 4.0, 4.1 and 4.2 on Python 3.8, 3.9, 3.10 and 3.11\n\n### Installation\n\nInstall from PIP\n\n    pip install django-simple-robots\n\nIn your root urlconf, add an entry as follows:\n\n    from django.conf.urls import url\n    from simple_robots.views import serve_robots\n\n    urlpatterns = [\n        path(\"robots.txt\", serve_robots),\n        # ..... other stuff\n    ]\n\nThen, add `simple_robots` to `INSTALLED_APPS` in your `settings.py`\n\nOptionally, set `ROBOTS_ALLOW_HOST` settings variable.\n\n    ROBOTS_ALLOW_HOST = \"myproductionurl.com\"\n\nThat's it!\n\n### Customization\n\nThe allow and disallow template are stored at `robots.txt` and `robots-disallow.txt` respectively. You can override these in your projects templates directory to customize the responses.\n\n### Code of conduct\n\nFor guidelines regarding the code of conduct when contributing to this repository please review [https://www.dabapps.com/open-source/code-of-conduct/](https://www.dabapps.com/open-source/code-of-conduct/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdabapps%2Fdjango-simple-robots","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdabapps%2Fdjango-simple-robots","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdabapps%2Fdjango-simple-robots/lists"}