https://github.com/kadukeitor/robots.txt
A robots.txt file is a file at the root of your site that indicates those parts of your site you don’t want accessed by search engine crawlers.
https://github.com/kadukeitor/robots.txt
crawlers docker robots
Last synced: 5 months ago
JSON representation
A robots.txt file is a file at the root of your site that indicates those parts of your site you don’t want accessed by search engine crawlers.
- Host: GitHub
- URL: https://github.com/kadukeitor/robots.txt
- Owner: kadukeitor
- Created: 2017-06-16T19:58:45.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2017-06-16T21:47:09.000Z (over 8 years ago)
- Last Synced: 2025-02-17T11:15:00.272Z (8 months ago)
- Topics: crawlers, docker, robots
- Language: HTML
- Homepage: https://hub.docker.com/r/kadukeitor/robots.txt
- Size: 1000 Bytes
- Stars: 0
- Watchers: 3
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# robots.txt
A ``robots.txt`` file is a file at the root of your site that indicates those parts of your site you don’t want accessed by search engine crawlers.
## How to use
docker run -p 8080:80 kadukeitor/robots.txt