https://github.com/google/robotstxt

The repository contains Google's robots.txt parser and matcher as a C++ library (compliant to C++11).
https://github.com/google/robotstxt

Last synced: about 2 months ago
JSON representation

The repository contains Google's robots.txt parser and matcher as a C++ library (compliant to C++11).

• Host: GitHub
• URL: https://github.com/google/robotstxt
• Owner: google
• License: apache-2.0
• Created: 2019-06-11T17:30:19.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2024-05-22T12:12:00.000Z (4 months ago)
• Last Synced: 2024-05-22T13:33:40.598Z (4 months ago)
• Language: C++
• Homepage:
• Size: 117 KB
• Stars: 3,350
• Watchers: 92
• Forks: 223
• Open Issues: 13
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

• my-awesome - google/robotstxt - 08 star:3.4k fork:0.2k The repository contains Google's robots.txt parser and matcher as a C++ library (compliant to C++11). (C++)
• awesome-list - robotstxt