https://github.com/thomasleveil/pyrobots

python binding for Google robots.txt parser C++ library
https://github.com/thomasleveil/pyrobots

googlebot python robots-exclusion-protocol robots-parser robots-txt robotstxt

Last synced: about 1 month ago
JSON representation

python binding for Google robots.txt parser C++ library

• Host: GitHub
• URL: https://github.com/thomasleveil/pyrobots
• Owner: thomasleveil
• Created: 2019-07-22T16:42:31.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2019-07-30T08:55:49.000Z (over 5 years ago)
• Last Synced: 2023-03-10T20:33:45.475Z (almost 2 years ago)
• Topics: googlebot, python, robots-exclusion-protocol, robots-parser, robots-txt, robotstxt
• Language: Python
• Homepage:
• Size: 11.7 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0

Awesome Lists containing this project