https://github.com/webcoast-dk/versatile-crawler

Extendable and easy to use crawler extension for TYPO3 CMS
https://github.com/webcoast-dk/versatile-crawler

crawler extendable indexing search typo3

Last synced: 17 days ago
JSON representation

Extendable and easy to use crawler extension for TYPO3 CMS

• Host: GitHub
• URL: https://github.com/webcoast-dk/versatile-crawler
• Owner: webcoast-dk
• License: gpl-3.0
• Created: 2017-10-19T14:09:48.000Z (about 7 years ago)
• Default Branch: main
• Last Pushed: 2024-06-27T09:36:12.000Z (6 months ago)
• Last Synced: 2024-11-13T12:53:47.233Z (about 2 months ago)
• Topics: crawler, extendable, indexing, search, typo3
• Language: PHP
• Homepage:
• Size: 124 KB
• Stars: 7
• Watchers: 3
• Forks: 5
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md

Awesome Lists containing this project

README

        
# EXT:versatile_crawler

Versatile Crawler is basically an extension to crawl pages and content in an TYPO3 CMS installation.

It is developed on the basis of TYPO3 CMS version 8. The extension has a clear and easily understandable

structure and provides queue and crawler functions for pages and records.

## Installation & Setup

Clone the extension from GitHub, manually or via composer, and activate the extension via the extension manager.

Create a crawler configuration record on the page you like to start the indexing on, e.g. on the homepage.

Go the scheduler module and create a queue task and a process task. Configure a cron job that triggers

TYPO3 CMS' scheduler.

### Prerequisites

* TYPO3 CMS 8

* PHP 7 w/ cURL

## Documentation

The documentation can be found in the GitHub wiki: https://github.com/webcoast-dk/versatile-crawler/wiki

## Contributing

Feel free to fork the repository, make changes and create a pull request. If you are not into coding

or do not have the time, open up an issue.

## Credits

The extension is developed and maintained by Thorben Nissen (https://www.kapp-hamburg.de/en/)

## License

Copyright (C) 2017 Thorben Nissen

This program is free software: you can redistribute it and/or modify

it under the terms of the GNU General Public License as published by

the Free Software Foundation, either version 3 of the License, or

(at your option) any later version.

This program is distributed in the hope that it will be useful,

but WITHOUT ANY WARRANTY; without even the implied warranty of

MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the

GNU General Public License for more details.

You should have received a copy of the GNU General Public License

along with this program.  If not, see .

### Icons

The icons are taken from the Material design (https://materialdesignicons.com/) and are licensed under

the SIL Open Font License 1.1.