Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/yoozi/miner
Miner is a PHP library that extracting metadata and interesting text content (like author, summary, and etc.) from HTML pages. It acts like a simplified HTML metadata parser in Apache Tika.
https://github.com/yoozi/miner
composer extract meta parsing php readability webpage
Last synced: 26 days ago
JSON representation
Miner is a PHP library that extracting metadata and interesting text content (like author, summary, and etc.) from HTML pages. It acts like a simplified HTML metadata parser in Apache Tika.
- Host: GitHub
- URL: https://github.com/yoozi/miner
- Owner: yoozi
- License: mit
- Created: 2014-01-13T05:06:51.000Z (almost 11 years ago)
- Default Branch: master
- Last Pushed: 2014-11-06T10:46:53.000Z (almost 10 years ago)
- Last Synced: 2024-10-01T04:32:09.415Z (about 1 month ago)
- Topics: composer, extract, meta, parsing, php, readability, webpage
- Language: PHP
- Homepage: http://golem.yoozi.cn/
- Size: 264 KB
- Stars: 14
- Watchers: 13
- Forks: 6
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Miner
==========
[](https://gitter.im/yoozi/miner?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
> This library is part of [Project Golem](http://golem.yoozi.cn/), see [yoozi/golem](https://github.com/yoozi/golem) for more info.
Miner is a PHP library that extracting metadata and interesting text content (like author, summary, and etc.) from HTML pages.
It acts like a simplified [HTML metadata parser](https://tika.apache.org/1.4/formats.html#HyperText_Markup_Language) in [Apache Tika](https://tika.apache.org/).
## WTF is Miner?
Ta-da! Consider the screenshot taken from LinkedIn below:
When you post a link to your connections on LinkedIn, it will automatically extract the title, summary, and even cover image for you. Miner can be typically used to achieve tasks like this.
## Installation
The best and easy way to install the Golem package is with [Composer](https://getcomposer.org).
1. Open your composer.json and add the following to the require array:
```
"yoozi/miner": "1.0.*"
```
2. Run Composer to install or update the new package dependencies.
```
php composer install
```
or
```
php composer update
```
## Usage
### Parsers
* **Meta**: Summarize a webpage by parsing its HTML meta tags. In most cases it favors Open Graph (OG) markup, and will fall back to standard meta tags if necessary.
* **Readability**: Summarize a webpage using [Arc90's Readability alogrithm](https://code.google.com/p/arc90labs-readability/). All credit goes to [@feelinglucky's PHP Port](https://github.com/feelinglucky/php-readability).
* **Hybrid**: In combination with the above two parsers, it simply takes Readability as the primary parser, and Meta as its fallback.
Hybrid is enabled by default. You can change parsers to best fit your needs:
```php
// Use the Readability Parser.
$extractor->getConfig()->set('parser', 'readability');
// Or...use the Hybrid Parser.
// $extractor->getConfig()->set('parser', 'hybrid');
// Or...use the Meta Parser.
// $extractor->getConfig()->set('parser', 'meta');
```
### Example
We can parse a remote url and extract its metadata directly.
```php
getConfig()->set('parser', 'hybrid');
// Strip all HTML tags in the description we parsed.
$extractor->getConfig()->set('strip_tags', true);
$meta = $extractor->fromUrl('http://www.example.com/', new Curl)->run();
var_dump($meta);
```
Data returned:
```php
array(9) {
["title"]=>
string(14) "Example Domain"
["author"]=>
NULL
["keywords"]=>
array(0) {
}
["description"]=>
string(220) "
Example Domain
This domain is established to be used for illustrative examples in documents. You may use this
domain in examples without prior coordination or asking for permission.
More information...
"
["image"]=>
NULL
["url"]=>
string(23) "http://www.example.com/"
["host"]=>
string(22) "http://www.example.com"
["domain"]=>
string(11) "example.com"
["favicon"]=>
string(52) "http://www.google.com/s2/favicons?domain=example.com"
}
```