Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/malina/metascraper
Metascraper is a Crystal library for web scraping.
https://github.com/malina/metascraper
crystal scrape scraped-data
Last synced: 18 days ago
JSON representation
Metascraper is a Crystal library for web scraping.
- Host: GitHub
- URL: https://github.com/malina/metascraper
- Owner: malina
- License: mit
- Created: 2016-08-08T09:48:03.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2018-03-16T11:55:43.000Z (over 6 years ago)
- Last Synced: 2024-08-01T00:41:26.222Z (3 months ago)
- Topics: crystal, scrape, scraped-data
- Language: Crystal
- Size: 20.5 KB
- Stars: 11
- Watchers: 4
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# metascraper
Metascraper is a little lib for web scraping purposes.
You give it an URL, and it lets you easily get its title, images, description, videos.
## Installation
Add this to your application's `shard.yml`:
```yaml
dependencies:
metascraper:
github: malina/metascraper
```
## Usage
```crystal
require "metascraper"
```
Initialize a Metascraper instance for an URL, like this:
```crystal
page = Metascraper.new("https://github.com/malina/metascraper")
puts page.title
```
## Accessing scraped data
```crystal
page.url # URL of the page
page.images # enumerable collection, with every img found on the page
page.title # title of the page from the head section, as string
page.description # returns the meta description, or the first long paragraph if no meta description is found
page.content # primary readability page content
```
You can also access most of the scraped data as a hash:
```crystal
page.to_hash
```
## Contributing
1. Fork it ( https://github.com/malina/metascraper/fork )
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request
## Contributors
- [malina](https://github.com/malina) Alexandr Shumov - creator, maintainer