Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mischov/reaver
A Clojure library for extracting data from HTML.
https://github.com/mischov/reaver
Last synced: 7 days ago
JSON representation
A Clojure library for extracting data from HTML.
- Host: GitHub
- URL: https://github.com/mischov/reaver
- Owner: mischov
- License: epl-1.0
- Created: 2014-12-09T19:23:30.000Z (about 10 years ago)
- Default Branch: master
- Last Pushed: 2021-04-06T01:02:25.000Z (over 3 years ago)
- Last Synced: 2024-12-17T00:12:19.446Z (14 days ago)
- Language: Clojure
- Homepage:
- Size: 20.5 KB
- Stars: 147
- Watchers: 7
- Forks: 12
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Reaver
Reaver is a Clojure library wrapping Jsoup and designed for extracting data out of HTML and into EDN/Clojure.
Here is how one might scrape the headlines and links from Hacker News into a Clojure map (though Hacker News has an API and you should use that):
```Clojure
(require '[reaver :refer [parse extract-from text attr]])
; Reaver doesn't tell you how fetch your HTML. Use `slurp` or
; aleph or clj-http or what-have-you.
(def hacker-news (slurp "https://news.ycombinator.com/"))
; Extract the headlines and urls from the HTML into a seq of maps.
(extract-from (parse hacker-news) ".itemlist .athing"
[:headline :url]
".title > a" text
".title > a" (attr :href))
;> ({:headline "...", :url "..."}, {:headline "...", :url "..."}, ...)
```
## Contents
- [Installation](#installation)
- [Why?](#why)
- [Status](#status)
## Installation
Add the following dependency to your project.clj file:
```clojure
[reaver "0.1.3"]
```
[**Back To Top ⇧**](#contents)
## Why?
Clojure doesn't have a simple, purposed library for extracting data from HTML and into EDN.
The libraries most commonly used for this, Enlive and Laser, are primarily templating libraries, and extracting data from the data.xml format (like that created by Crouton) can be complicated and/or slow.
Reaver leverages Jsoup's well-fleshed-out selection and extraction mechanisms to provide an easy API for extracting data from XML.
[**Back To Top ⇧**](#contents)
## Status
Reaver is in **early beta**.
Since Reaver is primarily a wrapper around the battle-tested Jsoup library and otherwise a small, simple library, you should be able to use Reaver in production (with caution, of course).
There may still be lurking bugs or inconsistencies, so please report if you encounter one.
[**Back To Top ⇧**](#contents)
## License
Copyright © 2014 Mischov
Distributed under the Eclipse Public License.