https://github.com/jimmylaurent/lycos
✨All the goodies you'll ever need to scrape the web (NodeJs / Browser)
https://github.com/jimmylaurent/lycos
api scraper scraping web
Last synced: about 1 year ago
JSON representation
✨All the goodies you'll ever need to scrape the web (NodeJs / Browser)
- Host: GitHub
- URL: https://github.com/jimmylaurent/lycos
- Owner: JimmyLaurent
- License: mit
- Created: 2019-02-02T11:09:04.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2022-12-10T01:02:12.000Z (over 3 years ago)
- Last Synced: 2025-03-16T09:37:35.344Z (over 1 year ago)
- Topics: api, scraper, scraping, web
- Language: JavaScript
- Homepage: https://jimmylaurent.github.io/lycos
- Size: 1.12 MB
- Stars: 5
- Watchers: 1
- Forks: 1
- Open Issues: 22
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
🐾 lycos.js
All the goodies you'll ever need to scrape the web
## Documentation
* [Get Started](https://jimmylaurent.github.io/lycos/#/README)
* [Examples](https://jimmylaurent.github.io/lycos/#/examples)
* [API Reference](https://jimmylaurent.github.io/lycos/#/api-reference)
## In-browser Playground
You can try the library on codesandbox, it uses a cors proxy fetcher to let you grab contents from any website inside your browser.
* CodeSandbox: https://codesandbox.io/s/njm2p72m
## Installation
```sh
yarn add lycos
# or
npm i lycos
```
## Features
- ⚡️️ All in one package to fetch and scrape data from the web
- ⭐ Node & Browser Support
- 💡 Powerful declarative API
- 🚀 Blazingly fast (supports concurrency)
- 🔧 Extensible
## Quick Example
```js
const lycos = require('lycos');
(async () => {
// Fetch the given url and return a page scraper
const page = await lycos.get('http://quotes.toscrape.com');
// Scrape all the quotes elements
const quoteElements = page.scrapeAll('.quote');
// For each quote element, scrape the text and the author
const quotes = quoteElements.map(element => ({
text: element.scrape('.text').text(),
author: element.scrape('.author').text()
}));
// Shortcut to scrape the collection of quotes
const quotes = page.scrapeAll('.quote', {
author: '.author@text',
text: '.text@text'
});
// Shortcut to fetch and scrape
const quotes = await lycos
.get('http://quotes.toscrape.com')
.scrapeAll('.quote', {
author: '.author@text',
text: '.text@text'
});
})();
```
## Credits
__• FB55:__ his work reprensents the core of this library.
__• Matt Mueller and cheerio contributors :__
A good portion of the code and concepts are copied/derived from the cheerio and x-ray libraries.
## License
MIT © 2019 [Jimmy Laurent](https://github.com/JimmyLaurent)