https://github.com/oltarasenko/autosites

A small example which demonstrates how to use Crawly in order to extract data from JS based website.
https://github.com/oltarasenko/autosites

Last synced: about 1 month ago
JSON representation

A small example which demonstrates how to use Crawly in order to extract data from JS based website.

• Host: GitHub
• URL: https://github.com/oltarasenko/autosites
• Owner: oltarasenko
• Created: 2020-02-24T20:58:04.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2020-02-24T20:58:28.000Z (over 4 years ago)
• Last Synced: 2024-05-02T00:25:09.951Z (7 months ago)
• Language: HTML
• Size: 789 KB
• Stars: 9
• Watchers: 2
• Forks: 4
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Autosites

A small example which demonstrates how to use Crawly in order to extract

data from JS based website. 

Uses meeseeks, as a parses and Splash.

## Getting started

 1. Start splash Server locally: `docker run -it -p 8050:8050 scrapinghub/splash --max-timeout 300` 

 2. Run the iex: `mix deps.get && iex -S mix`

 3. Start the spider itself `Crawly.Engine.start_spider(AutotraderCoUK)`