Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/sergioburdisso/solidscraper
Easy to use JQuery-Like API for Web Scraping/Crawling.
https://github.com/sergioburdisso/solidscraper
crawler crawling crawling-python jquery python scraper scraping tweets twitter web web-crawler web-scraping webscraping
Last synced: about 1 month ago
JSON representation
Easy to use JQuery-Like API for Web Scraping/Crawling.
- Host: GitHub
- URL: https://github.com/sergioburdisso/solidscraper
- Owner: sergioburdisso
- License: mit
- Created: 2017-10-10T12:11:24.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2020-07-02T12:45:03.000Z (over 4 years ago)
- Last Synced: 2024-11-11T14:19:26.109Z (about 2 months ago)
- Topics: crawler, crawling, crawling-python, jquery, python, scraper, scraping, tweets, twitter, web, web-crawler, web-scraping, webscraping
- Language: Python
- Homepage:
- Size: 31.3 KB
- Stars: 3
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# Solid Scraper
Easy to use JQuery-Like API for Web Scraping/Crawling. It also supports Cookies and custom User Agents. Solidscraper is compatible with **Python 2 and 3**.
---
## 1. Installation````
pip install solidscraper
````**Note:** if you already have installed it, and wanted the latest version, then use the following command to update `solidscraper`:
````
pip install --upgrade solidscraper
````---
## 2. "Hello World" ExamplesGetting all url of all links:
````python
import solidscraper as ssdoc = ss.load("https://www.example.com/the/path")
# print the list of urls from all elements
print(doc.select("a").getAttribute("href"))
````Getting all url of all links inside \
s whose class id is 'links':````python
import solidscraper as ssdoc = ss.load("https://www.example.com/the/path")