Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/kennethreitz/requests-html

Pythonic HTML Parsing for Humansâ„¢

html requests scraping scraping-framework scraping-websites

Last synced: 30 Jun 2024

https://github.com/baptisteArno/tinking

🧶 Extract data from any website without code, just clicks.

harvesting puppeteer scraping scraping-websites scrapper scrapping

Last synced: 27 Jun 2024

https://github.com/alash3al/scraply

Scraply a simple dom scraper to fetch information from any html based website

crawler crawling dom golang scraper scrapers scraping-websites scrapy server

Last synced: 21 Jun 2024

https://github.com/pavlovtech/WebReaper

Web scraper, crawler and parser in C#. Designed as simple, declarative and scalable web scraping solution.

crawler datamining parser parsing scraper scraping scraping-api scraping-data scraping-tool scraping-web scraping-websites webcrawler webscraping

Last synced: 15 Jun 2024

https://github.com/unixfox/pupflare

A webpage proxy that request through Chromium (puppeteer) - can be used to bypass Cloudflare anti bot / anti ddos on any application (like curl)

anti-bot-page chromium cloudflare cloudflare-bypass cloudflare-scrape docker koa protected-page proxy puppeteer scrape scraping-websites

Last synced: 12 Jun 2024

https://github.com/SilentDemonSD/FZBypassBot

A Elegant Fast Multi Threaded Bypass Bot for Bigger Deeds. Try Now !!

bypass bypasscaptcha bypassing link-bypasser link-shortener scrape scrapers scraping-websites telegram-bot

Last synced: 28 May 2024

https://github.com/driscoll42/ebayMarketAnalyzer

Scrape all eBay sold listings to determine average/median pricing, plot listings over time with trend lines, and extract to excel

ebay python scraping-websites webscraping

Last synced: 26 May 2024

https://github.com/elixir-crawly/crawly

Crawly, a high-level web crawling & scraping framework for Elixir.

crawler crawling elixir erlang extract-data scraper scraping scraping-websites spider

Last synced: 05 May 2024

https://github.com/slotix/dataflowkit

Extract structured data from web sites. Web sites scraping.

cdp chrome-fetcher crawling extract-data go golang golang-library headless scraper scraping scraping-websites

Last synced: 04 May 2024

https://github.com/DiegoCaraballo/Email-extractor

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

email email-extractor email-marketing emails extraction python scraper scrapers scraping scraping-websites scrapper scrapping scrapy scrapy-spider spyder stractor

Last synced: 01 May 2024

https://github.com/brahma-dev/metafetch

NodeJS package that fetches a given URL's title, description, images, links etc.

javascript meta-tags scraper scraping-websites

Last synced: 27 Apr 2024

https://github.com/Anorov/cloudflare-scrape

A Python module to bypass Cloudflare's anti-bot page.

anti-bot-page cloudflare protected-page scrape scraping-websites

Last synced: 26 Apr 2024

https://github.com/Python-World/Python_and_the_Web

Build Bots, Scrape a website or use an API to solve a problem.

api bots fun hacktoberfest python scraping-websites

Last synced: 19 Apr 2024

https://github.com/strugee/fulldom-server

Proxy-like server that will show you the DOM of a page after JS runs

daemon dom hacktoberfest nodejs scraping scraping-websites server

Last synced: 15 Apr 2024

https://github.com/RuthGnz/SpyScrap

CLI and GUI for OSINT. Are you very exhibited on the Internet? Check it! Twitter, Tinder, Facebook, Google, Yandex, BOE. It uses facial recognition to provide more accurate results.

boe cli docker facebook facial-recognition google osint reverse-image-search scraping-websites social-networks socmint tinder twitter

Last synced: 09 Apr 2024

https://github.com/NightMachinery/readability-cli

A CLI for Mozilla Readability. Get clean, uncluttered, ready-to-read HTML from any webpage!

cleaner cli html mercury-parser mozilla-readability read readability reader sanitize-html scrape scraping scraping-websites webpage

Last synced: 08 Apr 2024

https://github.com/w0ltage/subchase

Chase subdomains by parsing the results of Google and Yandex search results

enumeration scraping-websites search-engine subdomain-enumeration

Last synced: 08 Apr 2024

https://github.com/csbun/thal

译文：Puppeteer 与 Chrome Headless —— 从入门到爬虫

headless-chrome puppeteer scraping-websites

Last synced: 06 Apr 2024

https://github.com/Cartmanishere/zippyshare-scraper

A module to get direct downloadable links from zippyshare download page.

download-links multithreading scraping-websites zippyshare zippyshare-scraper

Last synced: 27 Mar 2024

https://github.com/JusticeRage/ApkTrack

ApkTrack is an Android app which checks if updates for installed APKs are available.

android android-app apk scraping-websites

Last synced: 26 Mar 2024

https://github.com/cotoami/cotoami-scraper

Cotoami Scraper is a Chrome Extension that scrapes web pages to generate inputs for Cotoami knowledge-bases.

kindle kindle-highlights knowledgebase scraper scraping scraping-websites

Last synced: 21 Mar 2024