Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/crwlrsoft/crawler
Library for Rapid (Web) Crawler and Scraper Development
crawler crawling hacktoberfest php scraper scraping scraping-websites web-crawler web-crawling web-scraper web-scraping
Last synced: 04 Jul 2024
https://github.com/php-curl-class/php-curl-class
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
api api-client class client curl framework http http-client http-proxy json php php-curl php-curl-library proxy requests restful web-scraper web-scraping web-service xml
Last synced: 30 Jun 2024
https://github.com/palahsu/ScraperImages
Simple Web Images Scraper From Websites!
google-photos-download google-scraper google-scraping image-scraper imagescraper imagescraping scraper scraping scraping-python web-scraper webscraper webscraper-website
Last synced: 27 Jun 2024
https://github.com/shaikhsajid1111/facebook_page_scraper
Scrapes facebook's pages front end with no limitations & provides a feature to turn data into structured JSON or CSV
csv facebook facebook-apis facebook-page facebook-page-post facebook-page-post-scraper facebook-page-scraper facebook-scraper fb fb-scrapper hacktoberfest open-source python scraper selenium social-media web-scraper web-scraping
Last synced: 27 Jun 2024
https://github.com/shaikhsajid1111/social-media-profile-scrapers
Fetch user's data across social media
facebook-scraper instagram-scraper medium-scraper pinterest pinterest-scrapper python quora-scraper reddit-scraper request scrapping-python selenium-python social-media tiktok-scraper twitter-scraper web-scraper web-scraping
Last synced: 27 Jun 2024
https://github.com/kyng-cytro/Screen-Time-Bot
Telegram bot to get Movies and TV-Shows updates
bot mongodb python3 telegram telegram-bot web-scraper web-scraping
Last synced: 27 Jun 2024
https://github.com/juancarlospaco/faster-than-requests
Faster requests on Python 3
curl cython download-file faster-than-requests high-performance http-requests ndjson open-data python python-library python-requests python3 requests-toolbelt requests3 scrapy speed urllib urllib3 web-scraper web-scraping
Last synced: 26 Jun 2024
https://github.com/Oshan96/monkey-dl
Bulk download your favourite anime episodes from your favourite anime websites
4anime 9anime anime anime-downloader anime-fans anime-scraper anime-search animepahe animepahe-downloader animeultima ffmpeg hls-downloader monkey-dl web-scraper
Last synced: 24 Jun 2024
https://github.com/AlexMathew/scrapple
A framework for creating semi-automatic web content extractors
beautifulsoup crawler css-selector extractor lxml python scrapers scraping scrapy selector selector-expression tutorial web-scraper web-scraping xpath-expression
Last synced: 24 Jun 2024
https://github.com/tholian-network/stealth
:rocket: Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy
anonymity browser-automation privacy-protection web-browser web-filter web-proxy web-scraper
Last synced: 09 Jun 2024
https://github.com/suntong/cascadia
Go cascadia package command line CSS selector
cascadia command-line command-line-tool css-selector csv-table curl extract html-source html-text tsv web-scraper web-scraping
Last synced: 03 Jun 2024
https://github.com/gildas-lormeau/single-file-cli
CLI tool for saving a faithful copy of a complete web page in a single HTML file
cli deno nodejs single-file web-archiving web-scraper web-scraping
Last synced: 01 Jun 2024
https://github.com/k0rnh0li0/onlyfans-dl
OnlyFans content downloader
media-downloader onlyfans python web-scraper
Last synced: 31 May 2024
https://github.com/anthonygharvey/developer_news_cli_app
A Ruby CLI app that scrapes article information from FreeCodeCamp, HackerNoon, Codeburst and A List Apart.
cli-app nokogiri ruby ruby-gem web-scraper
Last synced: 28 May 2024
https://github.com/cassidoo/scrapers
A list of scrapers from around the web.
list scrape-websites scraper web-scraper
Last synced: 28 May 2024
https://github.com/khuyentran1401/top-github-scraper
Scape top GitHub repositories and users based on keywords
github github-api python scraping web-scraper web-scraping
Last synced: 27 May 2024
https://github.com/aaryanrr/DownDetector-CLI
CLI Client for DownDetector.com
cli downdetector python3 web-scraper web-scraping
Last synced: 26 May 2024
https://github.com/PhantomInsights/tweet-transcriber
A Reddit bot that transcribes tweets from comments and submissions links, mirrors their images and replies back with a formatted Markdown message.
beautifulsoup imgur praw python3 reddit-bot web-scraper
Last synced: 21 May 2024
https://github.com/PhantomInsights/summarizer
A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.
nlp praw python3 reddit-bot spacy web-scraper wordcloud
Last synced: 21 May 2024
https://github.com/arpit-omprakash/100ProjectsOfCode
A list of practical knowledge-building projects.
c cpp11 csharp java javascript music-player processing programming projects python search-engine web-scraper
Last synced: 14 May 2024
https://github.com/anaskhan96/soup
Web Scraper in Go, similar to BeautifulSoup
beautifulsoup go golang html-node web-scraper webscraper webscraping
Last synced: 11 May 2024
https://github.com/Alnyz/NewApkPure
Search and download applications from apkpure.com
android api api-wrapper apkpure apkpure-scraper downloader python python3 scraping web web-scraper web-scraping wrapper
Last synced: 06 May 2024
https://github.com/postmodern/spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
crawler ruby scraper spider spider-links web web-crawler web-scraper web-scraping web-spider
Last synced: 01 May 2024
https://github.com/austinoboyle/scrape-linkedin-selenium
`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
linkedin python scrape scraper scraping selenium selenium-webdriver web-scraper web-scraping
Last synced: 26 Apr 2024
https://github.com/BruceDone/awesome-crawler
A collection of awesome web crawler,spider in different languages
awesome crawler node-crawler scraper spider web-crawler web-scraper
Last synced: 19 Apr 2024
https://github.com/milahu/aiohttp_chromium
aiohttp-like interface to chromium. based on selenium_driverless to bypass cloudflare
aiohttp asyncio bypass-cloudflare chromium gui-scripting headful-chromium headful-scraper headful-web-scraper headful-webscraper selenium-driverless web-scraper web-scraping
Last synced: 14 Apr 2024
https://github.com/paulpierre/markdown-crawler
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file for each page, designed for LLM RAG
html-to-markdown html-to-markdown-converter html2md llm llmops markdown markdown-crawler markdown-parser markdown-scraper md-crawler rag web-scraper
Last synced: 11 Apr 2024
https://github.com/scrapehero/yellowpages-scraper
Yellowpages.com Web Scraper written in Python and LXML to extract business details available based on a particular category and location.
business-directory extract html lxml parsing python scraper web-scraper yellow-pages yellow-pages-scraper
Last synced: 08 Apr 2024
https://github.com/duyet/awesome-web-scraper
A collection of awesome web scaper, crawler.
awesome awesome-list goutte phantomjs php scrapy slimerjs spider storage web-crawler web-scraper
Last synced: 28 Mar 2024
https://github.com/spekulatius/PHPScraper
A universal web-util for PHP.
beautifulsoup chromium headless-chrome php php-crawler php-scraper php-spider php-spiders puppeteer pyppeteer scraper scraping scraping-websites scrapy web-scraper web-scraping
Last synced: 27 Mar 2024
https://github.com/epiqueras/getsy
A simple browser/client-side web scraper.
browser client-side scraper web-scraper
Last synced: 25 Mar 2024
https://github.com/serpapi/public-roadmap
Public roadmap for SerpApi, LLC (https://serpapi.com)
baidu-scraper google-image-scraper google-maps-scraping google-search-scraper scraper scraping serp-api serpapi web-scraper web-scraping webscraping yahoo-scraper
Last synced: 19 Mar 2024
https://github.com/gosom/google-maps-scraper
scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
distributed-scraper distributed-scraping golang google-maps google-maps-scraping web-scraper web-scraping
Last synced: 17 Mar 2024
https://github.com/cobalt-uoft/uoft-scrapers
Public web scraping scripts for the University of Toronto.
open-data toronto uoft web-scraper
Last synced: 16 Mar 2024
