Projects in Awesome Lists tagged with scrap

https://github.com/grindsa/dkb-robo

library to access the internet banking area of "Deutsche Kreditbank" to get account information and transactions

deutsche dkb kreditbank mechanize python robo scrap

Last synced: 08 Apr 2025

https://github.com/serp-spider/core

:spider: The PHP SERP Spider - A search engine scraper

scrap scrape scraping serp

Last synced: 05 Apr 2025

https://github.com/fernandod1/instagram-downloader

Instagram user's photos and videos downloader. Download all media files from any username. Working 2022!

crawler crawling-python instagram instagram-downloader instagram-feed instagram-photos instagram-scraper python scrap scraper scraping scraping-python scraping-tool scraping-websites

Last synced: 03 May 2025

https://github.com/codedotjs/porn-hub

:fire: :fire: Horny Porny Corny

porn pornhub pornhub-api scrap

Last synced: 18 Aug 2025

https://github.com/codedotjs/fbvideos

:link: Easily extract downloadable link of publicly available videos on facebook.

api extract facebook fetch link nodejs scrap url videos

Last synced: 18 Aug 2025

https://github.com/danielnieto/scrapman

Retrieve real (with Javascript executed) HTML code from an URL, ultra fast and supports multiple parallel loading of webs

electron javascript javascript-tools scrap scraper scraping scraping-websites

Last synced: 23 Apr 2025

https://github.com/codedotjs/twifo-cli

:baby_chick: Get user information of a Twitter user.

cli-app information scrap twitter user

Last synced: 18 Aug 2025

https://github.com/zeeshanahmad4/tweets-scraper-twitter-selenium

Recently twitter has stopped giving thier Apis to everyone, So i write simple Python script using selenium to scrap all the tweets from a user and store into a csv file

csv python python-script scrap selenium selenium-python tweepy tweets tweets-scraper tweetsscraperselenium twitter twitter-bot twitterscraper

Last synced: 12 Aug 2025

https://github.com/aianytime/project-websage

WebSage is an AI Engine that extracts content from any URL, generates summaries, and enables interaction using AI models. Choose between Closed Source (OpenAI) or Open Source (Ollama) for flexible summarization and conversation.

ai chatbot crawl langchain rag scrap

Last synced: 24 Jun 2025

https://github.com/dinhanhx/PerceiveYourMeme

A Python library to scrap from https://knowyourmeme.com/ . Written in Python 3.7!

beginner-friendly bs4 dataset hacktoberfest knowyourmeme library meme memes memes-api python-3 python3 scrap urllib3

Last synced: 09 May 2025

https://github.com/dinhanhx/perceiveyourmeme

A Python library to scrap from https://knowyourmeme.com/ . Written in Python 3.7!

beginner-friendly bs4 dataset hacktoberfest knowyourmeme library meme memes memes-api python-3 python3 scrap urllib3

Last synced: 12 Apr 2025

https://github.com/fckveza/create-proxy

HTTP / HTTPS Proxy

bot-line bot-wa proxy proxy-https proxy-server scrap scraping scraping-api

Last synced: 03 Mar 2026

https://github.com/mirukana/lunafind

Search, filter, inspect, download posts from Danbooru/Safebooru and find them back from the command line without setting up any service or database.

anime api booru cli client danbooru download filter image lunakit lunasync order safebooru scrap search tag terminal ugoira

Last synced: 07 Sep 2025

https://github.com/vigzmv/lyricaly

:musical_note: Lyricaly gets Lyrics delivered to your Terminal for any Song. Uses Python beautifulsoup4 to scrap lyrics. pypi: lyricaly

beautifulsoup lyrics musixmatch scrap songs

Last synced: 26 Oct 2025

https://github.com/shitgram/node-shitgram

A JavaScript library to make requests to Instagram

albums highlights images insta instagram javascript nodejs plugins promise scrap sessionid shitgram stories users videos

Last synced: 11 Apr 2025

https://github.com/moaminsharifi/image_scraper

You do want to find high quality image with custom setting? here is what you need with Search Engines Image Scraper You can easily find and download image with custom size , color and ratio. Let's start and Start project if you want help to improve It's My honor to check Your pull request.

aspect-ratios duckduckgo image image-processing image-scraper image-scrapping scrap

Last synced: 07 Sep 2025

https://github.com/krishcdbry/url-scraper

Url scraper which takes the text input and finds the links/urls, scraps them using cheerio and will returns an object with original text, parsed text (using npm-text-parser) and array of objects where each object contains scraped webpage's information.

cheerio scrap text-parser url-scraper

Last synced: 28 Jan 2026

https://github.com/ganevdev/actor-webdesignernews-scraper

Scraper for www.webdesignernews.com, using Apify.

actor apify scrap scraper scraping web-scraper

Last synced: 30 Oct 2025

https://github.com/ipazc/proxyup

ProxyUp is a PIP package to retrieve public proxies on the fly.

pip proxies proxy python scrap

Last synced: 12 Apr 2025

https://github.com/irevenko/genshin-scrapper

💾 A bunch of scripts for api.genshin.dev

genshin genshin-api genshin-impact genshinimpact parser scrap scrapper

Last synced: 01 May 2025

https://github.com/shaikhsajid1111/linkedin-serp-scraper

fetch results of LinkedIn public profiles from Bing search engine result page

bing bing-search linkedin linkedin-scraper profile-url python python3 scrap search-engine selenium serp

Last synced: 13 May 2026

https://github.com/conneroisu/seltabl

golang library for parsing html tables into stucts leveraging goquery to be configurable with useful developer tooling including a language server.

cli code-generation golang golang-library golang-package golang-tools goquery html jquery scrap scraper

Last synced: 20 Jul 2025

https://github.com/codedotjs/words-of-the-day

:eyeglasses::eyeglasses: Get word of the day and meanings from the different sources.

api day fetch find nodejs scrap word-of-the-day words

Last synced: 19 Jul 2025

https://github.com/mirukana/lunasync

Download and keep in sync tag searches from Danbooru-based sites, similar to Danbooru tag subscriptions/saved searches.

anime api booru cli danbooru download filter image lunafind lunakit safebooru saved scrap search sync tag terminal ugoira

Last synced: 13 Apr 2025

https://github.com/robertjgabriel/html-to-array

Scrap a html page and convert all text on the page into an array.

array daviddm-url html scrap travis-url

Last synced: 10 Apr 2025

https://github.com/redjanvier/scrap-igihe

This is a simple application of web scrapping JS Library Cheerio and Request with express framework.

cherrio igihe knex scrap

Last synced: 15 May 2025

https://github.com/jauharibill/scrapper

Scrapper is php library that provide simple way to get content/data from spesific website using scrap method.

cra php-library scrap scrapper

Last synced: 04 Jul 2025

https://github.com/aqur1n/unlockedsoilbag

Allows you to put soil bags everywhere

lua mod scrap sm

Last synced: 29 Mar 2025

https://github.com/dqbd/earthview

Sets a random satellite image from Earth View with Google as a wallpaper.

earthview scrap wallpaper

Last synced: 07 Apr 2025

https://github.com/prithivsakthiur/grab-doc-v

MS Word Like Content Creation System

app cdn chatbot chatgpt client docx fpdf grab gradio huggingface llama llama2 llama3 minimalist openai python python-docx scrap

Last synced: 09 Mar 2026

https://github.com/codedotjs/apod-nasa

:earth_americas: Astronomy Picture of the Day!

api day extract fetch nodejs picture picture-of-the-day scrap

Last synced: 05 Apr 2025

https://github.com/codedotjs/starexports

:key: Flask-based tool to scrape and export anyone's starred repositories as JSON or CSV.

csv exporter flask github json python scrap

Last synced: 12 Apr 2026

https://github.com/upi-0/komikuapi

Unofficial Self-Hosted Komiku.id API.

parser python python-scraper scrap

Last synced: 16 Jan 2026

https://github.com/krypton-byte/jagokata

kata-bijak katabijak peribahasa scrap

Last synced: 12 Aug 2025

https://github.com/epsoft/forexfactory

Forex Factory Scraping

epsoft factory forex python scrap scraping

Last synced: 15 May 2026

https://github.com/kusnierewicz/webclip_downloading_automatization

Small application that automaticly download TV commercials from polish version of Nielsen webclip webpage and rename files correctly. To scrap webiste I used Splinter

nielsen-webclip-webpage polish python scrap splinter tv-apps

Last synced: 20 Oct 2025

https://github.com/jesvijonathan/adidas-miner-bot

adidas miner tool allows you to automate, checkout, scrap, mine, etc in adidas.co.in, it was built for my personal use which went well, it can be used as a template for further development

adidas automation bot checkout miner payment python scrap selenium windows