Projects in Awesome Lists tagged with scraper-engine

https://github.com/fredwu/crawler

A high performance web crawler / scraper in Elixir.

crawler elixir files offline scraper scraper-engine spider

Last synced: 15 May 2025

https://github.com/metatube-community/metatube-sdk-go

MetaTube SDK & API Server in Golang

cloudnative emby gin go-colly gorm jellyfin pigo-face-detection postgresql scraper-engine sqlite

Last synced: 14 May 2025

https://github.com/tufayellus/linkedin-scraper

A LinkedIn Scraper to scrape up to 1k LinkedIn profiles(due to LinkedIn limit) from company profile links and save their e-mail addresses if available! (actively maintained, if anything doesn't work, open an issue in the repo)

crawler digital-marketing email-marketing email-scraper leads linkedin linkedin-bot linkedin-gui linkedin-scraper linkedin-scraper-gui scrape-email scrape-emails scraper scraper-engine

Last synced: 27 Oct 2025

https://github.com/santhoshse7en/news-fetch

A Python Package which helps to scrape all news details from any news websites

extracts felix google-search-using-python keyword lucas news news-details news-please news-website newspaper3k python scraper scraper-engine

Last synced: 15 May 2025

https://github.com/obetomuniz/tatooine

A powerful scraper for JavaScript Developers.

html javascript json nodejs scraper scraper-engine spa typescript xml

Last synced: 16 Jul 2025

https://github.com/warifp/shopee-scrape

Shopee Scrape is a tool that functions to collect data - the data needed, such as finding data from photos, prices, names, store locations and others.

curl curl-functions curl-library curlphp indonesia marketplace php php-library scrape scrape-images scrape-websites scraped-data scraper scraper-engine shopee shopee-api

Last synced: 16 Mar 2026

https://github.com/flulemon/sneakpeek

Sneakpeek is a framework that helps to quickly and conviniently develop scrapers. It’s the best choice for scrapers that have some specific complex scraping logic that needs to be run on a constant basis

crawler crawler-python crawlers crawling crawling-engine crawling-framework python python3 scraper scraper-api scraper-engine scrapers scraping scraping-framework vue website-crawler

Last synced: 14 Jan 2026

https://github.com/fernandod1/producthunt-scraper

Producthunt.com famous website scraper script. Scrap all offers and save in spreadsheet excel file.

crawler crawling crawling-sites data-mining datamining producthunt producthunt-api producthunt-users python python-script python3 scrape scraped-data scraper scraper-engine scraping scraping-bot scraping-python scraping-tool scraping-websites

Last synced: 16 Jun 2025

https://github.com/1x-eng/chart_data_extractor

This webservice will help scrape data out of chart(s) presented on any given website. (At this moment, I only support scrape from HighCharts and AmCharts. Other libraries, maybe next time).

amcharts amcharts-js-charts beautifulsoup chart-data chart-data-extractor charts hacktoberfest hacktoberfest2018 highcharts highcharts-js python-chart-scraper python-scraper python-web-scraper scraper scraper-engine

Last synced: 28 Jan 2026

https://github.com/lukluk/scraper-engine

Async Scraper Framework Based Nodejs

scraper-engine scraping

Last synced: 14 Jul 2025

https://github.com/Vitcesar/plugin.video.anitube

Add-on para AniTube, famoso site que ofereceu uma gigantesca biblioteca de animes para assistir online e de graça.

anime anime-scraper anitube kodi kodi-addon kodi-addons kodi-plugin portugues portuguese portuguese-language python scraper scraper-engine scrapers streaming streaming-video xbmc xbmc-addon xbmc-plugin xbmc-video-plugin

Last synced: 28 Mar 2025

https://github.com/scrape-do/node-client

Scrape.do's official http client for node.js

api http-client http-wrapper scrape scrape-data scrape-do scraper scraper-api scraper-engine scraping web-scraping

Last synced: 12 Jun 2026

https://github.com/philipperemy/japanese-street-addresses-scraper

Scraper for Japanese street addresses (住所).

data dataset-creation dataset-generation scraper scraper-engine

Last synced: 08 May 2025

https://github.com/muhammadahmed-0/search-scraper

This Python-based tool scrapes Google search results and presents the top 10 results along with their URLs. Useful for SEO optimization and content writing.

google google-search-scraper google-search-using-python google-serp-scraper google-serp-scraping scraper scraper-engine scraper-python search serp

Last synced: 05 Apr 2025

https://github.com/animeshkundu/pyscrape

Lightweight headless web scraper with javascript rendering in python with http api

headless javascript python scrapejs scraper scraper-engine scrapy

Last synced: 11 May 2025

https://github.com/tufayellus/linkedin-cv-downloader

A Python based GUI automation software for downloading bulk LinkedIn CV / LinkedIn Resume from a list of profile links

crawler digital-marketing email-marketing email-scraper leads linkedin-bot linkedin-cv linkedin-cv-downloader linkedin-download linkedin-downloader linkedin-resume linkedin-resume-downloader linkedin-scraper scrape-emails scrape-websites scraper scraper-engine

Last synced: 17 Mar 2025

https://github.com/davesimoes/web-scraping

The goal of this project is to provide a basic structure for web scraping HTML pages and collecting specific data. The main script (main.py) initializes a WebScraper object and calls the scrape() method to collect data from a specific URL.

collect collect-date html python scraper scraper-engine scraper-python scraper-web webscraper-website webscraping