https://github.com/phantomdd92/web_scrapers

Various website scrapers
https://github.com/phantomdd92/web_scrapers

nodejs playwright python requests scrapy selenium webscraper

Last synced: about 2 months ago
JSON representation

Various website scrapers

• Host: GitHub
• URL: https://github.com/phantomdd92/web_scrapers
• Owner: phantomDD92
• License: mit
• Created: 2024-05-22T11:44:28.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2025-01-27T10:56:09.000Z (over 1 year ago)
• Last Synced: 2025-02-26T12:17:51.656Z (over 1 year ago)
• Topics: nodejs, playwright, python, requests, scrapy, selenium, webscraper
• Language: Python
• Homepage:
• Size: 398 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# web_scrapers

Various website scrapers

## dtchub scraper

Web scraper for https://dtc-hub.com/, Uploader for https://green-triangle-uk.github.io/OrderCloudUIUAT/DTC

### web scraper

Web scraping scripts for https://dtc-hub.com/

Backend : ASP.NET

Frontend: Angular

Authorization: Bearer token

API Endpoint : https://dtchub-api.azurewebsites.net/api

Scraping results: json files, image files

Tools: requests

### uploader

Uploading scripts for https://green-triangle-uk.github.io/OrderCloudUIUAT/DTC

API Endpoint: https://gtlibs.com:7014/api

Swagger : https://gtlibs.com:7014/swagger/index.html

## Biorisk Scraper

Web scraper for https://biorisk-site.s3-website-eu-west-1.amazonaws.com

## Google Downloader

Google Search Page Downloader

### google_image : Google Image Downloader

## Hebeiyoungwill Scraper

Web scraper for https://hebeiyoungwill.com

Framework: Shopify, React