https://github.com/sanjeev662/auction-scraper

Auction Scraper is a web application that allows users to scrape auction data, view auction listings, and analyze user bidding statistics.
https://github.com/sanjeev662/auction-scraper

cheerio mysql nodejs puppeteer reactjs

Last synced: about 2 months ago
JSON representation

Auction Scraper is a web application that allows users to scrape auction data, view auction listings, and analyze user bidding statistics.

• Host: GitHub
• URL: https://github.com/sanjeev662/auction-scraper
• Owner: sanjeev662
• Created: 2024-09-15T22:34:07.000Z (4 months ago)
• Default Branch: main
• Last Pushed: 2024-09-25T21:04:39.000Z (3 months ago)
• Last Synced: 2024-10-13T22:21:39.567Z (3 months ago)
• Topics: cheerio, mysql, nodejs, puppeteer, reactjs
• Language: JavaScript
• Homepage:
• Size: 497 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Auction Scraper

Auction Scraper is a web application that allows users to scrape auction data, view auction listings, and analyze user bidding statistics.

## Main Features

### Frontend

1. **Auction List**

   - Display a paginated list of auctions

   - Sort auctions by date or price

   - Filter auctions by domain name, user, and date range

   - Export auction data to XLSX or CSV format

2. **Scraper Form**

   - Input form to initiate scraping process

   - Real-time updates on scraping progress

3. **User Details**

   - View detailed bidding statistics for individual users

   - Display user's first and second place bids

   - Filter and sort user's bidding history

### Backend

1. **Web Scraping**

   - Scrape auction data from a specified website

   - Handle authentication for accessing bid information

   - Store scraped data in a MySQL database

2. **API Endpoints**

   - `/api/scrape`: Initiate the scraping process

   - `/api/auctions`: Retrieve auction data with filtering and sorting options

   - `/api/user-stats/:username`: Get bidding statistics for a specific user

3. **Database Operations**

   - Save and update auction data

   - Retrieve auction data with various filters and sorting options

   - Calculate user bidding statistics

## Technologies Used

- Frontend: React, Axios, XLSX

- Backend: Node.js, Express, MySQL, Puppeteer, Cheerio

- Database: MySQL

## Getting Started

1. Clone the repository

2. Install dependencies for both frontend and backend

3. Set up the MySQL database and configure the connection

4. Start the backend server

5. Start the frontend development server

6. Access the application in your web browser