https://github.com/moinulmoin/web-scraper-starter
https://github.com/moinulmoin/web-scraper-starter
Last synced: 8 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/moinulmoin/web-scraper-starter
- Owner: moinulmoin
- Created: 2024-04-05T20:01:44.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-04-05T22:41:00.000Z (over 1 year ago)
- Last Synced: 2025-04-05T15:12:23.366Z (9 months ago)
- Language: TypeScript
- Size: 13.7 KB
- Stars: 3
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Web Scraper Starter
## Introduction
A starter project for web scraping using Node.js and Playwright. This project demonstrates how to scrape data from a website using Playwright and store it in a database.
## Local Setup
1. Clone the repository.
2. Install dependencies with `bun install`.
3. Create a `.env` file from `.env.example` and add your Redis host and port.
3. Start the development server with `bun run dev`.
## Usage
Scraping data from a website using headless browser, queues the data to Redis and then processes it in the background and stores it in a database.