https://github.com/lucastaliberti/amazon-product-search

Amazon Product Search via web scraping
https://github.com/lucastaliberti/amazon-product-search

Last synced: 12 days ago
JSON representation

Amazon Product Search via web scraping

• Host: GitHub
• URL: https://github.com/lucastaliberti/amazon-product-search
• Owner: lucastaliberti
• Created: 2020-03-23T02:46:53.000Z (almost 5 years ago)
• Default Branch: master
• Last Pushed: 2023-01-24T01:41:18.000Z (almost 2 years ago)
• Last Synced: 2024-11-12T12:16:37.731Z (2 months ago)
• Language: TypeScript
• Homepage:
• Size: 2.51 MB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 40
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Amazon Product Search

This project shows how is possible to search products at amazon.com using a scraping technique.

To make this possible, I've used the `puppeteer` package to simulate a user browsing the website and then using the web API to query the correct selectors.

The data is then saved to a document database that acts like a cache to following queries.

The frontend uses the apollo client to query the backend results (either cache or scrape) through the GraphQL server and cache it on the frontend. 

## Getting started

*Make sure you have docker installed*

1. `yarn run docker:up`

2. `cd ./backend && yarn start:dev`

3. `cd ./frontend && yarn start`

## Tests

1. `cd ./backend && yarn test`

2. `cd ./frontend && yarn test`