https://github.com/yoheimuta/chromedp-example
chromedp Example Go App Scraping Sneaker Price List in StockX
https://github.com/yoheimuta/chromedp-example
chromedp example golang headless-chrome stockx
Last synced: about 2 months ago
JSON representation
chromedp Example Go App Scraping Sneaker Price List in StockX
- Host: GitHub
- URL: https://github.com/yoheimuta/chromedp-example
- Owner: yoheimuta
- Created: 2019-03-07T05:55:26.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2019-03-14T09:38:17.000Z (about 6 years ago)
- Last Synced: 2025-04-10T00:15:35.530Z (about 2 months ago)
- Topics: chromedp, example, golang, headless-chrome, stockx
- Language: Go
- Homepage:
- Size: 7.16 MB
- Stars: 12
- Watchers: 2
- Forks: 6
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# chromedp-example
[](https://circleci.com/gh/yoheimuta/chromedp-example)
Scraping specific sneakers' bid price list in StockX:
- How to work:
- Go to https://stockx.com/buy/air-jordan-1-retro-high-og-defiant-couture.
- Wait to complete loading the confirmation page.
- Click the confirm button.
- Wait to complete loading the size list page.
- Retrieve the sizes and prices.
## Setup
Set the project root path to GOPATH.
```bash
mkdir chromedp-example
cd chromedp-example
export GOPATH=$(pwd)
go get -d github.com/yoheimuta/chromedp-example # ignore a `no Go files` error.
cd src/github.com/yoheimuta/chromedp-example
```
Run your chrome headless-shell.
```bash
docker pull chromedp/headless-shell
docker run -d -p 9222:9222 --rm --name headless-shell chromedp/headless-shell
```
## Run
There are 3 main.go which use different logic to demonstrate how to utilize the chromedp.
In fact, each code differs only repository injected in main.go.
```bash
# normal implementation.
go run cmd/stockx/main.go
# faster implementation. This uses the cookie to skip the confirmation page.
go run cmd/faststockx/main.go
# parallel support implementation. This is more performant.
## NOTE: This uses a local chrome app.
go run cmd/parallelstockx/main.go
```
## Testing
```bash
go test -v -count 1 -timeout 240s -race ./...
```