https://github.com/osaguild/scheduled-scraper
scraping library for data provider
https://github.com/osaguild/scheduled-scraper
scraping selenium typescript
Last synced: about 2 months ago
JSON representation
scraping library for data provider
- Host: GitHub
- URL: https://github.com/osaguild/scheduled-scraper
- Owner: osaguild
- License: mit
- Created: 2022-08-01T09:58:02.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2023-09-18T08:20:53.000Z (over 2 years ago)
- Last Synced: 2025-01-25T04:54:58.862Z (over 1 year ago)
- Topics: scraping, selenium, typescript
- Language: TypeScript
- Homepage:
- Size: 97.7 MB
- Stars: 0
- Watchers: 2
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# scheduled-scraper
It is automatically executed and data is stored using github actions. to make it easy for everyone to use, we saved it in json format and made it available through the github api.
## scraping data
| name | description | url | data | since |
| :---------------------- | :---------- | :---------------------------------------------------------------------------------------------------- | :--------------- | :--------- |
| kaldi sale info | TBD | https://map.kaldi.co.jp/kaldi/articleList?account=kaldi&accmd=1&ftop=1&kkw001=2010-03-12T13%3A10%3A35 | ./data/kaldi | 2022-08-05 |
| shamaison building info | TBD | https://www.shamaison.com | ./data/shamaison | 2022-08-09 |
## how to use
through the github api is better way to use. example commands are below.
| use case | command |
| :------------- | :-------------------------------------------------------------------------------------------- |
| get file lists | `curl 'https://api.github.com/repos/osaguild/scheduled-scraper/contents/data/kaldi?ref=main'` |