https://github.com/jxeng/site-info-crawler
A tool for batch crawling website's title, description, favicon.
https://github.com/jxeng/site-info-crawler
crawler favicon title
Last synced: 23 days ago
JSON representation
A tool for batch crawling website's title, description, favicon.
- Host: GitHub
- URL: https://github.com/jxeng/site-info-crawler
- Owner: jxeng
- License: mit
- Created: 2022-06-28T10:27:43.000Z (almost 4 years ago)
- Default Branch: master
- Last Pushed: 2022-06-29T02:10:30.000Z (almost 4 years ago)
- Last Synced: 2025-12-17T07:44:31.751Z (6 months ago)
- Topics: crawler, favicon, title
- Language: Go
- Homepage:
- Size: 5.86 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# site-info-crawler
A tool for batch crawling website's title, description, favicon.
# how to use
```
git clone https://github.com/jxeng/site-info-crawler.git
cd site-info-crawler
go mod tidy
go build
./site-info-crawler.exe
```
raw.json
```
[
{
"id": "binance",
"url": "https://www.binance.com/",
"title": "",
"describe": "",
"favicon": "binance_com.png"
},
...
]
```
filled.json
```
[
{
"id": "binance",
"url": "https://www.binance.com/",
"title": "交易比特币、以太币和altcoin | 加密货币交易平台 | 币安",
"description": "Binance cryptocurrency exchange - We operate the worlds biggest bitcoin exchange and altcoin crypto exchange in the world by volume",
"favicon": "./icons/binance.png"
},
...
]
```