https://github.com/jiusanzhou/surferua
Surfer User-Agent (surferua) is a lightweight and high performance Golang package that generate HTTP User-Agent strings with particular attention to device type. 高性能 Golang 爬虫浏览器随机库。
https://github.com/jiusanzhou/surferua
fake-data fake-header go-crawler golang user-agent useragent zoe-lab
Last synced: about 1 month ago
JSON representation
Surfer User-Agent (surferua) is a lightweight and high performance Golang package that generate HTTP User-Agent strings with particular attention to device type. 高性能 Golang 爬虫浏览器随机库。
- Host: GitHub
- URL: https://github.com/jiusanzhou/surferua
- Owner: jiusanzhou
- License: apache-2.0
- Created: 2017-11-02T11:36:31.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2020-11-04T08:30:52.000Z (over 4 years ago)
- Last Synced: 2025-01-22T17:07:29.053Z (4 months ago)
- Topics: fake-data, fake-header, go-crawler, golang, user-agent, useragent, zoe-lab
- Language: Go
- Homepage: https://labs.zoe.im/surferua
- Size: 90.8 KB
- Stars: 15
- Watchers: 2
- Forks: 1
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Logo from uasurfer
Surfer UA
High performance User-Agent generator in Golang.
---
Surfer User-Agent (surferua) is a lightweight Golang package that generate HTTP User-Agent strings with particular attention to device type.
## Start
### Basic usage
- Install with Go: `go get go.zoe.im/surferua`
- **Enjoy it!**
Some example code below:
```golang
package main
import (
"fmt"
"go.zoe.im/surferua"
)
func main() {
fmt.Println(surferua.New().String())
// functions depends on your generated inputting data.
fmt.Println(surferua.NewBot())
fmt.Println(surferua.NewBotGoogle())
fmt.Println(surferua.New().Phone().String())
fmt.Println(surferua.New().Android().String())
fmt.Println(surferua.New().Desktop().Chrome().String())
}
```
### Customize the User-Agent database
- Open the `config.yml` with any editor you like.
- Edit the c`yaml` file as you can image what it do
```yaml
# You can get all User-Agent from: http://www.webapps-online.com/online-tools/user-agent-strings
# If you want to add UA of some version or change them,
# you just need to edit this file and generate code with `go generate` again.
browsers:
Firefox:
engine:
name: Gecko
version:
major: [35, 56]
minor: [35, 56]
patch: [0, 3]
Chrome:
engine:
name: AppleWebKit
version:
major: [534, 603]
minor: [35, 56]
version:
major: [39, 64]
minor: 0
patch: [0, 3000]
Safari:
engine:
name: WebKit
version:
major: [534, 603]
minor: [0, 21]
patch: [0, 10]
version:
major: [5, 11]
minor: [0, 2]
patch: [0, 10]
platforms:
Desktop:
Linux:
MacOS:
Windows:
Phone:
iOS:
version:
major: [6, 11]
minor: [0, 3]
patch: [0, 3]
Android:
version:
major: [4, 8]
minor: [0, 4]
patch: [0, 4]
bots: # Auto upper the first letter
google: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
bing: Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
yahoo: Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
mj12: Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php)
simplePie: SimplePie/1.3.1 (Feed Parser; http://simplepie.org; Allow like Gecko)
blex: Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)
yandex: Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
scoutJet: Mozilla/5.0 (compatible; ScoutJet; +http://www.scoutjet.com/)
duck: DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)
baidu: Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
sogou: Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
sogouOrin: Sogou Orion spider/3.0( http://www.sogou.com/docs/help/webmasters.htm#07)
exa: Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Exabot-Thumbnails)
facebook: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
alexa: ia_archiver (+http://www.alexa.com/site/help/webmasters; [email protected])
msn: msnbot/2.0b (+http://search.msn.com/msnbot.htm)
```
- `go generate` or `make gen` in `*nix`
## TODO
- [ ] Add functional way to create UA factory
- [ ] Add testing