https://github.com/brandenc40/safer
An API to scrape data from the Department of Transportation's Safety and Fitness Electronic Records (SAFER) System.
https://github.com/brandenc40/safer
company dot fmcsa golang mc mx safer scaping scrape scraper snapshot usdot webscraping
Last synced: 7 months ago
JSON representation
An API to scrape data from the Department of Transportation's Safety and Fitness Electronic Records (SAFER) System.
- Host: GitHub
- URL: https://github.com/brandenc40/safer
- Owner: brandenc40
- License: apache-2.0
- Created: 2021-08-15T18:22:34.000Z (about 4 years ago)
- Default Branch: master
- Last Pushed: 2022-10-31T20:18:49.000Z (almost 3 years ago)
- Last Synced: 2025-04-11T05:49:39.838Z (7 months ago)
- Topics: company, dot, fmcsa, golang, mc, mx, safer, scaping, scrape, scraper, snapshot, usdot, webscraping
- Language: Go
- Homepage: https://safer.fmcsa.dot.gov/CompanySnapshot.aspx
- Size: 81.1 KB
- Stars: 6
- Watchers: 2
- Forks: 6
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# SAFER Scraper API [GoLang]
[](https://goreportcard.com/report/github.com/brandenc40/go-safer)
[](https://pkg.go.dev/github.com/brandenc40/safer)
[](https://codecov.io/gh/brandenc40/safer)
[](https://github.com/brandenc40/safer/actions/workflows/go.yml)
An API to scrape data from the Department of Transportation's Safety and Fitness Electronic Records
([SAFER](https://safer.fmcsa.dot.gov/CompanySnapshot.aspx)) System.
Scraping is performed using [htmlquery](https://github.com/antchfx/htmlquery), this project's only non std lib dependency.
## Installation
```shell
go get -u github.com/brandenc40/safer
```
## Client Methods
```go
// GetCompanyByDOTNumber - Get a company snapshot by the companies DOT number. Returns ErrCompanyNotFound if
// no company is found
func (c *Client) GetCompanyByDOTNumber(dotNumber string) (*CompanySnapshot, error)
// GetCompanyByMCMX - Get a company snapshot by the companies MC/MX number. Returns ErrCompanyNotFound if no
// company is found.
//
// Note: do not include the prefix. (e.g. use "133655" not "MC-133655")
func (c *Client) GetCompanyByMCMX(mcmx string) (*CompanySnapshot, error)
// SearchCompaniesByName - Search for all carriers with a given name. Name queries will return the best matched results
// in a slice of CompanyResult structs.
func (c *Client) SearchCompaniesByName(name string) ([]CompanyResult, error)
```
### Build a new Client
```go
package main
import (
"github.com/brandenc40/safer"
)
func main() {
client := safer.NewClient()
//... use the client
}
```
### Scraping Benchmark
Benchmarks only test the time taken to parse the html and map it back to the output. Server time is ignored here.
```shell
goos: darwin
goarch: arm64
pkg: github.com/brandenc40/safer
BenchmarkClient_GetCompanyByDOTNumber-8 9145 132111 ns/op 91557 B/op 2672 allocs/op
BenchmarkClient_Search_4Results-8 94274 12821 ns/op 9559 B/op 285 allocs/op
BenchmarkClient_Search_484Results-8 1048 1137919 ns/op 826245 B/op 24774 allocs/op
PASS
ok github.com/brandenc40/safer 4.942s
```