https://github.com/devops-works/slowql
A package to parse slow query logs in Go, with associated tooling (digester, replayer...), and compatible with multiple databases (MySQL, MariaDB, PerconaDB...).
https://github.com/devops-works/slowql
go mariadb mysql package percona slow-query-logs sql
Last synced: 8 months ago
JSON representation
A package to parse slow query logs in Go, with associated tooling (digester, replayer...), and compatible with multiple databases (MySQL, MariaDB, PerconaDB...).
- Host: GitHub
- URL: https://github.com/devops-works/slowql
- Owner: devops-works
- License: mit
- Created: 2021-02-23T10:09:58.000Z (almost 5 years ago)
- Default Branch: main
- Last Pushed: 2024-02-15T08:36:09.000Z (almost 2 years ago)
- Last Synced: 2024-11-16T14:36:07.460Z (about 1 year ago)
- Topics: go, mariadb, mysql, package, percona, slow-query-logs, sql
- Language: Go
- Homepage:
- Size: 121 KB
- Stars: 38
- Watchers: 5
- Forks: 8
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# slowql
[](https://pkg.go.dev/devops-works/slowql)
[](https://goreportcard.com/report/github.com/devops-works/slowql)
A slow query logs parser in Golang.
- [slowql](#slowql)
- [Getting started](#getting-started)
- [Basic usage](#basic-usage)
- [Performance](#performance)
- [Associated tools](#associated-tools)
- [Notes](#notes)
- [Tested databases](#tested-databases)
- [Contributing](#contributing)
- [License](#license)
## Getting started
```
go get github.com/devops-works/slowql
```
## Basic usage
In a nutshell, you can use it as follows:
```go
package main
import (
"fmt"
"github.com/devops-works/slowql"
)
func main() {
// Imagine that fd is an io.Reader of your slow query logs file...
// Create the parser
p := slowql.NewParser(slowql.MySQL, fd)
// Get the next query from the log
q, err := p.GetNext()
if err != nil {
panic(err)
}
// Do your stuff, for example:
fmt.Printf("at %d, %s did the request: %s\n", q.Time, q.User, q.Query)
fp, err := q.Fingerprint()
if err != nil {
panic(err)
}
fmt.Printf("the query fingerprint is: %s\n", fp)
srv := p.GetServerMeta()
fmt.Printf("the SQL server listens to port %d", srv.Port)
}
```
## Performance
Running the example given in cmd/ without any `fmt.Printf` against a 292MB slow query logs from a MySQL database provides the following output:
```
parsed 278077 queries in 8.099622786s
```
which is approx. **34760 queries/second** (Intel i5-8250U (8) @ 3.400GHz).
## Associated tools
With this package we created some tools:
* [slowql-replayer](https://github.com/devops-works/slowql/tree/develop/cmd/slowql-replayer): replay and benchmark queries from a slow query log
* [slowql-digest](https://github.com/devops-works/slowql/tree/develop/cmd/slowql-digest): digest and analyze slow query logs. Similar to `pt-query-digest`, but faster. :upside_down_face:
## Notes
### Tested databases
Not all kind of slow query logs have been tested yet:
- [X] MySQL
- [X] MariaDB
- [ ] Percona-db
- [X] Percona-cluster (pxc)
## Contributing
Issues and pull requests are welcomed ! If you found a bug or want to help and improve this package don't hesitate to fork it or open an issue :smile:
## License
MIT