https://github.com/rvflash/ds
Estimates data size in bytes
https://github.com/rvflash/ds
Last synced: 2 months ago
JSON representation
Estimates data size in bytes
- Host: GitHub
- URL: https://github.com/rvflash/ds
- Owner: rvflash
- License: mit
- Created: 2020-10-21T20:40:55.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-03-30T20:56:38.000Z (about 2 years ago)
- Last Synced: 2025-01-28T16:16:29.989Z (4 months ago)
- Language: Go
- Size: 36.1 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ds
[](https://godoc.org/github.com/rvflash/ds)
[](https://travis-ci.org/rvflash/ds?branch=master)
[](https://codecov.io/gh/rvflash/ds)
[](https://goreportcard.com/report/github.com/rvflash/ds)
`ds` estimates the data size in bytes.
## ds mysql
The subcommand `mysql` allows estimating data size of databases, tables, by columns or keys, based on SQL statements.
`ds` estimates data sizes by parsing the SQL statements from the named files (or standard input) and
prints the result in ASCII or CVS format, if the batch mode is enabled.
For example, based on the SQL statements in the [sample.sql](testdata/mysql/sample.sql) file,
which generates the following table:
```
mysql> desc city;
+--------------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+----------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | char(35) | NO | | | |
| country_code | char(3) | NO | MUL | | |
| district | char(20) | NO | | | |
| population | int(11) | NO | | 0 | |
+--------------+----------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
```
With the same file, we can estimate its size per one, and a million rows.
See the following command line which also prints details per column or key (verbose mode):
```
$ ds mysql -v -n 1000000 testdata/mysql/sample.sql
+--------------+------------------------+---------------+---------------+-----------------+-----------------+
| DATA | TYPE | PER ROW (MIN) | PER ROW (MAX) | X 1000000 (MIN) | X 1000000 (MAX) |
+--------------+------------------------+---------------+---------------+-----------------+-----------------+
| id | int | 4.00 B | 4.00 B | 4.00 MB | 4.00 MB |
| name | char(35, utf8mb4) | 35.00 B | 35.00 B | 35.00 MB | 35.00 MB |
| country_code | char(3, utf8mb4) | 3.00 B | 3.00 B | 3.00 MB | 3.00 MB |
| district | char(20, utf8mb4) | 20.00 B | 20.00 B | 20.00 MB | 20.00 MB |
| population | int | 4.00 B | 4.00 B | 4.00 MB | 4.00 MB |
| PRIMARY | key(id) | 4.00 B | 4.00 B | 4.00 MB | 4.00 MB |
| country_code | key(country_code) | 7.00 B | 7.00 B | 7.00 MB | 7.00 MB |
| city | table(InnoDB, dynamic) | 77.00 B | 77.00 B | 77.00 MB | 77.00 MB |
| | | | | | |
| country | database | 77.00 B | 77.00 B | 77.00 MB | 77.00 MB |
+--------------+------------------------+---------------+---------------+-----------------+-----------------+
```
Without the verbose mode, only tables with aggregates by database are displayed.
```
$ cat testdata/mysql/sample.sql | ds mysql -n 1000000
+---------+------------------------+---------------+---------------+-----------------+-----------------+
| DATA | TYPE | PER ROW (MIN) | PER ROW (MAX) | X 1000000 (MIN) | X 1000000 (MAX) |
+---------+------------------------+---------------+---------------+-----------------+-----------------+
| city | table(InnoDB, dynamic) | 77.00 B | 77.00 B | 77.00 MB | 77.00 MB |
| country | database | 77.00 B | 77.00 B | 77.00 MB | 77.00 MB |
+---------+------------------------+---------------+---------------+-----------------+-----------------+
```
### Features
- Supports MyISAM engine with Static (Fixed-Length), Dynamic and Compressed table characteristics.
- Supports InnoDB engine with Redundant, Compact, Dynamic and Compressed row formats.
- Supports various statements `CREATE DATABASE`, `DROP DATABASE` or `CREATE TABLE`. More incoming!
- The charset is takes account in the computation.
If no one is defined on the table, the database's charset is used as failover, otherwise `utf8mb4` is used.
- Display the minimum and maximum sizes estimations to handle variable data types.
- Data sizes are calculated per column, per key, per table and per database.
- Results are aggregated by database. If not specified, `unknown` name is used by default.
### Usage
By default, it uses ASCII table to display the report.
Size per row, and for 100 rows are calculated and displayed using 2 decimals.
```
ds mysql [flags] [file.sql]
```
It supports the following flags:
* `-B`: batch mode, print results using comma as the column separator, with each row on a new line.
* `-n`: number of lines to considerate by table (default 100).
* `-p`: number of decimals to display (default 2).
* `-v`: verbose output, produce more output about what the program does.
## Installation
### Go
```shell
GO111MODULE=on go get github.com/rvflash/[email protected]
```
### Docker
```shell
docker run --rm -v $(pwd):/pkg rvflash/ds:v0.1.0 ds
```