{"id":22018138,"url":"https://github.com/rvflash/ds","last_synced_at":"2026-05-09T09:01:57.421Z","repository":{"id":57549901,"uuid":"306142583","full_name":"rvflash/ds","owner":"rvflash","description":"Estimates data size in bytes","archived":false,"fork":false,"pushed_at":"2023-03-30T20:56:38.000Z","size":37,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-23T09:35:13.804Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rvflash.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-21T20:40:55.000Z","updated_at":"2023-03-30T20:56:44.000Z","dependencies_parsed_at":"2024-06-20T01:37:22.629Z","dependency_job_id":"5c8a3d27-a31d-4ed0-87c4-da74e84284dc","html_url":"https://github.com/rvflash/ds","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rvflash/ds","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rvflash%2Fds","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rvflash%2Fds/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rvflash%2Fds/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rvflash%2Fds/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rvflash","download_url":"https://codeload.github.com/rvflash/ds/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rvflash%2Fds/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266524876,"owners_count":23942833,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-30T05:10:05.099Z","updated_at":"2026-05-09T09:01:57.309Z","avatar_url":"https://github.com/rvflash.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ds\n\n[![GoDoc](https://godoc.org/github.com/rvflash/ds?status.svg)](https://godoc.org/github.com/rvflash/ds)\n[![Build Status](https://api.travis-ci.org/rvflash/ds.svg?branch=master)](https://travis-ci.org/rvflash/ds?branch=master)\n[![Code Coverage](https://codecov.io/gh/rvflash/ds/branch/master/graph/badge.svg)](https://codecov.io/gh/rvflash/ds)\n[![Go Report Card](https://goreportcard.com/badge/github.com/rvflash/ds?)](https://goreportcard.com/report/github.com/rvflash/ds)\n\n\n`ds` estimates the data size in bytes.\n\n## ds mysql\n\nThe subcommand `mysql` allows estimating data size of databases, tables, by columns or keys, based on SQL statements.\n\n`ds` estimates data sizes by parsing the SQL statements from the named files (or standard input) and \nprints the result in ASCII or CVS format, if the batch mode is enabled. \n\nFor example, based on the SQL statements in the [sample.sql](testdata/mysql/sample.sql) file, \nwhich generates the following table:\n\n```\nmysql\u003e desc city;\n+--------------+----------+------+-----+---------+----------------+\n| Field        | Type     | Null | Key | Default | Extra          |\n+--------------+----------+------+-----+---------+----------------+\n| id           | int(11)  | NO   | PRI | NULL    | auto_increment |\n| name         | char(35) | NO   |     |         |                |\n| country_code | char(3)  | NO   | MUL |         |                |\n| district     | char(20) | NO   |     |         |                |\n| population   | int(11)  | NO   |     | 0       |                |\n+--------------+----------+------+-----+---------+----------------+\n5 rows in set (0.00 sec)\n ```\n\nWith the same file, we can estimate its size per one, and a million rows.\nSee the following command line which also prints details per column or key (verbose mode):\n\n```\n$ ds mysql -v -n 1000000 testdata/mysql/sample.sql \n+--------------+------------------------+---------------+---------------+-----------------+-----------------+\n| DATA         | TYPE                   | PER ROW (MIN) | PER ROW (MAX) | X 1000000 (MIN) | X 1000000 (MAX) |\n+--------------+------------------------+---------------+---------------+-----------------+-----------------+\n| id           | int                    |        4.00 B |        4.00 B |         4.00 MB |         4.00 MB |\n| name         | char(35, utf8mb4)      |       35.00 B |       35.00 B |        35.00 MB |        35.00 MB |\n| country_code | char(3, utf8mb4)       |        3.00 B |        3.00 B |         3.00 MB |         3.00 MB |\n| district     | char(20, utf8mb4)      |       20.00 B |       20.00 B |        20.00 MB |        20.00 MB |\n| population   | int                    |        4.00 B |        4.00 B |         4.00 MB |         4.00 MB |\n| PRIMARY      | key(id)                |        4.00 B |        4.00 B |         4.00 MB |         4.00 MB |\n| country_code | key(country_code)      |        7.00 B |        7.00 B |         7.00 MB |         7.00 MB |\n| city         | table(InnoDB, dynamic) |       77.00 B |       77.00 B |        77.00 MB |        77.00 MB |\n|              |                        |               |               |                 |                 |\n| country      | database               |       77.00 B |       77.00 B |        77.00 MB |        77.00 MB |\n+--------------+------------------------+---------------+---------------+-----------------+-----------------+\n```\n\nWithout the verbose mode, only tables with aggregates by database are displayed.\n\n```\n$ cat testdata/mysql/sample.sql | ds mysql -n 1000000 \n+---------+------------------------+---------------+---------------+-----------------+-----------------+\n| DATA    | TYPE                   | PER ROW (MIN) | PER ROW (MAX) | X 1000000 (MIN) | X 1000000 (MAX) |\n+---------+------------------------+---------------+---------------+-----------------+-----------------+\n| city    | table(InnoDB, dynamic) |       77.00 B |       77.00 B |        77.00 MB |        77.00 MB |\n| country | database               |       77.00 B |       77.00 B |        77.00 MB |        77.00 MB |\n+---------+------------------------+---------------+---------------+-----------------+-----------------+\n```\n\n\n### Features\n\n- Supports MyISAM engine with Static (Fixed-Length), Dynamic and Compressed table characteristics.\n- Supports InnoDB engine with Redundant, Compact, Dynamic and Compressed row formats.\n- Supports various statements `CREATE DATABASE`, `DROP DATABASE` or `CREATE TABLE`. More incoming!\n- The charset is takes account in the computation. \nIf no one is defined on the table, the database's charset is used as failover, otherwise `utf8mb4` is used.  \n- Display the minimum and maximum sizes estimations to handle variable data types.\n- Data sizes are calculated per column, per key, per table and per database.\n- Results are aggregated by database. If not specified, `unknown` name is used by default.\n\n\n### Usage\n\nBy default, it uses ASCII table to display the report.\nSize per row, and for 100 rows are calculated and displayed using 2 decimals.\n\n```\nds mysql [flags] [file.sql]\n```\n\nIt supports the following flags:\n\n* `-B`: batch mode, print results using comma as the column separator, with each row on a new line.\n* `-n`: number of lines to considerate by table (default 100).\n* `-p`: number of decimals to display (default 2).\n* `-v`: verbose output, produce more output about what the program does.\n\n\n## Installation\n\n### Go\n\n```shell\nGO111MODULE=on go get github.com/rvflash/ds@v0.1.0\n```\n\n### Docker\n\n```shell\ndocker run --rm -v $(pwd):/pkg rvflash/ds:v0.1.0 ds\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frvflash%2Fds","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frvflash%2Fds","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frvflash%2Fds/lists"}