{"id":15111479,"url":"https://github.com/instaclustr/cassandra-sstable-tools","last_synced_at":"2025-04-05T16:07:37.303Z","repository":{"id":15361283,"uuid":"76549328","full_name":"instaclustr/cassandra-sstable-tools","owner":"instaclustr","description":"Tools for working with sstables","archived":false,"fork":false,"pushed_at":"2025-02-08T02:56:46.000Z","size":273,"stargazers_count":97,"open_issues_count":4,"forks_count":27,"subscribers_count":14,"default_branch":"cassandra-5.0","last_synced_at":"2025-03-29T15:06:37.364Z","etag":null,"topics":["cassandra","column-family","database","db","histogram","ic-sstable","instaclustr","keyspace","netapp-public","sstable","sstables","table","tool","tooling"],"latest_commit_sha":null,"homepage":"https://instaclustr.com","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/instaclustr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-12-15T10:19:12.000Z","updated_at":"2025-01-22T07:37:00.000Z","dependencies_parsed_at":"2024-02-02T05:24:27.547Z","dependency_job_id":"49ecd18e-2e6f-42c2-852a-427b8c8eba7e","html_url":"https://github.com/instaclustr/cassandra-sstable-tools","commit_stats":{"total_commits":52,"total_committers":7,"mean_commits":7.428571428571429,"dds":0.5576923076923077,"last_synced_commit":"f2d98e5e48315827ee4122ba13693da7a4c72320"},"previous_names":[],"tags_count":33,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instaclustr%2Fcassandra-sstable-tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instaclustr%2Fcassandra-sstable-tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instaclustr%2Fcassandra-sstable-tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instaclustr%2Fcassandra-sstable-tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/instaclustr","download_url":"https://codeload.github.com/instaclustr/cassandra-sstable-tools/tar.gz/refs/heads/cassandra-5.0","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247361687,"owners_count":20926643,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cassandra","column-family","database","db","histogram","ic-sstable","instaclustr","keyspace","netapp-public","sstable","sstables","table","tool","tooling"],"created_at":"2024-09-26T00:20:24.666Z","updated_at":"2025-04-05T16:07:37.249Z","avatar_url":"https://github.com/instaclustr.png","language":"Java","funding_links":[],"categories":["Packages"],"sub_categories":["Tools"],"readme":"# Instaclustr SSTable tools\n\n[![Maven Badge](https://img.shields.io/maven-central/v/com.instaclustr/ic-sstable-tools-4.1.0.svg?label=Maven%20Central)](https://search.maven.org/search?q=g:%22com.instaclustr%22%20AND%20a:%22ic-sstable-tools-4.1.0%22)\n[![Circle CI](https://circleci.com/gh/instaclustr/cassandra-sstable-tools.svg?style=svg)](https://circleci.com/gh/instaclustr/cassandra-sstable-tools)\n\nhttps://www.instaclustr.com/update-released-instaclustr-sstable-analysis-tools-apache-cassandra/\nhttps://www.instaclustr.com/instaclustr-open-sources-cassandra-sstable-analysis-tools/\n\n## Compile\n```\n$ git clone git@github.com:instaclustr/cassandra-sstable-tools.git\n$ cd cassandra-sstable-tools\n# Select the correct branch for major version (default is cassandra-4.1)\n$ git checkout cassandra-4.1\n```\n\n```\n$ mvn clean install\n```\n\n## Install\nCopy `ic-sstable-tools.jar` to Cassandra JAR folder, eg. `/usr/share/cassandra/lib`\n\nCopy the `bin/ic-sstable-tools` script into your `$PATH`\n\nWe also offer RPM and DEB packages. In case you install them, you do not need to execute steps above, obviously.\n\n## Documentation\n\n```\n$ ./bin/ic-sstable-tools \nMissing required sub-command.\nUsage: \u003cmain class\u003e [-hV] [COMMAND]\n -h, --help Show this help message and exit.\n -V, --version Print version information and exit.\nCommands:\n cfstats Detailed statistics about cells in a column family\n pstats Partition size statistics for a column family\n purge Statistics about reclaimable data for a column family\n sstables Print out metadata for sstables that belong to a column family\n summary Summary information about all column families including how much of the data is repaired\n```\n\nIf you want invoke help command for each subcommand, do it like:\n\n```\n$ ./bin/ic-sstable-tools help cfstats\n```\n\n## summary ##\nProvides summary information about all column families. Useful for finding\nthe largest column families and how much data has been repaired by incremental repairs.\n\n### Usage ###\n\n ic-sstable-tools summary\n\n### Output ###\n| Column | Description |\n|---------------|-------------------------------------------------------------|\n| Keyspace | Keyspace the column family belongs to |\n| Column Family | Name of column family |\n| SSTables | Number of sstables on this node for the column family |\n| Disk Size | Compressed size on disk for this node |\n| Data Size | Uncompressed size of the data for this node |\n| Last Repaired | Maximum repair timestamp on sstables |\n| Repair % | Percentage of data marked as repaired |\n\n## sstables ##\nPrint out sstable metadata for a column family. Useful in helping to tune compaction settings.\n\n### Usage ###\n\n ic-sstable-tools sstables \u003ckeyspace\u003e \u003ccolumn-family\u003e\n\n### Output ###\n| Column | Description |\n|--------------------|-----------------------------------------------------------|\n| SSTable | Data.db filename of sstable |\n| Disk Size | Size of sstable on disk |\n| Total Size | Uncompressed size of data contained in the sstable |\n| Min Timestamp | Minimum cell timestamp contained in the sstable |\n| Max Timestamp | Maximum cell timestamp contained in the sstable |\n| Duration | The time span between minimum and maximum cell timestamps |\n| Min Deletion Time | The minimum deletion time |\n| Max Deletion Time | The maximum deletion time |\n| Level | Leveled Tiered Compaction sstable level |\n| Keys | Number of partition keys |\n| Avg Partition Size | Average partition size |\n| Max Partition Size | Maximum partition size |\n| Avg Column Count | Average number of columns in a partition |\n| Max Column Count | Maximum number of columns in a partition |\n| Droppable | Estimated droppable tombstones |\n| Repaired At | Time when marked as repaired by incremental repair |\n\n\n## pstats ##\nTool for finding largest partitions. Reads the Index.db files so is relatively quick.\n\n### Usage ###\n\n ic-sstable-tools pstats [-n \u003cnum\u003e] [-t \u003csnapshot\u003e] [-f \u003cfilter\u003e] \u003ckeyspace\u003e \u003ccolumn-family\u003e\n\n| -h | Display help |\n|------------|---------------------------------------------------------------------------------|\n| -b | Batch mode. Uses progress indicator that is friendly for running in batch jobs. |\n| -n \u003cnum\u003e | Number of partitions to display |\n| -t \u003cname\u003e | Snapshot to analyse. Snapshot is created if none is specified. |\n| -f \u003cfiles\u003e | Comma separated list of Data.db sstables to filter on |\n\n### Output ###\nSummary: Summary statistics about partitions\n\n| Column | Description |\n|------------------------------------------------------------------------|----------------------------------------------------------|\n| Count (Size) | Number of partition keys on this node |\n| Total (Size) | Total uncompressed size of all partitions on this node |\n| Total (SSTable) | Number of sstables on this node |\n| Minimum (Size) | Minimum uncompressed partition size |\n| Minimum (SSTable) | Minimum number of sstables a partition belongs to |\n| Average (Size) | Average (mean) uncompressed partition size |\n| Average (SSTable) | Average (mean) number of sstables a partition belongs to |\n| std dev. (Size) | Standard deviation of partition sizes |\n| std dev. (SSTable) | Standard deviation of number of sstables for a partition |\n| 50% (Size) | Estimated 50th percentile of partition sizes |\n| 50% (SSTable) | Estimated 50th percentile of sstables for a partition |\n| 75% (Size) | Estimated 75th percentile of partition sizes |\n| 75% (SSTable) | Estimated 75th percentile of sstables for a partition |\n| 90% (Size) | Estimated 90th percentile of partition sizes |\n| 90% (SSTable) | Estimated 90th percentile of sstables for a partition |\n| 95% (Size) | Estimated 95th percentile of partition sizes |\n| 95% (SSTable) | Estimated 95th percentile of sstables for a partition |\n| 99% (Size) | Estimated 99th percentile of partition sizes |\n| 99% (SSTable) | Estimated 99th percentile of sstables for a partition |\n| 99.9% (Size) | Estimated 99.9th percentile of partition sizes |\n| 99.9% (SSTable) | Estimated 99.9th percentile of sstables for a partition |\n| Maximum (Size) | Maximum uncompressed partition size |\n| Maximum (SSTable) | Maximum number of sstables a partition belongs to |\n\nLargest partitions: The top N largest partitions\n\n| Column | Description |\n|------------------------------------------------------------------------|----------------------------------------------------------|\n| Key | The partition key |\n| Size | Total uncompressed size of the partition |\n| SSTable Count | Number of sstables that contain the partition |\n\nSSTable Leaders: The top N partitions that belong to the most sstables\n\n| Column | Description |\n|------------------------------------------------------------------------|----------------------------------------------------------|\n| Key | The partition key |\n| SSTable Count | Number of sstables that contain the partition |\n| Size | Total uncompressed size of the partition |\n\nSSTables: Metadata about sstables as it relates to partitions.\n\n| Column | Description |\n|------------------------------------------------------------------------|----------------------------------------------------------|\n| SSTable | Data.db filename of SSTable |\n| Size | Uncompressed size |\n| Min Timestamp | Minimum cell timestamp in the sstable |\n| Max Timestamp | Maximum cell timestamp in the sstable |\n| Level | Leveled Tiered Compaction level of sstable |\n| Partitions | Number of partition keys in the sstable |\n| Avg Partition Size | Average uncompressed partition size in sstable |\n| Max Partition Size | Maximum uncompressed partition size in sstable |\n\n## cfstats ##\nTool for getting detailed cell statistics that can help identify issues with data model.\n\n### Usage ###\n\n ic-sstable-tools cfstats [-n \u003cnum\u003e] [-t \u003csnapshot\u003e] [-f \u003cfilter\u003e] \u003ckeyspace\u003e \u003ccolumn-family\u003e\n| -h | Display help |\n|------------|---------------------------------------------------------------------------------|\n| -b | Batch mode. Uses progress indicator that is friendly for running in batch jobs. | |\n| -n \u003cnum\u003e | Number of partitions to display |\n| -t \u003cname\u003e | Snapshot to analyse. Snapshot is created if none is specified. |\n| -f \u003cfiles\u003e | Comma separated list of Data.db sstables to filter on |\n\n### Output ###\nSummary: Summary statistics about partitions\n\n| Column | Description |\n|----------------------------------------------|----------------------------------------------------------|\n| Count (Size) | Number of partition keys on this node |\n| Rows (Size) | Number of clustering rows |\n| (deleted) | Number of clustering row deletions |\n| Total (Size) | Total uncompressed size of all partitions on this node |\n| Total (SSTable) | Number of sstables on this node |\n| Minimum (Size) | Minimum uncompressed partition size |\n| Minimum (SSTable) | Minimum number of sstables a partition belongs to |\n| Average (Size) | Average (mean) uncompressed partition size |\n| Average (SSTable) | Average (mean) number of sstables a partition belongs to |\n| std dev. (Size) | Standard deviation of partition sizes |\n| std dev. (SSTable) | Standard deviation of number of sstables for a partition |\n| 50% (Size) | Estimated 50th percentile of partition sizes |\n| 50% (SSTable) | Estimated 50th percentile of sstables for a partition |\n| 75% (Size) | Estimated 75th percentile of partition sizes |\n| 75% (SSTable) | Estimated 75th percentile of sstables for a partition |\n| 90% (Size) | Estimated 90th percentile of partition sizes |\n| 90% (SSTable) | Estimated 90th percentile of sstables for a partition |\n| 95% (Size) | Estimated 95th percentile of partition sizes |\n| 95% (SSTable) | Estimated 95th percentile of sstables for a partition |\n| 99% (Size) | Estimated 99th percentile of partition sizes |\n| 99% (SSTable) | Estimated 99th percentile of sstables for a partition |\n| 99.9% (Size) | Estimated 99.9th percentile of partition sizes |\n| 99.9% (SSTable) | Estimated 99.9th percentile of sstables for a partition |\n| Maximum (Size) | Maximum uncompressed partition size |\n| Maximum (SSTable) | Maximum number of sstables a partition belongs to |\n\nRow Histogram: Histogram of number of rows per partition\n\n| Column | Description |\n|----------------------------------------------|----------------------------------------------------------------------|\n| Percentile | Minimum, average, standard deviation (std dev.), percentile, maximum | |\n| Count | Estimated number of rows per partition for the given percentile | \n\nLargest partitions: Partitions with largest uncompressed size \n\n| Column | Description |\n|---------------------------------------------------------------|------------------------------------------------------------------|\n| Key | The partition key |\n| Size | Total uncompressed size of the partition |\n| Rows | Total number of clustering rows in the partition |\n| (deleted) | Number of row deletions in the partition |\n| Tombstones | Number of cell or range tombstones |\n| (droppable) | Number of tombstones that can be dropped as per gc_grace_seconds |\n| Cells | Number of cells in the partition |\n| SSTable Count | Number of sstables that contain the partition |\n\nWidest partitions: Partitions with the most cells\n\n| Column | Description |\n|---------------------------------------------------|------------------------------------------------------------------|\n| Key | The partition key |\n| Rows | Total number of clustering rows in the partition |\n| (deleted) | Number of row deletions in the partition |\n| Cells | Number of cells in the partition |\n| Tombstones | Number of cell or range tombstones |\n| (droppable) | Number of tombstones that can be dropped as per gc_grace_seconds |\n| Size | Total uncompressed size of the partition |\n| SSTable Count | Number of sstables that contain the partition |\n\nMost Deleted Rows: Partitions with the most row deletions\n\n| Column | Description |\n|---------------------------------------------------|------------------------------------------------------------------|\n| Key | The partition key |\n| Rows | Total number of clustering rows in the partition |\n| (deleted) | Number of row deletions in the partition |\n| Size | Total uncompressed size of the partition |\n| SSTable Count | Number of sstables that contain the partition |\n\n\nTombstone Leaders: Partitions with the most tombstones\n\n| Column | Description |\n|--------------------------------------------------------|------------------------------------------------------------------|\n| Key | The partition key |\n| Tombstones | Number of cell or range tombstones |\n| (droppable) | Number of tombstones that can be dropped as per gc_grace_seconds |\n| Rows | Total number of clustering rows in the partition |\n| Cells | Number of cells in the partition |\n| Size | Total uncompressed size of the partition |\n| SSTable Count | Number of sstables that contain the partition |\n\nSSTable Leaders: Partitions that are in the most sstables\n\n| Column | Description |\n|-----------------------------------------------------------|------------------------------------------------------------------|\n| Key | The partition key |\n| SSTable Count | Number of sstables that contain the partition |\n| Size | Total uncompressed size of the partition |\n| Rows | Total number of clustering rows in the partition |\n| Cells | Number of cells in the partition |\n| Tombstones | Number of cell or range tombstones |\n| (droppable) | Number of tombstones that can be dropped as per gc_grace_seconds |\n\nSSTables: Metadata about sstables as it relates to partitions. \n\n| Column | Description |\n|----------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| SSTable | Data.db filename of SSTable |\n| Size | Uncompressed size |\n| Min Timestamp | Minimum cell timestamp in the sstable |\n| Max Timestamp | Maximum cell timestamp in the sstable |\n| Partitions | Number of partitions |\n| (deleted) | Number of row level partition deletions |\n| (avg size) | Average uncompressed partition size in sstable |\n| (max size) | Maximum uncompressed partition size in sstable |\n| Rows | Total number of clustering rows in sstable |\n| (deleted) | Number of row deletions in sstable |\n| Cells | Number of cells in the SSTable |\n| Tombstones | Number of cell or range tombstones in the SSTable |\n| (droppable) | Number of tombstones that are droppable according to gc_grace_seconds |\n| (range) | Number of range tombstones |\n| Cell Liveness | Percentage of live cells. Does not consider tombstones or cell updates shadowing cells. That is it is percentage of non-tombstoned cells to total number of cells. |\n\n## purge ##\nFinds the largest reclaimable partitions (GCable). Intensive process, effectively does \"fake\" compactions to calculate metrics.\n\n### Usage ###\n\n ic-sstable-tools purge [-n \u003cnum\u003e] [-t \u003csnapshot\u003e] [-f \u003cfilter\u003e] \u003ckeyspace\u003e \u003ccolumn-family\u003e\n\n| -h | Display help |\n|------------|---------------------------------------------------------------------------------|\n| -b | Batch mode. Uses progress indicator that is friendly for running in batch jobs. |\n| -n \u003cnum\u003e | Number of partitions to display |\n| -t \u003cname\u003e | Snapshot to analyse. Snapshot is created if none is specified. |\n\n### Output ###\nLargest reclaimable partitions: Partitions with the largest amount of reclaimable data\n\n| Column | Description |\n|----------------------------------------------------------------------------------------|----------------------------------------------|\n| Key | The partition key |\n| Size | Total uncompressed size of the partition |\n| Reclaim | Reclaimable uncompressed size |\n| Generations | SSTable generations the partition belongs to |\n\n\nPlease see https://www.instaclustr.com/support/documentation/announcements/instaclustr-open-source-project-status/ for Instaclustr support status of this project\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finstaclustr%2Fcassandra-sstable-tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finstaclustr%2Fcassandra-sstable-tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finstaclustr%2Fcassandra-sstable-tools/lists"}