{"id":14974053,"url":"https://github.com/noteflakes/pmts","last_synced_at":"2025-08-18T07:12:25.760Z","repository":{"id":66788290,"uuid":"115793342","full_name":"noteflakes/pmts","owner":"noteflakes","description":"Poor man's time series functionality for PostgreSQL","archived":false,"fork":false,"pushed_at":"2024-09-09T07:10:28.000Z","size":31,"stargazers_count":32,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-17T05:38:21.244Z","etag":null,"topics":["plpgsql","postgresql","sql","time-series"],"latest_commit_sha":null,"homepage":null,"language":"PLpgSQL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/noteflakes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-30T11:45:10.000Z","updated_at":"2024-11-25T08:35:31.000Z","dependencies_parsed_at":"2025-02-12T09:39:24.182Z","dependency_job_id":"8ed00753-bd3f-415a-ad57-1701085028f2","html_url":"https://github.com/noteflakes/pmts","commit_stats":null,"previous_names":["ciconia/pmts"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/noteflakes/pmts","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noteflakes%2Fpmts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noteflakes%2Fpmts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noteflakes%2Fpmts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noteflakes%2Fpmts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/noteflakes","download_url":"https://codeload.github.com/noteflakes/pmts/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/noteflakes%2Fpmts/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270957523,"owners_count":24675172,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["plpgsql","postgresql","sql","time-series"],"created_at":"2024-09-24T13:49:53.024Z","updated_at":"2025-08-18T07:12:25.742Z","avatar_url":"https://github.com/noteflakes.png","language":"PLpgSQL","readme":"# PMTS - Poor man's time series functionality for PostgreSQL\n\nPMTS is a collection of tools for working with time-series data in PostgreSQL\nwritten in SQL and PL/pgSQL, without needing to install extensions or work\nwith outside tools. Its features include:\n\n- Automatic partitioning (sharding) of time-series tables by time range.\n- Automatic dropping of old partitions according to data retention settings.\n- Aggregation and summarizing utilities (WIP).\n\nPMTS delivers some of the principal benefits of using tools such as TimeScaleDB\nor CitusDB on a stock PostgreSQL setup. PMTS has been employed successfuly in a\n~2TB production database with hundreds of tables and billions of records.\n\n\u003e **Note**: A new version of PMTS, using declarative table partitioning\n\u003e (supported on PostgreSQL 10.x+) is currently under development. The following\n\u003e documentation relates to PMTS 1.0, which works with PostgreSQL 9.x+. The\n\u003e source code for PMTS 1.0 is in [pmts.pg9.sql](pmts.pg9.sql).\n\n## Installing PMTS\n\n```bash\n$ git clone https://github.com/ciconia/pmts.git\n$ cd pmts\n$ psql -d mydb -f pmts.pg9.sql\n```\n\n## Getting Started\n\nAfter installing PMTS, create a time series table and start working with it:\n\n```SQL\n-- create our time-series table\ncreate table measurements (sensor text, stamp timestamptz, value numeric);\n\n-- setup partitioning with per-week partitioning, retaining data for one year.\n-- we also provide any index fields in order for PMTS to setup an index on\n-- relevant columns. in this case, PMTS will create an index on (sensor, stamp).\nselect pmts_setup_partitions('measurements', 86400 * 7, 86400 * 365, '{sensor}');\n\n-- continue to work normally with our table\ninsert into measurements values ('temp', now(), 12.345);\n\nselect stamp, value\nfrom measurements\nwhere sensor = 'temp' and stamp \u003e= now() - interval '1 month';\n```\n\nOnce partitioning is setup for your table, any inserts will automatically be\n*redirected* to the right partition, and partitions will be automatically\ncreated.\n\n## Removing old data\n\nPMTS makes it easy to drop old data by defining a retention period for each\npartitioned table. Instead of deleting old records, PMTS drops entire\npartitions, making the process much less error-prone, and also much faster. To\ndrop old data, just call the `pmts_drop_old_partitions()` function periodically\n(say, once a day):\n\n```sql\nselect pmts_drop_old_partitions();\n```\n\n## Tuning partition sizes\n\nPMTS also includes an API for automatically tuning partition sizes. The\n`pmts_tune_partition_size` looks at existing partitions and tunes the partition\ntime span so as to produce a certain byte size, within the given limits. You\nneed to provide three arguments: the desired byte size for partitions, the\nminimum and maximum time spans for partitions (in days), e.g.:\n\n```sql\n-- aim for 1GB partitions, with a minimum of 30 days and maximum of 365 days\n-- per partition.\nselect pmts_tune_partition_size(1000000000, 30, 365);\n```\n\n## API\n\nThe PMTS API is made of PL/pgSQL functions. Those functions can be called just\nlike any built-in function using `select`:\n\n```SQL\n[dbuser]=\u003e select pmts_setup_partitions(...);\n```\n\nYou can also invoke these functions from your shell:\n\n```bash\n$ psql -c \"select pmts_setup_partitions(...);\"\n```\n\n### pmts_setup_partitions(tbl_name, partition_size, retention_period, index_fields)\n\nSets up partitioning for the specified table. PMTS will use the supplied\narguments to control partition creation and deletion. Partitions are created on\nthe fly by using an insert trigger. The trigger will create partitions as needed\nand insert records into the correct partition.\n\n#### Arguments\n\nName|Type|Description\n----|----|-----------\ntbl_name|TEXT|Identifier of table to be partitioned\npartition_size|INTEGER|The partition size in seconds\nretention_period|INTEGER|The retention period in seconds\nindex_fields|TEXT[]|An array of fields to use for indexing\n\nPMTS partitions tables by time ranges. The `partition_size` argument controls\nthe size of each partitions in terms of time range. The `retention_period`\nargument is used to control how the amount of time to retain old data.\n\nThe `index_fields` argument is used to control the index created automatically\nby PMTS. By default, if no index fields are specified, PMTS will create an index\non the `stamp` field for each created partition. In most cases, though, A\ncompound index will be more useful, as usually data is filtered not only by time\nrange but also by one or more dimensions in other columns.\n\nFor example, consider the following scenario:\n\n```SQL\ncreate table measurements (unit text, metric text, stamp timestamptz, value numeric);\n```\n\nIn such a case, we'll usually query by unit, metric *and* stamp. We therefore\npass the relevant columns to PMTS:\n\n```SQL\nselect pmts_setup_partitions ('measurements', 86400 * 7, 86400 * 365, '{unit, metric}');\n```\n\nPMTS will then create an index on `(unit, metric, stamp)` for each partition.\n\n### pmts_drop_table(tbl_name)\n\nDrops a table that was previously partitioned with `pmts_setup_partitions`,\nremoving information about its partitions from the pmts tables. This will remove\nall partitions.\n\n#### Arguments\n\nName|Type|Description\n----|----|-----------\ntbl_name|TEXT|Table identifier\n\n### pmts_drop_old_partitions()\n\nDrops old partitions according to retention period specified for each table.\nThis function should be called periodically to remove old partitions. Use your\nfavorite tool to setup a recurring job that invokes the function.\n\n### pmts_total_size(tbl_name)\n\nReturns the sum of total relation size for all partitions of the specified table\nusing `pg_total_relation_size`.\n\n#### Arguments\n\nName|Type|Description\n----|----|-----------\ntbl_name|TEXT|Table identifier\n\n### pmts_info\n\nA view returning total size and number of partitions for each table managed by\nPMTS, with the following columns:\n\nName|Type|Description\n----|----|-----------\ntbl_name|TEXT|Table identifier\ntotal_size|NUMERIC|Total size of all table partitions in bytes\npartition_count|BIGINT|Number of existing partitions\navg_size|BIGINT|Average partition size in bytes\ncurrent_partition_size|INTEGER|Current partition size setting in seconds\n\nSample usage:\n\n```sql\nselect * from pmts_info where tbl_name = 'measurements';\n```\n\n### pmts_ideal_partition_size(desired_byte_size, current_byte_size, current_partition_size, min_days, max_days)\n\nReturns the ideal partition size in seconds for the given arguments.\n\n#### Arguments\n\nName|Type|Description\n----|----|-----------\ndesired_byte_size|BIGINT|The desired partition size in bytes\ncurrent_byte_size|BIGINT|The current average partition size in bytes\ncurrent_partition_size|BIGINT|The current partition size in seconds\nmin_days|INT|The minimum partition size in days (by default 7)\nmax_days|INT|The maximum partition size in days (by default 56)\n\n### pmts_tune_partition_size(desired_byte_size, min_days, max_days)\n\nAdjusts the partition size for all PMTS-managed tables according to the given\narguments. Note: this function should be invoked only seldomly in order for the\ncurrent average partition size (which is used to calculate the ideal partition\nsize in seconds) to faithfully reflect the current partition size settings.\n\n#### Arguments\n\nName|Type|Description\n----|----|-----------\ndesired_byte_size|BIGINT|The desired partition size in bytes\nmin_days|INT|The minimum partition size in days (by default 7)\nmax_days|INT|The maximum partition size in days (by default 56)\n\n## FAQ\n\n**Q:** What is a time series table?\n\n**A:** For PMTS, any table that includes a `stamp` column, of type `timestamp`\nor `timestamptz`.\n\n**Q:** Why should I partition (or shard) my time series tables?\n\n**A:** Partitioning a table allows you to keep your tables to a manageable size\nand maintain a good insertion rate. It also makes it easier to get rid of old\ndata by simply dropping partitions instead of deleting recores and then\nvacuuming.\n\n**Q:** How does partitioning work?\n\n**A:** For each partitioned table, PMTS maintains a list of partitions and\ninstalls an insert trigger that will insert each record into the correct\npartition according to the time stamp. Partition tables are created using\nPostgreSQL's table inheritance mechanism. That way, the query planner will\nautomatically include and exclude partition tables according to the time range\nspecified for the query.\n\n**Q:** What versions of PostgreSQL can be used with PMTS?\n\n**A:** PMTS has been developed and used on PostgreSQL 9.6. It will most\nprobably not work with anything before 9.0.\n\n**Q:** How do I install it?\n\n**A:** Simply download `pmts.pg9.sql` and load into your database using `psql`:\n\n    $ psql -f pmts.pg9.sql\n\n**Q:** Does PMTS work in AWS RDS?\n\n**A:** Yes.\n\n**Q:** Do I need to install anything besides PostgreSQL for PMTS to work?\n\n**A:** No.\n\n## Contributing\n\nPMTS can be improved in lots of ways - additional features, more customization,\nmore introspection tools... Please feel free to contribute ideas and code!","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnoteflakes%2Fpmts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnoteflakes%2Fpmts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnoteflakes%2Fpmts/lists"}