{"id":28396331,"url":"https://github.com/ssbc/ssb-blobs-purge","last_synced_at":"2025-07-07T06:39:00.227Z","repository":{"id":54815129,"uuid":"243811671","full_name":"ssbc/ssb-blobs-purge","owner":"ssbc","description":"SSB plugin to automatically remove old and large blobs","archived":false,"fork":false,"pushed_at":"2021-11-04T08:03:16.000Z","size":36,"stargazers_count":6,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-09T15:38:11.818Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ssbc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-28T16:54:37.000Z","updated_at":"2022-05-16T08:47:47.000Z","dependencies_parsed_at":"2022-08-14T03:40:33.196Z","dependency_job_id":null,"html_url":"https://github.com/ssbc/ssb-blobs-purge","commit_stats":null,"previous_names":["staltz/ssb-blobs-purge"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/ssbc/ssb-blobs-purge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbc%2Fssb-blobs-purge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbc%2Fssb-blobs-purge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbc%2Fssb-blobs-purge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbc%2Fssb-blobs-purge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ssbc","download_url":"https://codeload.github.com/ssbc/ssb-blobs-purge/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbc%2Fssb-blobs-purge/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260085147,"owners_count":22956562,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-31T21:38:06.507Z","updated_at":"2025-07-07T06:39:00.221Z","avatar_url":"https://github.com/ssbc.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ssb-blobs-purge\n\nSSB plugin to automatically remove old and large blobs.\n\nWorks by recursively traversing the blobs folder, and prioritizing deletes of blobs that are old (small creation timestamp) and big (large file size), or both. Does not delete blobs that your own feed posted (or mentioned first). Stops deletion once the `storageLimit` target is reached.\n\n**Note!** This deletion process creates a bias against old content, and this may not be the best choice for every SSB app. For instance, in some cases it may make sense to favor blobs in a particular hashtag (\"channel\") over other hashtags, regardless of how old or large they are. The choices in this module also favor blobs that were mentioned by the local `ssb.id` feed, and does not consider 'same as' accounts. The heuristic used to pick the next deletable blob also favors implementation performance, instead of picking the best possible deletable blob (see section at the bottom of this readme).\n\n## Usage\n\n**Prerequisites:**\n\n- **Node.js 6.5** or higher\n- `secret-stack@^6.2.0`\n- `ssb-blobs` installed\n- Either of these two (preferably not both!):\n  - `ssb-db2` (plus `ssb-db2/full-mentions`) installed\n  - `ssb-backlinks` installed\n\n```\nnpm install --save ssb-blobs-purge\n```\n\nAdd this plugin to ssb-server like this:\n\n```diff\n var createSsbServer = require('ssb-server')\n     .use(require('ssb-onion'))\n     .use(require('ssb-unix-socket'))\n     .use(require('ssb-no-auth'))\n     .use(require('ssb-plugins'))\n     .use(require('ssb-master'))\n     .use(require('ssb-db2'))\n     .use(require('ssb-db2/full-mentions'))\n     .use(require('ssb-conn'))\n     .use(require('ssb-blobs'))\n+    .use(require('ssb-blobs-purge'))\n     .use(require('ssb-replicate'))\n     .use(require('ssb-friends'))\n     // ...\n```\n\nNow you should be able to access the following muxrpc APIs under `ssb.blobsPurge.*`:\n\n| API | Type | Description |\n|-----|------|-------------|\n| **`start(opts?)`** | `sync` | Triggers the start of purge task. Optionally, you may pass an object with two fields: `cpuMax` and `storageLimit` |\n| **`stop()`** | `sync` | Stops the purge task if it is currently active. |\n| **`changes()`** | `source` | A pull-stream that emits events notifying of progress done by the purge task, these events are objects as described below. |\n\nA `changes()` event object is one these three possibile types:\n\n- `{ event: 'deleted', blobId }`\n- `{ event: 'paused' }`\n- `{ event: 'resumed' }`\n\n## Setting the parameters\n\nThere are two parameters you can tweak with this module:\n\n- `cpuMax`: **(default 50)** a number between `0` and `100` that represents a threshold for CPU usage; when your device's CPU usage is *below* this percentual number, the purge task will run, otherwise it will be paused\n- `storageLimit`: **(default 10 gigabytes)** a number (in bytes) that represents the desired maximum storage for blobs; when your blobs folder is *larger* than this number, the purge task will run, otherwise it will be paused\n\nThere are two ways you can set these parameters:\n\n**(1) In the SSB config object**:\n\n```diff\n {\n   path: ssbPath,\n   keys: keys,\n   port: 8008,\n   blobs: {\n     sympathy: 2\n   },\n+  blobsPurge: {\n+    cpuMax: 30,\n+    storageLimit: 2000000000\n+  },\n   conn: {\n     autostart: false\n   }\n }\n```\n\n**(2) When calling the `start` function**, note that these numbers are **ignored** in case you supply (1) above:\n\n```javascript\nssb.blobsPurge.start({ cpuMax: 30, storageLimit: 2000000000 })\n```\n\n## How does this work?\n\nFor each blob, we calculate a heuristic:\n\n```\nheuristic = blobSizeInKb * ageInDays\n```\n\nThen, instead of sorting the blobs in descending heuristic order (that would be `O(n log n)`), we do at most 6 linear passes (hence `O(6 n) = O(n)`) over the blobs set:\n\n- On the first pass, we delete any blob whose heuristic number is greater than `1e7`\n- On the second pass, we delete any blob whose heuristic number is greater than `1e6`\n- On the third pass, we delete any blob whose heuristic number is greater than `1e5`\n- On the fourth pass, we delete any blob whose heuristic number is greater than `1e4`\n- On the fifth pass, we delete any blob whose heuristic number is greater than `1e3`\n- On the sixth pass, we delete any blob\n\nAs soon as the `storageLimit` is met, we immediately cancel the current pass, and begin waiting for new blobs to be added. Once there are sufficiently new blobs, we resume the 6 passes.\n\nThere is one exception: blobs that your own account has mentioned in posts are considered \"yours\", and are never deleted.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fssbc%2Fssb-blobs-purge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fssbc%2Fssb-blobs-purge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fssbc%2Fssb-blobs-purge/lists"}