{"id":20017702,"url":"https://github.com/foomo/dumpster","last_synced_at":"2025-05-04T22:32:07.916Z","repository":{"id":36412333,"uuid":"40717284","full_name":"foomo/dumpster","owner":"foomo","description":"dump project data and restore them","archived":true,"fork":false,"pushed_at":"2016-02-22T22:02:11.000Z","size":28,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-03-02T03:16:18.407Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/foomo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-08-14T13:51:16.000Z","updated_at":"2023-05-07T14:22:59.000Z","dependencies_parsed_at":"2022-09-25T00:24:53.775Z","dependency_job_id":null,"html_url":"https://github.com/foomo/dumpster","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foomo%2Fdumpster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foomo%2Fdumpster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foomo%2Fdumpster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foomo%2Fdumpster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/foomo","download_url":"https://codeload.github.com/foomo/dumpster/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252408519,"owners_count":21743134,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T08:18:06.892Z","updated_at":"2025-05-04T22:32:07.674Z","avatar_url":"https://github.com/foomo.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Dumpster\n\n_Dump project data and restore them_\n\n![Image dumpster from wikipedia](https://upload.wikimedia.org/wikipedia/commons/thumb/5/55/Dumpsters_in_Anaktuvuk_Pass%2C_Alaska.jpg/440px-Dumpsters_in_Anaktuvuk_Pass%2C_Alaska.jpg)\n\n\u003csmall\u003eimage \u0026copy; [Paxson Woelber](https://en.wikipedia.org/wiki/Dumpster#/media/File:Dumpsters_in_Anaktuvuk_Pass,_Alaska.jpg)\u003c/small\u003e\n\nDumpster allows you to define dumps. **A dump is genrated by a program**, that\nwill write the dump into a file and another program that will be able to\n**restore** the previously created dump from that file.\n\nA simple **RESTful service** allows you to create, access, restore and\ndelete those dumps.\n\nThings become very interesting with remotes. **Remotes allow you to restore remote\ndumps locally**. Common use cases are backups and development.\n\n## Security\n\nDumpster supports basic auth - that is it (only use over https!). You have to be very careful on how to make it accessible and **with which privileges you will let it run**.\n\n### A few notes on basic auth\n\nDo not use basic auth over http it is not secure!\n\nHow to create a htpasswd file\n\n```bash\n# create a new htpasswd file with bcrypt as the hashing algo and add user mr foo\nhtpasswd -c -B /path/to/file mister-foo\n```\n\nSince this program is using Lev Shamardins go-http-auth https://github.com/abbot/go-http-auth it will check, if the htpasswd file has changed.\n\n## Dump and restore programs\n\nSince we do not (want to) know how to dump or restore your data let us define\na very simple contract for dumping and restoring:\n\n```bash\n# DUMPING\n#   program [as many args as you like] but the last arg is the file to dump to\n/path/to/dump-program arg1 arg2 ... /path/to/dump/file\n\n# RESTORING\n#   same rules as with the dump - last arg is the dumpfile\n/path/to/restore-program arg1 arg2 ... /path/to/dump/file\n\n```\n\nYou are cordially invited to use stdout and stderr to provide information on\nyour dump or restore. As long as you exit with code 0 and write the data\nsynchronously to the filename passed as last arg dumpster will be happy.\n\nYour output to stdout and stderr will be recorded and presented to the user.\n\nTypically your dump programs will be shell scripts on a nix system.\n\n## How to configure it\n\nDumpsters config is a yaml file:\n\n```yaml\n# this is where I listen and provide a REST interface\nhttp:\n  # create a new htpasswd file and use bcrypt as the hashing mechanism\n  # htpassd -c -B /path/to/basic/auth/file \u003cusername\u003e\n  basicauthfile: /path/to/basic/auth/file\n  # http for debugging do not use in production\n  address: 127.0.0.1:8080\n  tls:\n    address: www.my-dumpster.com:8443\n    cert: /path/to/cert.file\n    key: /path/to/key/file\n# a directory to put my data, those are plain files in a simple data structure\ndatadir: /var/lib/dumpster\n# these are my dumps, which is what I am all about\ndumps:\n  # this is a dump config and it defines a dump called dumpster\n  dumpster:\n    # dump defines what program should be called to create a dump\n    dump:\n      # this example program will simply tar the directory passed as $1\n      program: /home/jan/go/src/github.com/foomo/dumpster/examples/files/dump.sh\n      # with some args\n      args:\n        - /home/jan/go/src/github.com/foomo/dumpster\n    # restore defines a program, which has to restore a previously created dump\n    restore:\n      program: tar\n      args:\n        - -tzvf\n  # this is another dump\n  projectx:\n    # only the dump part is defined, thus restore can not be called\n    dump:\n      program: /home/jan/go/src/github.com/foomo/dumpster/examples/files/dump.sh\n      args:\n        - /home/jan/go/src/github.com/foomo/projectx\n# remotes are pointing to other dumpster instances\nremotes:\n    myself:\n        endpoint: http://127.0.0.1:8080\n\n```\n\n## REST interface\n\nThere is a fairly extensive documentation [http://docs.dumpster.apiary.io/](http://docs.dumpster.apiary.io/)\n\nA little curl guide\n\n```bash\n# list dumps\ncurl 127.0.0.1:8080/dump\n\n# list dumps of type \"dumpster\"\ncurl 127.0.0.1:8080/dump/dumpster\n\n# create a dump of type \"dumpster\" with id \"my-first-dump\"\ncurl -X POST -d '{ \"id\" : \"my-first-dump\", \"comment\" : \"what a great day ...\" }' 127.0.0.1:8080/dump/dumpster\n\n# restore a dump of type \"dumpster\" with id \"my-first-dump\"\ncurl -X POST 127.0.0.1:8080/restore/dumpster/my-first-dump\n\n# list all remote dumps\ncurl 127.0.0.1:8080/dumpremote\n\n# list remote dumps on server \"myself\"\ncurl 127.0.0.1:8080/dumpremote/myself\n\n# restore a remote dump from server \"myself\" of type \"dumpster\" and id \"my-first-dump\"\ncurl -X POST 127.0.0.1:8080/restoreremote/myself/dumpster/my-first-dump\n\n```\n\n## TODO\n\n- add tests\n- daemonize\n- make some debian... packages\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoomo%2Fdumpster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffoomo%2Fdumpster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoomo%2Fdumpster/lists"}