{"id":16713975,"url":"https://github.com/tktech/ckanext-cloudstorage","last_synced_at":"2025-08-10T10:25:04.800Z","repository":{"id":46475431,"uuid":"60545266","full_name":"TkTech/ckanext-cloudstorage","owner":"TkTech","description":"Implements support for resource storage against multiple popular providers via apache-libcloud (S3, Azure Storage, etc...)","archived":false,"fork":false,"pushed_at":"2024-02-01T08:28:52.000Z","size":102,"stargazers_count":35,"open_issues_count":23,"forks_count":54,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-05-22T13:33:28.337Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TkTech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":null,"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":"TkTech","issuehunt":null,"otechie":null,"custom":null}},"created_at":"2016-06-06T16:57:42.000Z","updated_at":"2024-08-03T05:00:59.225Z","dependencies_parsed_at":"2024-08-03T05:00:57.990Z","dependency_job_id":"8af4ac76-4027-4f4d-8d29-05a76a0860c1","html_url":"https://github.com/TkTech/ckanext-cloudstorage","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TkTech%2Fckanext-cloudstorage","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TkTech%2Fckanext-cloudstorage/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TkTech%2Fckanext-cloudstorage/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TkTech%2Fckanext-cloudstorage/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TkTech","download_url":"https://codeload.github.com/TkTech/ckanext-cloudstorage/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232949833,"owners_count":18601385,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-12T20:48:42.841Z","updated_at":"2025-01-07T22:40:19.131Z","avatar_url":"https://github.com/TkTech.png","language":"Python","funding_links":["https://liberapay.com/TkTech"],"categories":[],"sub_categories":[],"readme":"# ckanext-cloudstorage\n\nImplements support for using S3, Azure, or any of 15 different storage\nproviders supported by [libcloud][] to [CKAN][].\n\n# Setup\n\nAfter installing `ckanext-cloudstorage`, add it to your list of plugins in\nyour `.ini`:\n\n    ckan.plugins = stats cloudstorage\n\nIf you haven't already, setup [CKAN file storage][ckanstorage] or the file\nupload button will not appear.\n\nEvery driver takes two options, regardless of which one you use. Both\nthe name of the driver and the name of the container/bucket are\ncase-sensitive:\n\n    ckanext.cloudstorage.driver = AZURE_BLOBS\n    ckanext.cloudstorage.container_name = demo\n\nYou can find a list of driver names [here][storage] (see the `Provider\nConstant` column.)\n\nEach driver takes its own setup options. See the [libcloud][] documentation.\nThese options are passed in using `driver_options`, which is a Python dict.\nFor most drivers, this is all you need:\n\n    ckanext.cloudstorage.driver_options = {\"key\": \"\u003cyour public key\u003e\", \"secret\": \"\u003cyour secret key\u003e\"}\n\n# Support\n\nMost libcloud-based providers should work out of the box, but only those listed\nbelow have been tested:\n\n| Provider | Uploads | Downloads | Secure URLs (private resources) |\n| --- | --- | --- | --- |\n| Azure    | YES | YES | YES (if `azure-storage` is installed) |\n| AWS S3   | YES | YES | YES (if `boto` is installed) |\n| Rackspace | YES | YES | No |\n\n# What are \"Secure URLs\"?\n\n\"Secure URLs\" are a method of preventing access to private resources. By\ndefault, anyone that figures out the URL to your resource on your storage\nprovider can download it. Secure URLs allow you to disable public access and\ninstead let ckanext-cloudstorage generate temporary, one-use URLs to download\nthe resource. This means that the normal CKAN-provided access restrictions can\napply to resources with no further effort on your part, but still get all the\nbenefits of your CDN/blob storage.\n\n    ckanext.cloudstorage.use_secure_urls = 1\n\nThis option also enables multipart uploads, but you need to create database tables\nfirst. Run next command from extension folder:\n    `paster cloudstorage initdb -c /etc/ckan/default/production.ini `\n\nWith that feature you can use `cloudstorage_clean_multipart` action, which is available\nonly for sysadmins. After executing, all unfinished multipart uploads, older than 7 days,\nwill be aborted. You can configure this lifetime, example:\n\n     ckanext.cloudstorage.max_multipart_lifetime  = 7\n\n# Migrating From FileStorage\n\nIf you already have resources that have been uploaded and saved using CKAN's\nbuilt-in FileStorage, cloudstorage provides an easy migration command.\nSimply setup cloudstorage as explained above, enable the plugin, and run the\nmigrate command. Provide the path to your resources on-disk (the\n`ckan.storage_path` setting in your CKAN `.ini` + `/resources`), and\ncloudstorage will take care of the rest. Ex:\n\n    paster cloudstorage migrate \u003cpath to files\u003e -c ../ckan/development.ini\n\n# Notes\n\n1. You should disable public listing on the cloud service provider you're\n   using, if supported.\n2. Currently, only resources are supported. This means that things like group\n   and organization images still use CKAN's local file storage.\n\n# FAQ\n\n- *DataViews aren't showing my data!* - did you setup CORS rules properly on\n  your hosting service? ckanext-cloudstorage can try to fix them for you automatically,\n  run:\n\n        paster cloudstorage fix-cors \u003clist of your domains\u003e -c=\u003cCKAN config\u003e\n\n- *Help! I can't seem to get it working!* - send me a mail! tk@tkte.ch\n\n[libcloud]: https://libcloud.apache.org/\n[ckan]: http://ckan.org/\n[storage]: https://libcloud.readthedocs.io/en/latest/storage/supported_providers.html\n[ckanstorage]: http://docs.ckan.org/en/latest/maintaining/filestore.html#setup-file-uploads\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftktech%2Fckanext-cloudstorage","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftktech%2Fckanext-cloudstorage","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftktech%2Fckanext-cloudstorage/lists"}