{"id":13567245,"url":"https://github.com/Kinto/kinto-attachment","last_synced_at":"2025-04-04T01:31:29.608Z","repository":{"id":37431834,"uuid":"45928944","full_name":"Kinto/kinto-attachment","owner":"Kinto","description":"Attach files to Kinto records","archived":false,"fork":false,"pushed_at":"2025-03-25T09:57:19.000Z","size":903,"stargazers_count":42,"open_issues_count":15,"forks_count":7,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-30T00:11:03.595Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://kinto.github.io/kinto-attachment/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kinto.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-11-10T17:37:16.000Z","updated_at":"2025-03-25T09:57:23.000Z","dependencies_parsed_at":"2023-09-27T16:21:24.899Z","dependency_job_id":"31f92e29-46bb-48a4-a718-f7b50383e834","html_url":"https://github.com/Kinto/kinto-attachment","commit_stats":{"total_commits":661,"total_committers":17,"mean_commits":38.88235294117647,"dds":0.7291981845688351,"last_synced_commit":"ed4c79f2829125c4a6dd90cbdefaaea98ad035fb"},"previous_names":[],"tags_count":35,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kinto%2Fkinto-attachment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kinto%2Fkinto-attachment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kinto%2Fkinto-attachment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kinto%2Fkinto-attachment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kinto","download_url":"https://codeload.github.com/Kinto/kinto-attachment/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247107816,"owners_count":20884793,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T13:02:26.706Z","updated_at":"2025-04-04T01:31:25.703Z","avatar_url":"https://github.com/Kinto.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"================\nKinto Attachment\n================\n\n.. image:: https://github.com/Kinto/kinto-attachment/actions/workflows/test.yml/badge.svg\n        :target: https://github.com/Kinto/kinto-attachment/actions\n\n.. image:: https://img.shields.io/pypi/v/kinto-attachment.svg\n        :target: https://pypi.python.org/pypi/kinto-attachment\n\nAttach files to `Kinto records \u003chttp://kinto.readthedocs.io\u003e`_.\n\n\nInstall\n=======\n\n::\n\n    pip install kinto-attachment\n\n\nSetup\n=====\n\nIn the Kinto project settings\n\n.. code-block:: ini\n\n    kinto.includes = kinto_attachment\n    kinto.attachment.base_url = http://cdn.service.org/files/\n\n\nLocal File storage\n------------------\n\nStore files locally:\n\n.. code-block:: ini\n\n    kinto.attachment.base_path = /tmp\n\n\nS3 File Storage\n---------------\n\nStore on Amazon S3:\n\n.. code-block:: ini\n\n    kinto.attachment.aws.access_key = \u003cAWS access key\u003e\n    kinto.attachment.aws.secret_key = \u003cAWS secret key\u003e\n    kinto.attachment.aws.bucket_name = \u003cbucket name\u003e\n    kinto.attachment.aws.acl = \u003cAWS ACL permissions|public-read\u003e\n\n\n.. note::\n\n    ``access_key`` and ``secret_key`` may be omitted when using AWS Identity\n    and Access Management (IAM).\n\nSee `Pyramid Storage \u003chttps://pythonhosted.org/pyramid_storage/\u003e`_.\n\n\nGoogle Cloud Storage\n--------------------\n\n.. code-block:: ini\n\n    kinto.attachment.gcloud.credentials = \u003cPath to the Service Accounts credentials JSON file\u003e\n    kinto.attachment.gcloud.bucket_name = \u003cbucket name\u003e\n    kinto.attachment.gcloud.acl = publicRead\n\nSee `Google Cloud ACL permissions \u003chttps://cloud.google.com/storage/docs/access-control/making-data-public\u003e`_\n\n\nThe ``folder`` option\n---------------------\n\nWith this option, the files will be stored in sub-folders.\n\nUse the ``{bucket_id}`` and ``{collection_id}`` placeholders to organize the files\nby bucket or collection.\n\n.. code-block:: ini\n\n    kinto.attachment.folder = {bucket_id}/{collection_id}\n\nOr only for a particular bucket:\n\n.. code-block:: ini\n\n    kinto.attachment.resources.blog.folder = blog-assets\n\nOr a specific collection:\n\n.. code-block:: ini\n\n    kinto.attachment.resources.blog.articles.folder = articles-images\n\n\nThe ``keep_old_files`` option\n-----------------------------\n\nWhen set to ``true``, the files won't be deleted from disk/S3 when the associated record\nis deleted or when the attachment replaced.\n\n.. code-block:: ini\n\n    kinto.attachment.keep_old_files = true\n\nOr only for a particular bucket:\n\n.. code-block:: ini\n\n    kinto.attachment.resources.blog.keep_old_files = false\n\nOr a specific collection:\n\n.. code-block:: ini\n\n    kinto.attachment.resources.blog.articles.keep_old_files = true\n\nThe ``randomize`` option\n------------------------\n\nIf you want uploaded files to be stored with a random name (default: True):\n\n.. code-block:: ini\n\n    kinto.attachment.randomize = true\n\nOr only for a particular bucket:\n\n.. code-block:: ini\n\n    kinto.attachment.resources.blog.randomize = true\n\nOr a specific collection:\n\n.. code-block:: ini\n\n    kinto.attachment.resources.blog.articles.randomize = true\n\nThe ``extensions`` option\n-------------------------\n\nIf you want to upload files which are not in the default allowed extensions (see `Pyramid extensions groups \u003chttps://pythonhosted.org/pyramid_storage/#configuration\u003e`_ (default: ``default``):\n\n.. code-block:: ini\n\n    kinto.attachment.extensions = default+video\n\n\nThe ``mimetypes`` option\n------------------------\n\nBy default, the mimetype is guessed from the filename using Python standard mimetypes module.\n\nIf you want to add or override mimetypes, use the following setting and the associated syntax:\n\n.. code-block:: ini\n\n    kinto.attachment.mimetypes = .ftl:application/vnd.fluent;.db:application/vnd.sqlite3\n\n\nDefault bucket\n--------------\n\nIn order to upload files on the ``default`` bucket, the built-in default bucket\nplugin should be enabled before the ``kinto_attachment`` plugin.\n\nIn the configuration, this means adding it explicitly to includes:\n\n.. code-block:: ini\n\n    kinto.includes = kinto.plugins.default_bucket\n                     kinto_attachment\n\nProduction\n----------\n\n* Make sure the ``base_url`` can be reached (and points to ``base_path`` if\n  files are stored locally)\n* Adjust the max size for uploaded files (e.g. ``client_max_body_size 10m;`` for NGinx)\n\nFor example, with NGinx\n\n::\n\n    server {\n        listen 80;\n\n        location /v1 {\n            ...\n        }\n\n        location /files {\n            root /var/www/kinto;\n        }\n    }\n\n\nAPI\n===\n\n**POST /{record-url}/attachment**\n\nIt will create the underlying record if it does not exist.\n\nRequired\n\n- ``attachment``: a single multipart-encoded file\n\nOptional\n\n- ``data``: attributes to set on record (serialized JSON)\n- ``permissions``: permissions to set on record (serialized JSON)\n\n\n**DELETE /{record-url}/attachment**\n\nDeletes the attachement from the record.\n\n\nAttributes\n----------\n\nWhen a file is attached, the related record is given an ``attachment`` attribute\nwith the following fields:\n\n- ``filename``: the original filename\n- ``hash``: a SHA-256 hex digest\n- ``location``: the URL of the attachment\n- ``mimetype``: the `media type \u003chttps://en.wikipedia.org/wiki/Media_type\u003e`_ of\n  the file\n- ``size``: size in bytes\n\n.. code-block:: json\n\n    {\n        \"data\": {\n            \"attachment\": {\n                \"filename\": \"IMG_20150219_174559.jpg\",\n                \"hash\": \"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad\",\n                \"location\": \"http://cdn.service.org/files/ffa9c7b9-7561-406b-b7f9-e00ac94644ff.jpg\",\n                \"mimetype\": \"image/jpeg\",\n                \"size\": 1481798\n            },\n            \"id\": \"c2ce1975-0e52-4b2f-a5db-80166aeca688\",\n            \"last_modified\": 1447834938251,\n            \"theme\": \"orange\",\n            \"type\": \"wallpaper\"\n        },\n        \"permissions\": {\n            \"write\": [\"basicauth:6de355038fd943a2dc91405063b91018bb5dd97a08d1beb95713d23c2909748f\"]\n        }\n    }\n\n\nUsage\n=====\n\nUsing HTTPie\n------------\n\n.. code-block:: bash\n\n    http --form POST http://localhost:8888/v1/buckets/website/collections/assets/records/c2ce1975-0e52-4b2f-a5db-80166aeca689/attachment \\\n        data='{\"type\": \"wallpaper\", \"theme\": \"orange\"}' \\\n        attachment\"@~/Pictures/background.jpg\" \\\n        --auth alice:passwd\n\n.. code-block:: http\n\n    HTTP/1.1 201 Created\n    Access-Control-Expose-Headers: Retry-After, Content-Length, Alert, Backoff\n    Content-Length: 209\n    Content-Type: application/json; charset=UTF-8\n    Date: Wed, 18 Nov 2015 08:22:18 GMT\n    Etag: \"1447834938251\"\n    Last-Modified: Wed, 18 Nov 2015 08:22:18 GMT\n    Location: http://localhost:8888/v1/buckets/website/collections/font/assets/c2ce1975-0e52-4b2f-a5db-80166aeca689\n    Server: waitress\n\n    {\n        \"filename\": \"IMG_20150219_174559.jpg\",\n        \"hash\": \"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad\",\n        \"location\": \"http://cdn.service.org/files/ffa9c7b9-7561-406b-b7f9-e00ac94644ff.jpg\",\n        \"mimetype\": \"image/jpeg\",\n        \"size\": 1481798\n    }\n\nIn order to force a specific file attachment mimetype:\n\n.. code-block:: bash\n\n    http -f POST $URL attachment\"~/files/data.bin;type=application/pdf\"\n\n\nUsing cURL\n----------\n\n.. code-block:: bash\n\n    curl -X POST ${SERVER}/buckets/${BUCKET}/collections/${COLLECTION}/records/${RECORD_ID}/attachment \\\n         -H 'Content-Type:multipart/form-data' \\\n         -F attachment=\"@$FILEPATH;type=application/x-protobuf\" \\\n         -F 'data={\"name\": \"Mac Fly\", \"age\": 42}' \\\n         -H \"Authorization: $STAGE_AUTH\"\n\n\nUsing Python requests\n---------------------\n\n.. code-block:: python\n\n    auth = (\"alice\", \"passwd\")\n    attributes = {\"type\": \"wallpaper\", \"theme\": \"orange\"}\n    perms = {\"read\": [\"system.Everyone\"]}\n\n    files = [(\"attachment\", (\"background.jpg\", open(\"Pictures/background.jpg\", \"rb\"), \"image/jpeg\"))]\n\n    payload = {\"data\": json.dumps(attributes), \"permissions\": json.dumps(perms)}\n    response = requests.post(SERVER_URL + endpoint, data=payload, files=files, auth=auth)\n\n    response.raise_for_status()\n\n\nUsing JavaScript\n----------------\n\n.. code-block:: javascript\n\n    var headers = {Authorization: \"Basic \" + btoa(\"alice:passwd\")};\n    var attributes = {\"type\": \"wallpaper\", \"theme\": \"orange\"};\n    var perms = {\"read\": [\"system.Everyone\"]};\n\n    // File object from input field\n    var filefield = form.elements.attachment.files[0];\n    // If necessary, force the file content-type:\n    var file = new Blob([filefield], { type: \"application/pdf\" });\n\n    // Build form data\n    var payload = new FormData();\n    // Multipart attachment\n    payload.append('attachment', file, \"background.jpg\");\n    // Record attributes and permissions JSON encoded\n    payload.append('data', JSON.stringify(attributes));\n    payload.append('permissions', JSON.stringify(perms));\n\n    // Post form using GlobalFetch API\n    var url = `${server}/buckets/${bucket}/collections/${collection}/records/${record}/attachment`;\n    fetch(url, {method: \"POST\", body: payload, headers: headers})\n      .then(function (result) {\n        console.log(result);\n      });\n\n\nScripts\n=======\n\nTwo scripts are provided in this repository.\n\nThey rely on the ``kinto-client`` Python package, which can be installed in a\nvirtualenv:\n\n::\n\n    $ virtualenv env --python=python3\n    $ source env/bin/activate\n    $ pip install kinto-client\n\nOr globally on your system (**not recommended**):\n\n::\n\n    $ sudo pip install kinto-client\n\n\nUpload files\n------------\n\n``upload.py`` takes a list of files and posts them on the specified server,\nbucket and collection::\n\n    $ python3 scripts/upload.py --server=$SERVER --bucket=$BUCKET --collection=$COLLECTION --auth \"token:mysecret\" README.rst pictures/*\n\nSee ``python3 scripts/upload.py --help`` for more details about options.\n\n\nDownload files\n--------------\n\n``download.py`` downloads the attachments from the specified server, bucket and\ncollection and store them on disk::\n\n    $ python3 scripts/download.py --server=$SERVER --bucket=$BUCKET --collection=$COLLECTION --auth \"token:mysecret\"\n\nIf the record has an ``original`` attribute, the script decompresses the attachment\nafter downloading it.\n\nFiles are stored in the current folder by default.\nSee ``python3 scripts/download.py --help`` for more details about options.\n\n\nKnown limitations\n=================\n\n* No support for chunk upload (#10)\n* Files are not removed when server is purged with ``POST /v1/__flush__``\n\nRelative URL in records (workaround)\n------------------------------------\n\nCurrently the full URL is returned in records. This is very convenient for API consumers\nwhich can access the attached file just using the value in the ``location`` attribute.\n\nHowever, the way it is implemented has a limitation: the full URL is stored in each record\ndirectly. This is annoying because changing the ``base_url`` setting\nwon't actually change the ``location`` attributes on existing records.\n\nAs workaround, it is possible to set the ``kinto.attachment.base_url`` to an empty\nvalue. The ``location`` attribute in records will now contain a *relative* URL.\n\nUsing another setting ``kinto.attachment.extra.base_url``, it is possible to advertise\nthe base URL that can be preprended by clients to obtain the full attachment URL.\nIf specified, it is going to be exposed in the capabilities of the root URL endpoint.\n\n\nRun tests\n=========\n\nRun a fake Amazon S3 server in a separate terminal::\n\n    make run-moto\n\nRun the tests suite::\n\n    make tests\n\n\nReleasing\n=========\n\n1. Create a release on Github on https://github.com/Kinto/kinto-attachment/releases/new\n2. Create a new tag `X.Y.Z` (*This tag will be created from the target when you publish this release.*)\n3. Generate release notes\n4. Publish release\n\n\nNotes\n=====\n\n* `API design discussion \u003chttps://github.com/Kinto/kinto/issues/256\u003e`_ about mixing up ``attachment`` and record fields.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKinto%2Fkinto-attachment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FKinto%2Fkinto-attachment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKinto%2Fkinto-attachment/lists"}