{"id":15371423,"url":"https://github.com/codingjoe/django-s3file","last_synced_at":"2025-04-12T21:29:42.264Z","repository":{"id":995620,"uuid":"26443516","full_name":"codingjoe/django-s3file","owner":"codingjoe","description":"A lightweight file upload input for Django and Amazon S3","archived":false,"fork":false,"pushed_at":"2024-10-29T18:53:20.000Z","size":708,"stargazers_count":79,"open_issues_count":5,"forks_count":16,"subscribers_count":10,"default_branch":"main","last_synced_at":"2024-10-29T20:45:24.171Z","etag":null,"topics":["amazon","aws","aws-s3","django","django-storages","files","heroku","heroku-deployment","s3","storage"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/codingjoe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"codingjoe","custom":"https://www.paypal.me/codingjoe"}},"created_at":"2014-11-10T15:57:22.000Z","updated_at":"2024-10-10T08:29:22.000Z","dependencies_parsed_at":"2023-12-18T09:29:47.934Z","dependency_job_id":"35cff466-0ee5-4f99-800f-dc0d8644028a","html_url":"https://github.com/codingjoe/django-s3file","commit_stats":{"total_commits":352,"total_committers":19,"mean_commits":"18.526315789473685","dds":0.5909090909090908,"last_synced_commit":"071b17ed504bcfa19ac4fc68a0dcb1260f864a95"},"previous_names":[],"tags_count":47,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codingjoe%2Fdjango-s3file","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codingjoe%2Fdjango-s3file/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codingjoe%2Fdjango-s3file/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codingjoe%2Fdjango-s3file/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/codingjoe","download_url":"https://codeload.github.com/codingjoe/django-s3file/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248634591,"owners_count":21137075,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon","aws","aws-s3","django","django-storages","files","heroku","heroku-deployment","s3","storage"],"created_at":"2024-10-01T13:46:50.574Z","updated_at":"2025-04-12T21:29:42.238Z","avatar_url":"https://github.com/codingjoe.png","language":"Python","funding_links":["https://github.com/sponsors/codingjoe","https://www.paypal.me/codingjoe"],"categories":[],"sub_categories":[],"readme":"# django-s3file\n\nA lightweight file upload input for Django and Amazon S3.\n\nDjango-S3File allows you to upload files directly AWS S3 effectively\nbypassing your application server. This allows you to avoid long running\nrequests from large file uploads. This is particularly helpful for if\nyou run your service on AWS Lambda or Heroku where you have a hard\nrequest limit.\n\n[![PyPi\nVersion](https://img.shields.io/pypi/v/django-s3file.svg)](https://pypi.python.org/pypi/django-s3file/)\n[![Test\nCoverage](https://codecov.io/gh/codingjoe/django-s3file/branch/main/graph/badge.svg)](https://codecov.io/gh/codingjoe/django-s3file)\n[![GitHub\nlicense](https://img.shields.io/badge/license-MIT-blue.svg)](https://raw.githubusercontent.com/codingjoe/django-s3file/main/LICENSE)\n\n## Features\n\n-   lightweight: less 200 lines\n-   no JavaScript or Python dependencies (no jQuery)\n-   easy integration\n-   works just like the built-in\n-   extendable JavaScript API\n\n## For the Nerds\n\n```mermaid\nsequenceDiagram\n    autonumber\n    actor Browser\n    participant S3\n    participant Middleware\n    Browser-\u003e\u003eDjango: GET form view\n    activate Django\n    Django-\u003e\u003eBrowser: RESPONSE w/ presigned POST URL \u0026 signed middleware key\n    deactivate Django\n    Browser-\u003e\u003eS3: POST large file\n    activate S3\n    S3-\u003e\u003eBrowser: RESPONSE AWS S3 key\n    Browser-\u003e\u003eMiddleware: POST AWS S3 key (signed)\n    activate Middleware\n    Middleware-\u003e\u003eS3: GET AWS S3 key\n    S3-\u003e\u003eMiddleware: RESPONSE large file promise\n    deactivate S3\n    Middleware-\u003e\u003eDjango: request incl. large file promise\n    deactivate Middleware\n    activate Django\n    opt only if files is procssed by Django\n        Django--\u003e\u003eS3: GET large file\n        activate S3\n        S3--\u003e\u003eDjango: RESPONSE large file\n        deactivate S3\n    end\n    Django-\u003e\u003eBrowser: RESPONSE success\n    deactivate Django\n```\n\nIn a nutshell, we can bypass Django completely and have AWS handle\nthe upload or any processing. Of course, if you want to do something\nwith your file in Django, you can do so, just like before, with the\nadded advantage, that your file is served from within your datacenter.\n\n## Installation\n\nMake sure you have [Amazon S3\nstorage](http://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html)\nsetup correctly.\n\nJust install S3file using `pip`.\n\n```bash\npip install django-s3file\n# or\npipenv install django-s3file\n```\n\nAdd the S3File app and middleware in your settings:\n\n```python\n# settings.py\n\nINSTALLED_APPS = (\n    '...',\n    's3file',\n    '...',\n)\n\nMIDDLEWARE = (\n    '...',\n    's3file.middleware.S3FileMiddleware',\n    '...',\n)\n```\n\n## Usage\n\nS3File automatically replaces Django's `ClearableFileInput` widget, you\ndo not need to alter your code at all.\n\nThe `ClearableFileInput` widget is only than automatically replaced when\nthe `STORAGES[\"default\"]` setting is set to `django-storages`'\n`S3Boto3Storage` or the dummy `FileSystemStorage` is enabled.\n\n### Setting up the AWS S3 bucket\n\n#### Upload folder\n\nS3File uploads to a single folder. Files are later moved by Django when\nthey are saved to the `upload_to` location.\n\nIt is recommended to [setup\nexpiration](http://docs.aws.amazon.com/AmazonS3/latest/dev/intro-lifecycle-rules.html)\nfor that folder, to ensure that old and unused file uploads don't add up\nand produce costs.\n\nThe default folder name is: `tmp/s3file` You can change it by changing\nthe `S3FILE_UPLOAD_PATH` setting.\n\n#### CORS policy\n\nYou will need to allow `POST` from all origins. Just add the following\nto your CORS policy.\n\n```json\n[\n  {\n    \"AllowedHeaders\": [\n        \"*\"\n    ],\n    \"AllowedMethods\": [\n        \"POST\"\n    ],\n    \"AllowedOrigins\": [\n        \"*\"\n    ],\n    \"ExposeHeaders\": [],\n    \"MaxAgeSeconds\": 3000\n  }\n]\n```\n\n### Using S3File in development\n\nUsing S3File in development can be helpful especially if you want to use\nthe progress signals described above. Therefore, S3File comes with a AWS\nS3 dummy backend. It behaves similar to the real S3 storage backend. It\nis automatically enabled, if the `STORAGES[\"default\"]` setting is set\nto `FileSystemStorage`.\n\nTo prevent users from accidentally using the `FileSystemStorage` and the\ninsecure S3 dummy backend in production, there is also an additional\ndeployment check that will error if you run Django\\'s deployment check\nsuite:\n\n```shell\npython manage.py check --deploy\n```\n\nWe recommend always running the deployment check suite as part of your\ndeployment pipeline.\n\n### Uploading multiple files\n\nDjango does have limited support for [uploading multiple\nfiles](https://docs.djangoproject.com/en/stable/topics/http/file-uploads/#uploading-multiple-files).\nS3File fully supports this feature. The custom middleware ensures\nthat files are accessible via `request.FILES`, even though they have\nbeen uploaded to AWS S3 directly and not to your Django application\nserver.\n\n### Using optimized S3Boto3Storage\n\nSince `S3Boto3Storage` supports storing data from any other fileobj, it\nuses a generalized `_save` function. This leads to the frontend\nuploading the file to S3 and then copying it byte-by-byte to perform a\nmove operation just to rename the uploaded object. For large files this\nleads to additional loading times for the user.\n\nThat\\'s why S3File provides an optimized version of this method at\n`storages_optimized.S3OptimizedUploadStorage`. It uses the more\nefficient `copy` method from S3, given that we know that we only copy\nfrom one S3 location to another.\n\n```python\nfrom s3file.storages_optimized import S3OptimizedUploadStorage\n\nclass MyStorage(S3OptimizedUploadStorage):  # Subclass and use like any other storage\n    default_acl = 'private'\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodingjoe%2Fdjango-s3file","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodingjoe%2Fdjango-s3file","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodingjoe%2Fdjango-s3file/lists"}