{"id":13499966,"url":"https://github.com/antonagestam/collectfast","last_synced_at":"2025-03-29T05:32:39.935Z","repository":{"id":8462597,"uuid":"10059871","full_name":"antonagestam/collectfast","owner":"antonagestam","description":"A faster collectstatic command.","archived":true,"fork":false,"pushed_at":"2022-04-21T14:02:06.000Z","size":241,"stargazers_count":416,"open_issues_count":18,"forks_count":72,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-03-19T16:38:54.193Z","etag":null,"topics":["assets-management","django","performance","python"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/Collectfast/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antonagestam.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-05-14T16:44:26.000Z","updated_at":"2025-01-16T18:21:51.000Z","dependencies_parsed_at":"2022-08-03T02:45:22.040Z","dependency_job_id":null,"html_url":"https://github.com/antonagestam/collectfast","commit_stats":null,"previous_names":["jazzband/collectfast"],"tags_count":33,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonagestam%2Fcollectfast","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonagestam%2Fcollectfast/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonagestam%2Fcollectfast/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonagestam%2Fcollectfast/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antonagestam","download_url":"https://codeload.github.com/antonagestam/collectfast/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246145012,"owners_count":20730494,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["assets-management","django","performance","python"],"created_at":"2024-07-31T22:00:48.724Z","updated_at":"2025-03-29T05:32:39.582Z","avatar_url":"https://github.com/antonagestam.png","language":"Python","funding_links":[],"categories":["Asset Management","Python"],"sub_categories":[],"readme":"Hi there 👋\n\nI am looking for a maintainer to take over this project. I am longer working\nwith any project that uses the collectstatic framework and therefor can't be as\ninvested as a maintainer of this project should preferably be.\n\nPlease get in contact with me via issues if you are willing to take over.\n\n# Collectfast\n\nA faster collectstatic command.\n\n[![Test Suite](https://github.com/antonagestam/collectfast/workflows/Test%20Suite/badge.svg)](https://github.com/antonagestam/collectfast/actions?query=workflow%3A%22Test+Suite%22+branch%3Amaster)\n[![Static analysis](https://github.com/antonagestam/collectfast/workflows/Static%20analysis/badge.svg?branch=master)](https://github.com/antonagestam/collectfast/actions?query=workflow%3A%22Static+analysis%22+branch%3Amaster)\n[![Test Coverage](https://api.codeclimate.com/v1/badges/ae25f7543fea0bbccb01/test_coverage)](https://codeclimate.com/github/antonagestam/collectfast/test_coverage)\n[![Maintainability](https://api.codeclimate.com/v1/badges/ae25f7543fea0bbccb01/maintainability)](https://codeclimate.com/github/antonagestam/collectfast/maintainability)\n\n**Features**\n\n- Efficiently decide what files to upload using cached checksums\n- Parallel file uploads\n\n**Supported Storage Backends**\n\n- `storages.backends.s3boto3.S3Boto3Storage`\n- `storages.backends.gcloud.GoogleCloudStorage`\n- `django.core.files.storage.FileSystemStorage`\n\nRunning Django's `collectstatic` command can become painfully slow as more and\nmore files are added to a project, especially when heavy libraries such as\njQuery UI are included in source code. Collectfast customizes the builtin\n`collectstatic` command, adding different optimizations to make uploading large\namounts of files much faster.\n\n\n## Installation\n\nInstall the app using pip:\n\n```bash\n$ python3 -m pip install Collectfast\n```\n\nMake sure you have this in your settings file and add `'collectfast'` to your\n`INSTALLED_APPS`, before `'django.contrib.staticfiles'`:\n\n```python\nSTATICFILES_STORAGE = \"storages.backends.s3boto3.S3Boto3Storage\"\nCOLLECTFAST_STRATEGY = \"collectfast.strategies.boto3.Boto3Strategy\"\nINSTALLED_APPS = (\n    # ...\n    \"collectfast\",\n)\n```\n\n**Note:** `'collectfast'` must come before `'django.contrib.staticfiles'` in\n`INSTALLED_APPS`.\n\n**Note:** The boto strategy will set `preload_metadata` on the remote storage\nto `True`, see [#30][issue-30].\n\n[issue-30]: https://github.com/antonagestam/collectfast/issues/30\n\n##### Upload Strategies\n\nCollectfast Strategy|Storage Backend\n---|---\ncollectfast.strategies.boto3.Boto3Strategy|storages.backends.s3boto3.S3Boto3Storage\ncollectfast.strategies.gcloud.GoogleCloudStrategy|storages.backends.gcloud.GoogleCloudStorage\ncollectfast.strategies.filesystem.FileSystemStrategy|django.core.files.storage.FileSystemStorage\n\nCustom strategies can also be made for backends not listed above by\nimplementing the `collectfast.strategies.Strategy` ABC.\n\n\n## Usage\n\nCollectfast overrides Django's builtin `collectstatic` command so just run\n`python manage.py collectstatic` as normal.\n\nYou can disable Collectfast by using the `--disable-collectfast` option or by\nsetting `COLLECTFAST_ENABLED = False` in your settings file.\n\n### Setting Up a Dedicated Cache Backend\n\nIt's recommended to setup a dedicated cache backend for Collectfast. Every time\nCollectfast does not find a lookup for a file in the cache it will trigger a\nlookup to the storage backend, so it's recommended to have a fairly high\n`TIMEOUT` setting.\n\nConfigure your dedicated cache with the `COLLECTFAST_CACHE` setting:\n\n```python\nCACHES = {\n    \"default\": {\n        # Your default cache\n    },\n    \"collectfast\": {\n        # Your dedicated Collectfast cache\n    },\n}\n\nCOLLECTFAST_CACHE = \"collectfast\"\n```\n\nIf `COLLECTFAST_CACHE` isn't set, the `default` cache will be used.\n\n**Note:** Collectfast will never clean the cache of obsolete files. To clean\nout the entire cache, use `cache.clear()`. [See docs for Django's cache\nframework][django-cache].\n\n**Note:** We recommend you to set the `MAX_ENTRIES` setting if you have more\nthan 300 static files, see [#47][issue-47].\n\n[django-cache]: https://docs.djangoproject.com/en/stable/topics/cache/\n[issue-47]: https://github.com/antonagestam/collectfast/issues/47\n\n### Enable Parallel Uploads\n\nThe parallelization feature enables parallel file uploads using Python's\n`concurrent.futures` module. Enable it by setting the `COLLECTFAST_THREADS`\nsetting.\n\nTo enable parallel uploads, a dedicated cache backend must be setup and it must\nuse a backend that is thread-safe, i.e. something other than Django's default\nLocMemCache.\n\n```python\nCOLLECTFAST_THREADS = 20\n```\n\n\n## Debugging\n\nBy default, Collectfast will suppress any exceptions that happens when copying\nand let Django's `collectstatic` handle it. To debug those suppressed errors\nyou can set `COLLECTFAST_DEBUG = True` in your Django settings file.\n\n\n## Contribution\n\nPlease feel free to contribute by using issues and pull requests. Discussion is\nopen and welcome.\n\n### Testing\n\nThe test suite is built to run against live S3 and GCloud buckets. You can\ndisable live tests by setting `SKIP_LIVE_TESTS=true` or running `make\ntest-skip-live`. To run live tests locally you need to provide API credentials\nto test against. Add the credentials to a file named `storage-credentials` in\nthe root of the project directory:\n\n```bash\nexport AWS_ACCESS_KEY_ID='...'\nexport AWS_SECRET_ACCESS_KEY='...'\nexport GCLOUD_CREDENTIALS='{...}'  # Google Cloud credentials as JSON\n```\n\nInstall test dependencies and target Django version:\n\n```bash\npython3 -m pip install -r test-requirements.txt\npython3 -m pip install django==2.2\n```\n\nRun test suite:\n\n```bash\nmake test\n```\n\nCode quality tools are broken out from test requirements because some of them\nonly install on Python \u003e= 3.7.\n\n```bash\npython3 -m pip install -r lint-requirements.txt\n```\n\nRun linters and static type check:\n\n```bash\nmake lint\n```\n\n\n## License\n\nCollectfast is licensed under the MIT License, see LICENSE file for more\ninformation.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantonagestam%2Fcollectfast","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantonagestam%2Fcollectfast","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantonagestam%2Fcollectfast/lists"}