{"id":21070684,"url":"https://github.com/pyronear/pyro-storage","last_synced_at":"2026-02-04T12:02:36.687Z","repository":{"id":37798685,"uuid":"494043205","full_name":"pyronear/pyro-storage","owner":"pyronear","description":"Data curation for wildfire detection","archived":false,"fork":false,"pushed_at":"2024-07-06T08:53:17.000Z","size":793,"stargazers_count":1,"open_issues_count":5,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-29T13:27:53.210Z","etag":null,"topics":["backend-api","docker"],"latest_commit_sha":null,"homepage":"https://pyronear.org/pyro-storage","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pyronear.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"pyronear","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2022-05-19T11:30:06.000Z","updated_at":"2024-07-03T21:48:18.000Z","dependencies_parsed_at":"2023-09-22T16:28:36.428Z","dependency_job_id":"0768afc6-c36b-4646-8937-142c8ec86dfc","html_url":"https://github.com/pyronear/pyro-storage","commit_stats":{"total_commits":34,"total_committers":3,"mean_commits":"11.333333333333334","dds":0.2941176470588235,"last_synced_commit":"3432188d48b90663b8c8ba4b8c36e32a4b4b021e"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyronear%2Fpyro-storage","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyronear%2Fpyro-storage/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyronear%2Fpyro-storage/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyronear%2Fpyro-storage/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pyronear","download_url":"https://codeload.github.com/pyronear/pyro-storage/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241570825,"owners_count":19984001,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backend-api","docker"],"created_at":"2024-11-19T18:47:50.344Z","updated_at":"2026-02-04T12:02:31.650Z","avatar_url":"https://github.com/pyronear.png","language":"Python","funding_links":["https://github.com/sponsors/pyronear"],"categories":[],"sub_categories":[],"readme":"# Data curation API\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/pyronear/pyro-storage/actions?query=workflow%3Abuilds\"\u003e\n    \u003cimg alt=\"CI Status\" src=\"https://img.shields.io/github/actions/workflow/status/pyronear/pyro-storage/builds.yml?branch=main\u0026label=CI\u0026logo=github\u0026style=flat-square\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"http://pyronear-api.herokuapp.com/redoc\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/actions/workflow/status/pyronear/pyro-storage/builds.yml?branch=main\u0026label=docs\u0026logo=read-the-docs\u0026style=flat-square\" alt=\"Documentation Status\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://codecov.io/gh/pyronear/pyro-storage\"\u003e\n    \u003cimg src=\"https://img.shields.io/codecov/c/github/pyronear/pyro-storage.svg?logo=codecov\u0026style=flat-square\" alt=\"Test coverage percentage\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/ambv/black\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/code%20style-black-000000.svg?style=flat-square\" alt=\"black\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.codacy.com/gh/pyronear/pyro-storage/dashboard?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=pyronear/pyro-storage\u0026amp;utm_campaign=Badge_Grade\"\u003e\u003cimg src=\"https://app.codacy.com/project/badge/Grade/da9d595af69348b5882a6eec791a6acd\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://pypi.org/project/pyrostorage/\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/v/pyrostorage.svg?logo=python\u0026logoColor=fff\u0026style=flat-square\" alt=\"PyPi Status\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://anaconda.org/pyronear/pyrostorage\"\u003e\n    \u003cimg alt=\"Anaconda\" src=\"https://img.shields.io/conda/vn/pyronear/pyrostorage?style=flat-square?style=flat-square\u0026logo=Anaconda\u0026logoColor=white\u0026label=conda\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://hub.docker.com/r/pyronear/pyro-storage\"\u003e\n    \u003cimg alt=\"Docker Image Version\" src=\"https://img.shields.io/docker/v/pyronear/pyro-storage?style=flat-square\u0026logo=Docker\u0026logoColor=white\u0026label=docker\"\u003e\n  \u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/pyversions/pyrostorage.svg?style=flat-square\" alt=\"pyversions\"\u003e\n  \u003cimg src=\"https://img.shields.io/pypi/l/pyrostorage.svg?style=flat-square\" alt=\"license\"\u003e\n\u003c/p\u003e\n\n\nThe building blocks of our data curation API.\n\n## Quick Tour\n\n### Running/stopping the service\n\nYou can run the API containers using this command:\n\n```shell\nmake run\n```\n\nYou can now navigate to `http://localhost:8080/docs` to interact with the API (or do it through HTTP requests) and explore the documentation.\n\nIn order to stop the service, run:\n```shell\nmake stop\n```\n\n### How is the database organized\n\nThe back-end core feature is to interact with the metadata tables. For the service to be useful for data curation, multiple tables/object types are introduced and described as follows:\n\n#### Access-related tables\n\n- Accesses: stores the hashed credentials and access level for users \u0026 devices.\n\n#### Core data curation worklow tables\n\n- Media: metadata of a picture and its storage bucket key.\n- Annotations: metadata of an annotation file and its storage bucket key.\n\n![UML](https://github.com/pyronear/pyro-storage/releases/download/v0.1.0/uml_diagram.png)\n\n### What is the full data curation workflow through the API\n\nThe API has been designed to provide, for each data entry:\n- timestamp\n- the picture that was uploaded\n- the annotation file associated with that picture\n\nWith the previously described tables, here are all the steps to upload a data entry:\n- Prerequisites (ask the instance administrator): register user\n- Create a media object \u0026 upload content: save the picture metadata and upload the image content.\n- Create an annotation object \u0026 upload content: save the annotation metadata and upload the annotation content.\n\n## Installation\n\n### Prerequisites\n\n- [Docker](https://docs.docker.com/engine/install/)\n- [Docker compose](https://docs.docker.com/compose/)\n- [Make](https://www.gnu.org/software/make/) (optional)\n\nThe project was designed so that everything runs with Docker orchestration (standalone virtual environment), so you won't need to install any additional libraries.\n\n## Configuration\n\nIn order to run the project, you will need to specific some information, which can be done using a `.env` file.\nThis file will have to hold the following information:\n- `S3_ACCESS_KEY`: public key to access to the [S3 storage service](https://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-sign-up.html)\n- `S3_SECRET_KEY`: private key to access the resource.\n- `S3_REGION`: your S3 bucket is geographically identified by its location's region\n- `S3_ENDPOINT_URL`: the URL providing a S3 endpoint by your cloud provider\n- `BUCKET_NAME`: the name of the storage bucket\n\nOptionally, the following information can be added:\n- `SENTRY_DSN`: the URL of the [Sentry](https://sentry.io/) project, which monitors back-end errors and report them back.\n- `SERVER_NAME`: the server tag to apply to events.\n- `CORS_ORIGIN`: comma-separated list of allowed origins\n\nSo your `.env` file should look like something similar to:\n```\nS3_ACCESS_KEY=YOUR_ACCESS_KEY\nS3_SECRET_KEY=YOUR_SECRET_KEY\nS3_REGION=bucket-region\nS3_ENDPOINT_URL='https://s3.mydomain.com/'\nBUCKET_NAME=my_storage_bucket_name\nSENTRY_DSN='https://replace.with.you.sentry.dsn/'\nSERVER_NAME=my_storage_bucket_name\n```\n\nThe file should be placed at the root folder of your local copy of the project.\n\n## More goodies\n\n### Documentation\n\nThe full package documentation is available [here](https://pyronear.org/pyro-storage) for detailed specifications.\n\n### Python client\n\nThis project is a REST-API, and you can interact with the service through HTTP requests. However, if you want to ease the integration into a Python project, take a look at our [Python client](client).\n\n\n## Contributing\n\nAny sort of contribution is greatly appreciated!\n\nYou can find a short guide in [`CONTRIBUTING`](CONTRIBUTING.md) to help grow this project!\n\n\n\n## License\n\nDistributed under the Apache 2.0 License. See [`LICENSE`](LICENSE) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyronear%2Fpyro-storage","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpyronear%2Fpyro-storage","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpyronear%2Fpyro-storage/lists"}