{"id":35926677,"url":"https://github.com/BAMresearch/openbis-upload-helper","last_synced_at":"2026-01-16T15:00:55.716Z","repository":{"id":287591059,"uuid":"964698435","full_name":"BAMresearch/openbis-upload-helper","owner":"BAMresearch","description":"Upload Helper Tool and Parsers for OpenBIS in the BAM Data Store","archived":false,"fork":false,"pushed_at":"2026-01-12T13:58:08.000Z","size":574,"stargazers_count":3,"open_issues_count":10,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-01-12T18:56:12.576Z","etag":null,"topics":["bam-data-store","metadata-extraction","openbis","parser","uploader"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BAMresearch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-04-11T16:30:40.000Z","updated_at":"2026-01-12T10:36:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"c37f982c-1dfd-49bb-9217-aeb59debe106","html_url":"https://github.com/BAMresearch/openbis-upload-helper","commit_stats":null,"previous_names":["bamresearch/openbis-upload-helper"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/BAMresearch/openbis-upload-helper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BAMresearch%2Fopenbis-upload-helper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BAMresearch%2Fopenbis-upload-helper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BAMresearch%2Fopenbis-upload-helper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BAMresearch%2Fopenbis-upload-helper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BAMresearch","download_url":"https://codeload.github.com/BAMresearch/openbis-upload-helper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BAMresearch%2Fopenbis-upload-helper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28479406,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T11:59:17.896Z","status":"ssl_error","status_checked_at":"2026-01-16T11:55:55.838Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bam-data-store","metadata-extraction","openbis","parser","uploader"],"created_at":"2026-01-10T06:00:38.491Z","updated_at":"2026-01-16T15:00:55.681Z","avatar_url":"https://github.com/BAMresearch.png","language":"Python","funding_links":[],"categories":["Research Data Management"],"sub_categories":[],"readme":"# OpenBIS Upload Helper\n\nThe openBIS Upload Helper is a tool to parse files and extract their relevant metadata into objects and vocabularies in openBIS.\n\nThis tool works using the package [`bam-masterdata`](https://github.com/BAMresearch/bam-masterdata) to grep the object types definitions. Each parser is loaded via entry points from individual GitHub repositories (see e.g., [`masterdata-parser-example`](https://github.dev/BAMresearch/masterdata-parser-example)).\n\n\n## Usage\n\nIf you want to use this tool for your own openBIS instance, we recommend:\n1. Fork the `bam-masterdata` repository.\n2. Fork this `openbis-upload-helper` repository.\n3. Adapt both forks to your specific openBIS instance.\n4. Create your own GitHub repositories for your parsers.\n5. Deploy the app at your convenience.\n\n## Development\n\nIf you want to develop locally this package, clone the project and enter in the workspace folder:\n\n```sh\ngit clone https://github.com/BAMresearch/openbis-upload-helper.git\ncd openbis-upload-helper\n```\n\nCreate a virtual environment (you can use Python\u003e3.10) in your workspace:\n\n```sh\npython3 -m venv .venv\nsource .venv/bin/activate\n```\n\nRun the following commands to pip install the dependencies:\n```sh\npip install --upgrade pip\npip install uv\nuv pip install -e '.[dev,parsers]'\n```\n\n**Note**: The parsers are loaded as entry points via the optional dependencies in the `pyproject.toml` of this repository. More info under Parsers and Dynamic Plugins.\n\n\n### Configuration: settings.ini\n\nA sample settings file is provided at:\n`openbis_upload_helper\\uploader\\settings.ini.example`\n\nCopy this file to the project root and rename it to **`settings.ini`**. **Do not commit or push this file** to the repository because it contains secrets.\n\nWhat to set inside\n- **DEBUG**: set to `True` for local **development** so the Django development server will serve static files automatically.\n- **SECRET_KEY**: Django secret key for cryptographic signing.\n- **SECRET_ENCRYPTION_KEY**: application-specific encryption key used by this project.\n- **CSRF_TRUSTED_ORIGINS**: include your local host entries (for example `http://127.0.0.1:8000`). Use full scheme + host entries.\n- **OPENBIS_URL**: openBIS instance URL for uploading the data.\n- **SPACE_FILTER**: List of space codes to exclude from display (e.g. [\"DEFAULT\", \"SETTINGS\"])\n- **UPLOAD_SIZE_LIMIT**: Maximum total size of the uploaded files in bytes. Example: `UPLOAD_SIZE_LIMIT=10000000000` for 10 GiB. The value must be an integer (bytes).\n- **UPLOAD_TIMEOUT_SECONDS**: Maximum time allowed for the file upload and extraction phase (in seconds). If the loader spends longer than this value while receiving/writing/extracting files, the upload will be aborted. Example: `UPLOAD_TIMEOUT_SECONDS=3000`.\n\n\nStatic files note\n- With **DEBUG=True**, the development server serves static files automatically.\n- With **DEBUG=False** you must collect static files and serve them with a proper web server.\n\nDatabase and admin steps\n- Run migrations before starting the server:\n`python openbis_upload_helper\\manage.py migrate`\n\n- If you need the Django admin site, create a superuser after migrating:\n`python openbis_upload_helper\\manage.py createsuperuser`\n\nOrder recommendation (development)\n1. Copy and configure **`settings.ini`** or **`.env`**.\n2. `python openbis_upload_helper\\manage.py migrate`\n3. (Optional) `python openbis_upload_helper\\manage.py createsuperuser`\n\n### Run the app\n\nYou can locally deploy the app for development by running:\n```sh\npython openbis_upload_helper/manage.py runserver\n```\n\nThis will run the Django app server:\n```sh\nPerforming system checks...\n\nSystem check identified no issues (0 silenced).\nJune 05, 2025 - 06:24:20\nDjango version 5.2.1, using settings 'uploader.settings'\nStarting development server at http://127.0.0.1:8000/\nQuit the server with CONTROL-C.\n```\n\nSimply click on the localhost address, `http://127.0.0.1:8000/`, to launch the app locally.\n\n# Parsers and Dynamic Plugins\n\nThis project discovers parser plugins at runtime using Python entry points.\n\n## How it works\n\n1. **Include the parser dependency**\n\n   Add the parser to your `pyproject.toml` under `[project.optional-dependencies]` in the `parsers` section.\n\n2. **Install the parser**\n\n   After adding the dependency, install it with:\n\n   ```sh\n   uv pip install -e .[parsers]\n    ```\n3. **Load the parser at runtime**\n\n    The application uses the loader function [get_entry_point_parsers]\n    It scans the configured entry point group and returns the available parser plugins dynamically.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBAMresearch%2Fopenbis-upload-helper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBAMresearch%2Fopenbis-upload-helper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBAMresearch%2Fopenbis-upload-helper/lists"}