{"id":26447502,"url":"https://github.com/mycore-org/s3-mcr-plugin","last_synced_at":"2025-03-18T13:57:22.746Z","repository":{"id":77166593,"uuid":"428752561","full_name":"MyCoRe-Org/s3-mcr-plugin","owner":"MyCoRe-Org","description":"A MyCoRe-Plugin to link s3 buckets to a mycore object, including a GUI to browse them.","archived":false,"fork":false,"pushed_at":"2024-06-20T13:48:31.000Z","size":382,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-01-07T13:03:42.004Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MyCoRe-Org.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-11-16T17:29:45.000Z","updated_at":"2024-06-20T13:48:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"ba6bdda9-c125-4b43-8b0f-9ca8bec0be4b","html_url":"https://github.com/MyCoRe-Org/s3-mcr-plugin","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MyCoRe-Org%2Fs3-mcr-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MyCoRe-Org%2Fs3-mcr-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MyCoRe-Org%2Fs3-mcr-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MyCoRe-Org%2Fs3-mcr-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MyCoRe-Org","download_url":"https://codeload.github.com/MyCoRe-Org/s3-mcr-plugin/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244236086,"owners_count":20420752,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-18T13:57:21.908Z","updated_at":"2025-03-18T13:57:22.732Z","avatar_url":"https://github.com/MyCoRe-Org.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Plugin\nThis plugin provides an additional file view to view buckets stored in an external store.\nIt also adds some rest endpoint and makes it possible to link buckets to MODS objects.\nAn external store provides information about files and also allows to download them.\nTo ensure integrity, information for all files in the external store is stored in a derivate.\nAfter integration, it is no longer necessary to communicate with the store for file information, as the information is provided via a local index.\nIt is currently possible to integrate an s3 buckets as external stores.\nIn addition, archives can be resolved to provide information about the files. It may also be possible to download files directly from archives.\n\n## Configuration\n\nThe plugin comes preconfigured with the following properties. \n```\nMCR.Crypt.Cipher.external-store-settings.class=org.mycore.crypt.MCRAESCipher\nMCR.Crypt.Cipher.external-store-settings.KeyFile=%MCR.datadir%/external_store_settings.key\nMCR.Crypt.Cipher.external-store-settings.EnableACL=false\n\nMCR.ExternalStore.Service.StoreProviderSettings.Cipher=external-store-settings\nMCR.ExternalStore.Service.ResolveArchives=true\n\nMCR.ExternalStore.s3.Provider.Class=org.mycore.externalstore.s3.MCRExternalStoreS3Provider\n\nMCR.ExternalStore.MaxDownloadSize=1000000\n\nMCR.ExternalStore.InfoIndex.Class=org.mycore.externalstore.index.db.MCRExternalStoreDbStoreInfoIndex\n\nMCR.ExternalStore.ArchiveResolver.zip.Download=true\nMCR.ExternalStore.ArchiveResolver.tar.Download=false\n\nMCR.ExternalStore.ProxyServlet.Disabled=true\nMCR.ExternalStore.ProxyServlet.Url=%MCR.baseurl%/esp\n```\n### Create key file\nYou need to create the external-store-settings key with the CLI command:\n```\ngenerate keyfile for cipher external-store-settings\n```\n### Embed\nIf you are using a MIR you will get a preconfigured metadata-page id which you can insert e.g.\n```\nMIR.Layout.Start=mir-abstract-badges,mir-workflow,mir-abstract-title,mir-abstract-plus,toc,mir-file-upload,mir-viewer,mir-player,mir-extended-file-browser,mir-collapse-files,mir-metadata\n```\nIf not you can take a look at the file **src/main/resources/xsl/metadata/extended-file-browser.xsl** to see how it is done.\n\n\n### Derivate types\nYou need to add `derivate_types:external_store_\u003ctype\u003e` to the derivate_types classification, for s3 buckets it is `derivate_types:external_store_s3`.\n\n## How are settings and information saved?\nThe relevant data is stored in associated derivate.\nSensitive information, such as settings for communication with the provider of the store, is stored in encrypted form in provider_settings.json.\nInformation about the files is stored in files.json.\nInformation about archives is stored separately in archives.json because it depends on the configuration of the resolvers.\n\n## Download\nBefore a download, the integrity is checked and a download URL is generated.\nIn the case of S3, this is a presigned url.\nIt is possible that the provider endpoint cannot be reached publicly.\nIn this case, a proxy can be activated when creating a store.\nA proxy servlet is available for this purpose.\nDownload urls are adapted accordingly and routed through the servlet.\nAlternatively, an individual proxy url can be specified so that an independent proxy can also be used.\n\n### Activate proxy servlet\nThe proxy servlet is deactivated by default and can be activated as follows.\nAlternatively, the path can also be adapted.\n```\nMCR.ExternalStore.ProxyServlet.Disabled=false\nMCR.ExternalStore.ProxyServlet.Url=%MCR.baseurl%/esp\n```\n\n\n## How can I test local?\nYou can use minio for testing s3 with docker and docker-compose.\nCopy this to a docker-compose.yml and run `docker-compose up` to start a demo s3 server.\n```\nversion: '3.3'\nservices:\n    minio:\n        ports:\n            - '9000:9000'\n            - '9001:9001'\n        environment:\n            - MINIO_ROOT_USER=admin\n            - MINIO_ROOT_PASSWORD=alleswirdgut\n        volumes:\n            - './data:/data'\n        image: minio/minio\n        command: server /data --console-address :9001\n\n```\n\nThe login for the admin user is stored in the environment variables **MINIO_ROOT_USER** and **MINIO_ROOT_PASSWORD**.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmycore-org%2Fs3-mcr-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmycore-org%2Fs3-mcr-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmycore-org%2Fs3-mcr-plugin/lists"}