{"id":18321846,"url":"https://github.com/sorgerlab/omero.cloudarchive-docker","last_synced_at":"2026-05-08T01:45:52.128Z","repository":{"id":138588603,"uuid":"92304736","full_name":"sorgerlab/omero.cloudarchive-docker","owner":"sorgerlab","description":"OMERO Cloudarchive Docker","archived":false,"fork":false,"pushed_at":"2017-10-18T21:19:40.000Z","size":29,"stargazers_count":0,"open_issues_count":0,"forks_count":2,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-02-15T08:34:36.054Z","etag":null,"topics":["aws","cloud","docker","omero","openmicroscopy","s3"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sorgerlab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-05-24T14:58:25.000Z","updated_at":"2017-10-18T19:06:33.000Z","dependencies_parsed_at":"2023-04-15T12:01:00.966Z","dependency_job_id":null,"html_url":"https://github.com/sorgerlab/omero.cloudarchive-docker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorgerlab%2Fomero.cloudarchive-docker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorgerlab%2Fomero.cloudarchive-docker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorgerlab%2Fomero.cloudarchive-docker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sorgerlab%2Fomero.cloudarchive-docker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sorgerlab","download_url":"https://codeload.github.com/sorgerlab/omero.cloudarchive-docker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248055275,"owners_count":21040151,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","cloud","docker","omero","openmicroscopy","s3"],"created_at":"2024-11-05T18:22:08.955Z","updated_at":"2026-05-08T01:45:47.094Z","avatar_url":"https://github.com/sorgerlab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"OMERO.cloudarchive\n==================\n\nOMERO.cloudarchive can be used to provision the infrastructure for OMERO in a\nlocal or cloud environment. This mechanism can \"hydrate\" a previously created\narchive or be empty. Once running, data can be loaded/annotated/etc and then\nfinally \"dehydrated\" into object storage.\n\nInstructions for deploying on AWS are available at:\n[https://sorgerlab.github.io/omero.cloudarchive-cloudformation/](https://sorgerlab.github.io/omero.cloudarchive-cloudformation/)\n\nInstructions for deploying locally follow.\n\nAWS\n---\n\nDeploy a local setup using docker\n\n```bash\n# Quickstart\ndocker-compose up\n```\n\nTo customize the configuration either edit `docker-compose.yml` or run the\ndocker container's manually like this:\n\n```bash\n\ndocker run \\\n  -d \\\n  --name postgres \\\n  -e POSTGRES_PASSWORD=postgres \\\n  postgres:9.4\n\ndocker run \\\n  -d \\\n  --name omero-master \\\n  --link postgres:db \\\n  -e CONFIG_omero_db_name=postgres \\\n  -e CONFIG_omero_db_pass=postgres \\\n  -e CONFIG_omero_db_user=postgres \\\n  -e CONFIG_omero_web_public_enabled=true \\\n  -e ROOTPASS=omero \\\n  -p 4063:4063 \\\n  -p 4064:4064 \\\n  dpwrussell/omero.cloudarchive\n\ndocker run \\\n  -d \\\n  --name omero-web \\\n  --link omero-master:master \\\n  -e PUBLIC_GROUP=public-group \\\n  -p 8080:8080 \\\n  dpwrussell/omero-grid-web\n```\n\nNow the OMERO infrastructure is up and running normal importing using Insight\ncan be commenced on localhost, standard OMERO ports (unless changed in the\nabove examples). The web client will also be operational on port 8080 (again,\nunless changed in the above examples).\n\nIn-place\n--------\n\nIt is also possible to do in-place importing. This is a good way of creating\nan archive without losing the original file-structure of the data. This is\nuseful if the data archive is multipurpose, being accessed through OMERO, but\nalso directly, thus preserving the original filesystem structure can be\ndesirable.\n\nWhen running the docker container for `omero-master`, include an additional\nsetting `-v \u003cdata_dir\u003e:/mnt/\u003cdata_dir\u003e `, adding a local directory to the\ncontainer.\n\n```bash\n\ndocker run \\\n  -d \\\n  --name omero-master \\\n  --link postgres:db \\\n  -e CONFIG_omero_db_name=postgres \\\n  -e CONFIG_omero_db_pass=postgres \\\n  -e CONFIG_omero_db_user=postgres \\\n  -e CONFIG_omero_web_public_enabled=true \\\n  -p 4063:4063 \\\n  -p 4064:4064 \\\n  -v /TestData:/mnt/TestData \\\n  dpwrussell/omero.cloudarchive\n```\n\nThen log into the omero container through docker:\n\n```bash\ndocker exec -it --user omero omero-master /bin/bash\n```\n\nOnce logged in, create a symlink from the mounted data directory to a directory\nin `/OMERO`, like this:\n\n```bash\nln -s /mnt/TestData/project1 /OMERO/inplace\n```\n\nWarning: When dehydrating this OMERO instance, everything in the symlinked\ndirectory will be uploaded to S3. In the above example this would be everything\nin the `/mnt/TestData/project1` via the `/OMERO/inplace` symlink. This is an\nintentional effort to retain the files from the original structure which may\nor may not be relevant to OMERO.\n\nWe can now do an in-place imports of anything in `/OMERO/inplace`, in this\nexample we will import everything:\n\n```bash\n~/OMERO.server/bin/omero import -- --transfer=ln_s /OMERO/inplace/\n```\n\nDehydrate\n---------\n\nOnce an archive is built, it can be \"dehydrated\" into S3. A utility script is\nprovider for this which should be run from *outside* the container, on the\nhost. The reason for this is that currently a docker container can not\nintrospect details about itself so it is necessary to gather these details\nfrom outside.\n\nThis script requires container name and S3 bucket identifier as arguments. It\nalso uses the `aws` CLI to generate temporary credentials which are then used\nto interact with S3.\n\nBefore running the utility script, create a bucket in S3 to dehydrate the\narchive into. This can be done on the AWS console or through the CLI like this\n(In this case I make it publicly readable to anyone, and in the us-east-1\nregion):\n\n```bash\nexport BUCKETNAME=\"\u003cbucket_name\u003e\"\naws s3api create-bucket --bucket ${BUCKETNAME} --acl public-read --region us-east-1\necho \"{ \\\"Version\\\" : \\\"2012-10-17\\\", \\\"Statement\\\" : [ { \\\"Effect\\\" : \\\"Allow\\\", \\\"Principal\\\" : \\\"*\\\", \\\"Resource\\\" : [ \\\"arn:aws:s3:::ionewfioewn9023/*\\\" ], \\\"Sid\\\" : \\\"PublicReadGetObject\\\", \\\"Action\\\" : [ \\\"s3:GetObject\\\" ] } ] }\" \u003e s3_public.json\naws s3api put-bucket-policy --bucket ${BUCKETNAME} --policy file://s3_public.json --region us-east-1\n```\n\nNow use the utility script to start the dehydration process.\n\n```bash\n./dehydrate-docker.sh \u003ccontainer_id\u003e \u003cs3_bucket\u003e\n```\n\nThis may take some time depending on the size of the repository to upload.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsorgerlab%2Fomero.cloudarchive-docker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsorgerlab%2Fomero.cloudarchive-docker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsorgerlab%2Fomero.cloudarchive-docker/lists"}