{"id":23449960,"url":"https://github.com/natefoo/galaxy-pulsar-dev","last_synced_at":"2025-04-13T19:20:34.516Z","repository":{"id":46293472,"uuid":"387926836","full_name":"natefoo/galaxy-pulsar-dev","owner":"natefoo","description":"A tool for running Galaxy and Pulsar for development and testing using Docker Compose","archived":false,"fork":false,"pushed_at":"2025-02-10T18:20:05.000Z","size":24,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-27T09:52:34.692Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/natefoo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-07-20T22:20:22.000Z","updated_at":"2025-02-10T18:20:10.000Z","dependencies_parsed_at":"2023-01-18T16:30:55.251Z","dependency_job_id":null,"html_url":"https://github.com/natefoo/galaxy-pulsar-dev","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natefoo%2Fgalaxy-pulsar-dev","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natefoo%2Fgalaxy-pulsar-dev/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natefoo%2Fgalaxy-pulsar-dev/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/natefoo%2Fgalaxy-pulsar-dev/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/natefoo","download_url":"https://codeload.github.com/natefoo/galaxy-pulsar-dev/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248766685,"owners_count":21158302,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-23T23:31:13.522Z","updated_at":"2025-04-13T19:20:34.472Z","avatar_url":"https://github.com/natefoo.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# galaxy-pulsar-dev\n\n## Overview\n\nA [Docker Compose][docker-compose]-based solution for easing the development and testing of [Pulsar][pulsar] with\n[Galaxy][galaxy].\n\nMost development setups are not going to find the sorts of bugs that commonly occur in production since in development,\nall paths are on the same host and accessible to both the Galaxy and Pulsar servers, which run as the same user.\n**galaxy-pulsar-dev** exists to develop and test your changes on a production-like setup, with RabbitMQ and\nprivilege/host separation.\n\nIn addition, Galaxy and Pulsar are tightly coupled (Galaxy imports Pulsar libs, Pulsar imports Galaxy libs). This\nproject makes it easy to run your changes to the Pulsar client library in Galaxy.\n\nThis project evolved from a [single script][single-script] created in preparation for the Galaxy Community Conference\n2021 CoFest.\n\n## Usage\n\nYou must have [Docker][get-docker] and [Docker Compose][docker-compose] installed.\n\nTo start, run:\n\n```console\n$ make up\n```\n\nThis does a bit of one-time preparation:\n\n- Creating the Docker Compose `.env` file\n- Cloning Galaxy and Pulsar at `../galaxy` and `../pulsar`, respectively\n- Creating Galaxy's virtualenv at `galaxy/venv` and installing its dependencies\n- Building Galaxy's client, if necessary\n\nAfter which, it runs `docker-compose up`. As is normal with foreground Docker Compose sessions, hit `Ctrl-C` to\nterminate. To start daemonized, run:\n\n```console\n$ make up-d\n```\n\nAnd to stop:\n\n```console\n$ make down\n```\n\nOnce the one-time setup has been performed, you can forego the Makefile and run `docker-compose` commands directly, if\nyou prefer.\n\nIt is often ideal to leave PostgreSQL and RabbitMQ running while only restarting Galaxy and/or Pulsar, especially since\nno persistence is configured for PostgreSQL (intentionally). After starting (daemonized, or from another terminal), you\ncan restart Galaxy and Pulsar with:\n\n```console\n$ make restart\n```\n\nOr you can restart individual services like so:\n\n```console\n$ docker-compose restart galaxy-web galaxy-job  # restart Galaxy\n$ docker-compose restart pulsar                 # restart Pulsar\n```\n\nTo reset to the initial state, run:\n\n```console\n$ make clean\n```\n\nThis removes `.env`, the virtualenvs, and Galaxy and Pulsar state directories, but not the Galaxy and Pulsar clones\n(even if it created them). It also removes the Docker containers and images.\n\n## pulsar-galaxy-lib\n\nPulsar's client library is installed in Galaxy at a pinned version from the packages on PyPI, but it is often desirable\nwhen developing to install the Pulsar client library from your development clone of Pulsar instead. This can be acheived\nby running:\n\n```console\n$ make pulsar-galaxy-lib\n$ docker-compose restart galaxy-web galaxy-job\n```\n\nThe [single script][single-script] version of this project attempted to do this step for you automatically as-needed,\nbut that's a bit harder to do in the Compose setup, so you have to trigger it manually for now.\n\n## Customization\n\nTo use Galaxy and/or Pulsar clones at different (perhaps preexisting) paths, generate `.env` first with:\n\n```console\n$ make .env\n```\n\nThen edit `.env` and set `$GALAXY_ROOT` and/or `$PULSAR_ROOT` accordingly.\n\n## Additional Notes\n\nThe directory `galaxy/config/` is mounted into the container over Galaxy's config directory, so you can make changes to\nGalaxy's config there as needed.\n\nGalaxy and Pulsar's virtualenvs and state directories are maintained outside of their respective clones and bind mounted\nin, so using this should not interfere with your normal virtualenvs, job directories, etc. in those clones. The Galaxy\nclient is built and installed to the `static/` directory of the clone, however.\n\nThese Galaxy users may be useful (you need to register them yourself):\n  - admin@example.org: Galaxy Admin\n  - local@example.org: Runs all jobs locally\n  - pulsar@example.org: Runs all jobs (except upload1) via Pulsar\n\nTo query the database, you can use:\n\n```console\n$ docker-compose exec postgres psql -U galaxy -w galaxy\n```\n\nCurrently, Galaxy and Pulsar maintain their own Conda installs, for isolation and more realistic testing. You may find\nthis slow, space-consuming, and annoying if you're testing a lot of tools. If so, you can mount Galaxy's conda into\nPulsar's venv in `docker-compose.yml` and adjust `pulsar/config/app.yml` accordingly.\n\n[docker-compose]: https://docs.docker.com/compose/\n[pulsar]: https://github.com/galaxyproject/pulsar/\n[galaxy]: https://github.com/galaxyproject/galaxy/\n[single-script]: https://gist.github.com/natefoo/f4ddd72f1a07a70f6703d1b640deef17\n[get-docker]: https://docs.docker.com/get-docker/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnatefoo%2Fgalaxy-pulsar-dev","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnatefoo%2Fgalaxy-pulsar-dev","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnatefoo%2Fgalaxy-pulsar-dev/lists"}