{"id":15571241,"url":"https://github.com/jamesramm/qflow","last_synced_at":"2025-09-02T19:31:22.521Z","repository":{"id":22846322,"uuid":"97467872","full_name":"JamesRamm/qflow","owner":"JamesRamm","description":"Distributed ANUGA \u0026 Tuflow Modelling","archived":false,"fork":false,"pushed_at":"2022-12-26T20:35:40.000Z","size":491,"stargazers_count":3,"open_issues_count":19,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-17T00:11:41.528Z","etag":null,"topics":["cloud","distributed","gis","hydrology","python","task-queue","tuflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JamesRamm.png","metadata":{"files":{"readme":"README.rst","changelog":"HISTORY.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-17T11:16:39.000Z","updated_at":"2023-04-14T19:42:09.000Z","dependencies_parsed_at":"2023-01-13T22:19:37.325Z","dependency_job_id":null,"html_url":"https://github.com/JamesRamm/qflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JamesRamm%2Fqflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JamesRamm%2Fqflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JamesRamm%2Fqflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JamesRamm%2Fqflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JamesRamm","download_url":"https://codeload.github.com/JamesRamm/qflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":231804152,"owners_count":18429030,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud","distributed","gis","hydrology","python","task-queue","tuflow"],"created_at":"2024-10-02T17:57:28.572Z","updated_at":"2024-12-30T02:08:11.713Z","avatar_url":"https://github.com/JamesRamm.png","language":"Python","readme":"==========\nQFlow\n==========\n\n.. image:: https://codecov.io/gh/JamesRamm/qflow/branch/master/graph/badge.svg\n        :target: https://codecov.io/gh/JamesRamm/qflow\n\n.. image:: https://img.shields.io/travis/openfloodmap/qflow.svg\n        :target: https://travis-ci.org/openfloodmap/qflow\n\n.. image:: https://pyup.io/repos/github/openfloodmap/qflow/shield.svg\n     :target: https://pyup.io/repos/github/openfloodmap/qflow/\n     :alt: Updates\n\n\nQFlow runs Anuga or Tuflow models across a distributed computing cluster.\nQFlow manages the distribution of model runs across the cluster automatically, ensuring maximum usage\nof your HPC resources and keeping your local computer resources free for your day to day tasks.\n\nQFlow uses Celery \u0026 Redis to manage the task queue and runs on Python 3.5+\n\n.. figure:: docs/qflow_arch.png\n    :scale: 100 %\n    :align: center\n    :alt: QFlow conceptual diagram\n\n\nQuick start\n-----------\n\nFollow this to get started using QFlow from the source.\n\n- First install redis and setup conda environments for ANUGA. This can be done by running ``install.sh`` in the ``tools`` directory\n- Launch 1 or more celery workers: ``tools/run.sh``\n\n\nIn order to execute an ANUGA script:\n\n.. code-block:: python\n\n        from qflow import tasks\n\n        result = tasks.run_anuga.delay('/path/to/my/script.py')\n\n\nIn the above snippet, ``result`` is a celery ``AsyncResult``. You can check the status of the result by accessing ``result.state``.\nIf the task has finished, use ``result.result`` to access the task results.\n\nYou can easily queue up multiple ANUGA or Tuflow tasks:\n\n.. code-block:: python\n\n        from qflow import tasks\n\n        scripts = ['~/run1.py', '~/run2.py', '~/run3.py']\n\n        results = [tasks.run_anuga.delay(script) for script in scripts]\n\nThis script will return immeadiately after adding the tasks to the queue. Your worker(s) will then process each\ntask until no more are available. You can add another worker at any time and it will start picking up tasks from the queue.\n\nRun with Docker\n-----------------\n\nA docker image is provided for QFlow (openfloodmap/qflow).\nYou can run with ``docker run qflow``.\nNote that this image does not run redis - use a tool such as docker-compose to ensure Redis can be accessed from the qflow container.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjamesramm%2Fqflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjamesramm%2Fqflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjamesramm%2Fqflow/lists"}