{"id":15899416,"url":"https://github.com/cpbridge/deepsurfer-train","last_synced_at":"2025-10-16T07:26:47.031Z","repository":{"id":204200244,"uuid":"711309270","full_name":"CPBridge/deepsurfer-train","owner":"CPBridge","description":null,"archived":false,"fork":false,"pushed_at":"2024-01-23T17:34:36.000Z","size":293,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-02T19:22:14.283Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CPBridge.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-28T20:59:53.000Z","updated_at":"2023-10-28T21:00:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"e143c458-8467-4a6a-86d7-1c388a298ddf","html_url":"https://github.com/CPBridge/deepsurfer-train","commit_stats":null,"previous_names":["cpbridge/deepsurfer-train"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CPBridge/deepsurfer-train","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CPBridge%2Fdeepsurfer-train","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CPBridge%2Fdeepsurfer-train/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CPBridge%2Fdeepsurfer-train/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CPBridge%2Fdeepsurfer-train/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CPBridge","download_url":"https://codeload.github.com/CPBridge/deepsurfer-train/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CPBridge%2Fdeepsurfer-train/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279165386,"owners_count":26117873,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-16T02:00:06.019Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-06T10:21:03.969Z","updated_at":"2025-10-16T07:26:47.002Z","avatar_url":"https://github.com/CPBridge.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `deepsurfer-train`\n\nCode for training and evaluating Deepsurfer models.\n\nThis README describes the basic setup and use of the project codebase. For\ninformation about the specific processes implemented in this codebase, refer\nto the [docs](docs/) directory.\n\n### Development Installation\n\nThe project's dependencies are designed to be managed with the `poetry` tool.\nYou will need to have poetry installed on your system. See\n[here](https://python-poetry.org/docs/).\n\n##### On Martinos Machines\n\nYou should use poetry and pyenv to manage your environment for this project on\nMartinos servers. See\n[this page](https://github.com/QTIM-Lab/qtim-standards/blob/main/environment_setup.md)\nfor setup instructions for these tools.\n\nThere is a python environment called `deepsurfer-train` in the\nshared qtim pyenv installation allocated for this project. The easiest way to\nget started is just to use this shared environment by running:\n\n```\npyenv activate deepsurfer-train\n```\n\nIt may also be convenient to tell pyenv to activate the `deepsurfer-train` environment whenever your shell is currently in\nthe local repository, which should be cloned into your home directory. You can\ndo this by running this from the repository root:\n\n```\npyenv local deepsurfer-train\n```\n\nSince this environment is shared between multiple users who may be running\ndifferent versions of this repository from different local clones of it, the\npython package defined in this repository should **not** be installed in the\npyenv environment (even though it is set up to be an installable package).\n\nSince this repository is not in the installed environment, you need to make the\npython module implemented in this package (called\n`deepsurfer_train`) findable by\nPython, separately from the environment with dependencies. You can either\nsimply run all commands from the root directory of the repository and/or add\nthat location (e.g. `/home/user/repos/deepsurfer-train`) to your `PYTHONPATH`\nenvironment variable for the current shell session:\n\n```\nexport PYTHONPATH=/home/user/repos/deepsurfer-train:${PYTHONPATH}\n```\n\n##### On Other Machines (e.g. Laptop/Desktop)\n\nWhen working on other machines you need to create a virtualenv for this project\nusing your preferred tool (virtualenv, pyenv, poetry). Then you can use\n`poetry` to create the environment:\n\n```\npoetry install\n```\n\nWhen you are not sharing your python environment with others, it may be easiest\nto install the package in editable mode, rather than setting the `PYTHONPATH`\nenvironment variable.\n\n```\npip install -e .\n```\n\n### Basic Usage Installation\n\nIf you just need to run functionality implemented in the repo without editing\nthe code at all, you can just install the python package using pip, which will\nalso handle all dependencies. You can do this either from a clone of the code:\n\n```\npip install /path/to/cloned/repo\n```\n\nor from directly from GitHub:\n\n```\npip install git+https://github.com/CPBridge/deepsurfer-train\n```\n\nIt is still recommended to do this in a virtual environment.\n\n### Managing Dependencies\n\nManage all dependencies using poetry (either running `poetry add \u003cpackage\u003e` or\nediting `pyproject.toml` yourself), then running `poetry lock` to regenerate\nthe lock file, committing your changes to the repository, and then using\n`poetry install` to actually reflect these changes in the environment.\n\nYou should not just install packages using `pip`.\n\n### CLI Usage\n\nThe Python package provides a command line interface (CLI) when executed using\nthe `python -m deepsurfer_train` command, which should be\navailable whenever the environment that the package is on findable (either\nbecause it is installed or added to the `PYTHONPATH`). There are various\nsub-commands, each for a different process associated with the project, e.g.\ntraining, testing, preprocessing. Each sub-command corresponds to a Python\nmodule in the `deepsurfer_train.commands` package.\n\nTo see a list of available sub-commands, use:\n\n```bash\npython -m deepsurfer_train --help\n```\n\nTo see the documentation of an individual command, e.g. `train` use:\n\n```bash\npython -m deepsurfer_train train --help\n```\n\nTo add new processes to the project, add a new Python source file within the\n`deepsurfer_train/commands` directory and define a\nfunction within it with the same name as the Python file and decorated with\n`@click.command()`. This will create a new CLI sub-command with the same name\nas the file and function, except with any underscores replaced with hyphens.\n\n\n### Pre-commit\n\nThe repository is set up to use `pre-commit`, which will automatically check\nthe codebase for potential problems each time you make a commit. To ensure\nthat these are activated, run this once when you first clone the repository:\n\n```\npre-commit install\n```\n\nThis should be all you need to do, after this is done, various checks and\nprocesses will be run on your code automatically before you make a commit.\nSome of these processes find errors but do not fix them, others actually find\nand fix problems automatically for you.\n\nThe commit will fail if any issues are found, or if any files are automatically\nchanged by the pre-commit hooks. Read the pre-commit output to understand the\nerrors that it found and the changes that were made. If changes were made\nautomatically, all you need to do is `git add` the changed files and re-try\nthe commit. If pre-commit found errors that it cannot fix, you should fix the\nerrors yourself before adding and committing the changed files again.\n\nPlease make use of pre-commit to help avoid errors in your code, and make it\nwell-organized and readable for others. It can dramatically improve the\nquality of your code.\n\nSee [the pre-commit](https://pre-commit.com/) website for further guidance on\nusing the tool.\n\n### Slurm Jobs\n\nBatch scripts to run jobs associated with this project are found in the `slurm`\ndirectory for use on the MLSC cluster. To run these jobs, you should clone a\ncopy of this repository into your Martinos home directory and activate the\nproject's environment (as above) *BEFORE* submitting the job.\n\n### Docker Environment (Optional)\n\nYou should be able to use this codebase in a correctly set-up virtualenv\nwithout needing a docker environment. However, there are also a set of files\nto help you build and run a docker container set up to perform all tasks\nassociated with the project, if you need it.\n\nThe `build_docker.sh` script will build the container and tag it with the\nname `deepsurfer-train`.\n\nTo start an interactive session within the project Docker container, use the\n`run_docker.sh` script. This will mount the cluster storage on the host system\ninto the container (at the same path). Furthermore, your local copy of the\nproject repository on the host system will be bind-mounted into the container\nat `/deepsurfer_train`.\n\n### Singularity (Optional)\n\nYou can build a singularity container for the project using the\n`build_singularity.sh` script in the `singularity` directory. This container\nwill contain a suitable version of Python and CUDA, along with all the\nPython packages dependencies for the project defined in `pyproject.toml`. You\ncan run an interactive singularity container using the `run_singularity.sh`\nscript in the same directory.\n\n### Slurm Jobs with Docker (Optional)\n\nThe project Docker image is built with a special entrypoint script for use on\nthe Slurm computing clusters. This script is placed into the container at\n`/slurm_entrypoint.sh`. The script clones the project source code from the\nGitlab repository, installs the Python package within it, and then executes a\ncommand from the package as specified by its command line arguments.\n\n### Data\n\nThe data for this project is stored in the directory\n`/autofs/cluster/freesurfer/test_data` and other project files at\n`/autofs/cluster/qtim/projects/deepsurfer`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcpbridge%2Fdeepsurfer-train","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcpbridge%2Fdeepsurfer-train","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcpbridge%2Fdeepsurfer-train/lists"}