{"id":13587183,"url":"https://github.com/pop-os/tensorman","last_synced_at":"2025-04-05T03:10:04.546Z","repository":{"id":49157249,"uuid":"202021804","full_name":"pop-os/tensorman","owner":"pop-os","description":"Utility for easy management of Tensorflow containers","archived":false,"fork":false,"pushed_at":"2024-07-24T03:16:35.000Z","size":113,"stargazers_count":207,"open_issues_count":11,"forks_count":17,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-03-29T02:04:56.637Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pop-os.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-12T23:01:28.000Z","updated_at":"2024-12-20T05:13:16.000Z","dependencies_parsed_at":"2024-11-06T05:33:49.356Z","dependency_job_id":"ed53f5bb-0c44-4d34-9176-be8ca571a7d0","html_url":"https://github.com/pop-os/tensorman","commit_stats":{"total_commits":50,"total_committers":4,"mean_commits":12.5,"dds":0.14,"last_synced_commit":"b1125f71b55a8d9a4d674a62fa1e8868d40d0f0d"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pop-os%2Ftensorman","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pop-os%2Ftensorman/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pop-os%2Ftensorman/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pop-os%2Ftensorman/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pop-os","download_url":"https://codeload.github.com/pop-os/tensorman/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247280272,"owners_count":20912967,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:06:04.670Z","updated_at":"2025-04-05T03:10:04.525Z","avatar_url":"https://github.com/pop-os.png","language":"Rust","funding_links":[],"categories":["ML frameworks \u0026 applications","Rust","CUDA Tools Libraries, and Frameworks","CUDA Tools","ML Frameworks, Libraries, and Tools","Tools"],"sub_categories":["Interfaces","viii. Linear Regression","Winetricks","Objective-C Tools, Libraries, and Frameworks","Mesh networks"],"readme":"# Tensorflow Container Manager\n\nPackaging Tensorflow for Linux distributions is notoriously difficult, if not impossible. Every release of Tensorflow is accommodated by a myriad of possible build configurations, which requires building many variants of Tensorflow for each Tensorflow release. To make matters worse, each new version of Tensorflow will depend on a wide number of shared dependencies which may not be supported on older versions of a Linux distribution that is still actively supported by the distribution maintainers.\n\nTo solve this problem, the Tensorflow project provides official Docker container builds, which allows Tensorflow to operate in an isolated environment that is contained from the rest of the system. This virtual environment can operate independent of the base system, allowing you to use any version of Tensorflow on any version of a Linux distribution that supports the Docker runtime.\n\nHowever, configuring and managing Docker containers for Tensorflow using the `docker` command line is currently tedious, and managing multiple versions for different projects is even moreso. To solve this problem for our users, we have developed `tensorman` as a convenient tool to manage the installation and execution of Tensorflow Docker containers. It condenses the command-line soup into a set of simple commands that are easy to memorize.\n\n## Comparison to Docker Command\n\nTake the following Docker invocation as an example:\n\n```\ndocker run -u $UID:$UID -v $PWD:/project -w /project \\\n    --runtime=nvidia -it --rm tensorflow/tensorflow:latest-gpu \\\n    python ./script.py\n```\n\nThis designates for the latest version of Tensorflow with GPU support to be used, mounting the working directory to `/project`, launching the container with the current user account, and and executing `script.py` with the Python binary in the container. With `tensorman`, we can achieve the same with:\n\n```\ntensorman run --gpu python -- ./script.py\n```\n\nWhich defaults to the latest version, and whose version and tag variants can be set as defaults per-run, per-project, or user-wide.\n\n## Installing/Updating Containers\n\nBy default, docker will automatically install a container when running a container that it is not already installed. However, if you would like to install a container beforehand, you may do so using the `pull` subcommand.\n\n```\ntensorman pull 1.14.0\ntensorman pull latest\n```\n\n## Running commands in containers\n\nThe `run` subcommand allows you to execute a command from within the container. This could be the `bash` shell, for an interactive session inside the container, or the program / compiler which you wish to run.\n\n```\n# Default container version with Bash prompt\ntensorman run bash\n\n# Default container version with Python script\ntensorman run python -- script.py\n\n# Default container version with GPU support\ntensorman run --gpu bash\n\n# With GPU, Python3, and Juypyter support\ntensorman run --gpu --python3 --jupyter bash\n```\n\n## Setting the container version\n\nTaking inspiration from [rustup], there are methods to set the container version per-run, per-project, and per-user. The per-run version always takes priority over a per-project definition, which takes priority over the per-user configuration.\n\n[rustup]: https://rustup.rs\n\n### Setting per-run\n\nIf a version is specified following a `+` argument, `tensorman` will prefer this version.\n\n```\ntensorman +1.14.0 run --python3 --gpu bash\n```\n\nCustom images may be specified with a `=` argument.\n\n```\ntensorman =custom-image run --gpu bash\n```\n\n### Setting per-project\n\nThere are two files that can be used for configuring Tensorman locally: `tensorflow-toolchain`, and `Tensorman.toml`. These files will be automatically detected if they can be found in a parent directory.\n\n#### tensorflow-toolchain\n\nThis file overrides the tensorflow image, defined either in `Tensorman.toml`, or the user-wide configuration file.\n\n```\n1.14.0 gpu python3\n```\n\nOr specifying a custom image:\n\n```\n=custom-image gpu\n```\n\n#### Tensorman.toml\n\nThis file supports additional configuration parameters, with a user-wide configuration located at `~/.config/tensorman/config.toml`, and a project-wide location at `Tensorman.toml`. One of the reasons in which you may want to use this file is to declare some additional Docker flags, with the `docker_flags` key.\n\nUsing a default tensorflow image:\n\n```toml\ndocker_flags = [ '-p', '8080:8080' ]\ntag = '2.0.0'\nvariants = ['gpu', 'python3']\n```\n\nDefining a custom image:\n\n```toml\ndocker_flags = [ '-p', '8080:8080' ]\nimage = 'custom-image'\nvariants = ['gpu']\n```\n\nOne useful docker flag is the `-v` flag, that can be used at runtime to mount other directories not included in your image.  The syntax for the argument of `-v` is `source:destination`. For example, if you have a large dataset in your home directory that you don't want to include as part of your image, you can mount it at runtime by addding the following line to your `config.toml` file:\n\n```toml\ndocker_flags = [ '-v', '/home/\u003cusername\u003e/\u003cdataset\u003e:/home/\u003cusername\u003e/\u003cdataset\u003e' ]\n```\n\n### Setting per-user\n\nyou can set a default version user-wide using the `default` subcommand. This version of Tensorflow will be launched whenever you use the `tensorman run` command.\n\n```\ntensorman default 1.14.0\ntensorman default latest gpu python3\ntensorman default nightly\n```\n\n\u003e By default, `tensorman` will use `latest` as the default per-user version tag.\n\n## Showing the active container version\n\nIf you would like to know which container will be used when launched from the current working directory, you can use the `show` command.\n\n```\ntensorman show\n```\n\n## Removing container images\n\nHaving many containers installed simultaneously on the same system can quickly use a lot of disk storage. If you find yourself in need of culling the containers installed on your system, you may do so with the `remove` command.\n\n```\ntensorman remove 1.14.0\ntensorman remove latest\ntensorman remove 481cb7ea88260404\ntensorman remove custom-image\n```\n\n## Listing installed container images\n\nTo aid in discovering what containers are installed on the system, the `list` subcommand is available.\n\n```\ntensorman list\n```\n\n## Creating a custom image\n\nIn most projects, you will need to pull in more dependencies than the base Tensorflow image has. To do this, you will need to create the image by running a tensorflow container as root, installing and setting up the environment how you need it, and then saving those changes as a new custom image.\n\nTo do so, you will need to build the container in one terminal, and save it from another.\n\n### Build new image\n\nFirst launch a terminal where you will begin configuring the docker image:\n\n```\ntensorman run --gpu --python3 --root --name CONTAINER_NAME bash\n```\n\nOnce you've made the changes needed, open another terminal and save it as a new image:\n\n```\ntensorman save CONTAINER_NAME IMAGE_NAME\n```\n\n### Running the custom image\n\nYou should then be able to specify that container with tensorman, like so:\n\n```\ntensorman =IMAGE_NAME run --gpu bash\n```\n\n\u003e The `--python3` and `--jupyter` flags do nothing for custom containers, but `--gpu` is required to enable runtime support for the GPU.\n\n### Removing the custom image\n\nImages saved through tensorman are manageable through tensorman. Listing and removing works the same:\n\n```\ntensorman remove IMAGE_NAME\n```\n\n## License\n\nLicensed under the GNU General Public License, Version 3.0, ([LICENSE](LICENSE) or https://www.gnu.org/licenses/gpl-3.0.en.html)\n\n### Contribution\n\nAny contribution intentionally submitted for inclusion in the work by you, shall be licensed under the GNU GPLv3.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpop-os%2Ftensorman","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpop-os%2Ftensorman","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpop-os%2Ftensorman/lists"}