{"id":14967176,"url":"https://github.com/golemfactory/gvmkit-build-rs","last_synced_at":"2025-10-19T09:31:42.670Z","repository":{"id":67707539,"uuid":"366002055","full_name":"golemfactory/gvmkit-build-rs","owner":"golemfactory","description":null,"archived":false,"fork":false,"pushed_at":"2024-04-02T13:19:04.000Z","size":302,"stargazers_count":2,"open_issues_count":9,"forks_count":2,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-04-14T04:19:26.284Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/golemfactory.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-10T10:20:52.000Z","updated_at":"2024-06-21T19:06:43.820Z","dependencies_parsed_at":"2024-04-02T14:35:21.592Z","dependency_job_id":null,"html_url":"https://github.com/golemfactory/gvmkit-build-rs","commit_stats":{"total_commits":29,"total_committers":5,"mean_commits":5.8,"dds":0.5517241379310345,"last_synced_commit":"2dda7d02501e09b4551318f0fd5f0096067a4071"},"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/golemfactory%2Fgvmkit-build-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/golemfactory%2Fgvmkit-build-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/golemfactory%2Fgvmkit-build-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/golemfactory%2Fgvmkit-build-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/golemfactory","download_url":"https://codeload.github.com/golemfactory/gvmkit-build-rs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219869246,"owners_count":16555571,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T13:37:34.583Z","updated_at":"2025-10-19T09:31:37.317Z","avatar_url":"https://github.com/golemfactory.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gvmkit-build\n\nGolem VM Image builder used as companion app for Golem Registry: https://registry.golem.network\n\n## Release procedure\n\nThis section is intended for developers maintaining this repository, skip it if you are using this tool.\n\n1. Bump versions in Cargo.toml (and according lock) and pyproject.toml and node/package.json\n    * The version 0.x.y HAS to be the same in all three files\n2. Create tag with the same version number like v0.x.y\n3. Push tag to github\n4. Run custom action in github with selected tag\n    * Actions -\u003e Actions -\u003e Build Release -\u003e Run workflow -\u003e Select tag v0.x.y -\u003e Run workflow\n    * Observe in logs if everything wen smoothly.\n    * Package should be published to pip and npm repositories, release should be created automatically\n\n## Requirements\n\nRunning docker engine is required. Tool supports Linux, Windows and macOS.\nNote that when using macOS ARM version use --platform linux/amd64 option for docker builds.\n\n## Installation\n\nYou can install gvmkit-build using pip (python3 with pip installer is required)\n```\npip install gvmkit-build\n```\nor install form npm (npm installation is required)\n```\nnpm install -g gvmkit-build\n```\nor install using cargo (Rust toolchain is required, a bit slow because it compiles from sources)\n```\ncargo install gvmkit-build\n```\nor download prebuild from github releases page:\n\nhttps://github.com/golemfactory/gvmkit-build-rs/releases\n\nor build from sources, you can find binary in target/release/gvmkit-build (or gvmkit-build.exe on Windows)\n```\ncargo build --release\n```\n\n## Images\n\nGolem Network is using gvmi images as base for creating VMs for tasks.\nThese images are basically squashfs images with some additional metadata.\nThey can be prepared from docker images using this (gvmkit-build) tool.\n\n## Quick start\n\n1. Make sure your docker service is running and you have gvmkit-build installed\n\n2. Go to the folder with your dockerfile and run\n\n```docker build . -t my_image```\n\n3. Create account on registry portal https://registry.golem.network\n\nLet's assume your user name is golem\n\n4. Create repository on registry portal\n\nLet's assume you created repository named my_example\n\n5. Create and copy personal access token from registry portal\n\n6. Run (you will be asked for login and personal access token)\n\n```gvmkit-build my_image --push-to golem/my_example:latest```\n\n7. Your tag ```golem/my_example:latest``` is ready to use in one of Golem Network APIs\n\n## Naming image\n\nThe tool as main argument takes docker image name.\n\nDocker image name can be resolved using ImageId + tag or Repository name + tag.\nRepository name can be composed of maximum two parts: ```\u003cusername\u003e/\u003crepository\u003e```\n\nExamples:\n```python```  resolves to ```python:latest```\n```python:3.8``` resolves to ```python:3.8```\n```golemfactory/blender``` resolves to ```golemfactory/blender:latest```\nYou can also use image id instead of name, use ```docker image ls``` to find your image id.\n\nFollowing command will build image and create *.gvmi file in current directory.\n\n```\ngvmkit-build \u003cimage_name\u003e\n```\n\nIf image not exist locally the tool is trying to pull it from docker hub.\n\nTo use it in Golem Network, you have to upload it to registry portal.\n\nTo successfully add image to registry portal you have to name image accordingly or use\n```\ngvmkit-build \u003cuser_name\u003e/\u003cimage_name\u003e:\u003ctag\u003e --push\n```\nor if your local image name is not compatible use\n```\ngvmkit-build \u003cdocker_image_id\u003e --push-to \u003cuser_name\u003e/\u003cimage_name\u003e:\u003ctag\u003e\n```\n\n## Build process explained a bit\n\nTool is creating new container and is copying data from given image to new container.\nAfter copying is finished mksquashfs command is used inside tool container to create squashfs image.\nAfter adding metadata *.gvmi file is created and tool container removed.\nNote that tool image will stay downloaded on your machine for future use (it is quite small so no worry about disk space)\n\n## [Optional] - building squashfs-tools image\n\nIf you want to use your own tool without pulling from dockerhub:\nGo to squashfs-tools directory and run\n```\ndocker build . -t my_squash_fs_builder\n```\nadd environment variable to .env in folder where you run gvmkit-build\n```\nSQUASHFS_IMAGE_NAME=my_squash_fs_builder \n```\n\nFor managing docker images and containers bollard library is used. https://docs.rs/bollard/latest/bollard/\n\n## Troubleshooting login to registry portal\n\nThe tool is using https://registry.golem.network as default registry portal.\nYou can change this behaviour by setting `REGISTRY_URL` environment variable.\nYou should create account on registry portal and generate access token for your user.\nIf you really don't want to create account you can use anonymous upload (see section below about anonymous upload).\n\nThe tool will ask for login when --login --push or --push-to option is specified.\n\nFor storing login information rpassword library is used: https://docs.rs/crate/rpassword/latest\n\nOnly one instance of login/token is kept saved at \"gvmkit-build-rs/default\". So if you login with new user/token old pair will be forgotten.\n\nThis option will ask for login to registry portal\n```\ngvmkit-build --login\n```\nYou can use command for check if your login information is correct\n```\ngvmkit-build --login-check \n```\nIf you want to forget your login information you can use, it will clear your login information\n```\ngvmkit-build --logout \n```\n\nAbove command are optional, because you will be asked for login automatically if you use --push or --push-to option.\n\nOn some systems when no secure store is provided you will be forced to use following method of keeping login/token pair.\nYou can optionally put them in .env file from convenience in your working directory.\nRemember to set proper permissions to the file (chmod 600 .env) if using on shared machine.\n\n```\nREGISTRY_USER=\u003cusername-in-registry-portal\u003e\nREGISTRY_TOKEN=\u003caccess-token-generated-in-registry-portal\u003e\n```\n\nWhen REGISTRY_USER/REGISTRY_TOKEN is set in environment variables, secure rpassword storage won't be used.\n\n## Uploading to multiple tags\n\nThe tool cannot upload to multiple tags, but you can call it multiple times with different tags.\nAll steps of operations are cached so no worry about re-uploading same file multiple times.\n\n## Uploading large files\n\nThe tool is using chunked upload with default chunk size 10MiB for images greater than 500MiB (changing not recommended).\nFour upload workers are created by default (you can increase/decrease number of workers using --upload-workers argument depending on your network conditions). \nIf you think your upload is stuck you can always stop and run tool again to finish download. Only chunks that were not uploaded\nwill be uploaded again.\n\nNote: Total limit of chunks is set to 1000 (so around 10GB by default). If you want to upload larger file you have to set greater chunk size accordingly.\n\n## Uploading image without login\n\nYou can upload images anonymously. Note that lifetime of such images is limited, \nand they can be removed from registry portal after some time without notice\n\n```\ncargo run --release -- \u003cimage_name\u003e --push --nologin\n```\n\n## Uploading image without building part\n\nIf you are sure that you have proper *.gvmi file for example my-test.gvmi you can use \n\n```\ngvmkit-build --direct-file-upload my-test.gvmi --push-to \u003cuser_name\u003e/\u003cimage_name\u003e:\u003ctag\u003e\n```\nor anonymously\n```\ngvmkit-build --direct-file-upload my-test.gvmi --push --nologin\n```\n\n## Changing squashfs options when creating image\n\nYou can change compression used in mksquashfs to produce more or less compact images. \n\nLook for help for more information. Note that currently zstd is not supported by Golem Network (you can use xz instead for extra compact images).\n```\ngvmkit-build --help\n```\n\n## Integration with scripts\n\nUse --extra-json-info-path=my-output.json to save additional information about image in json format.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgolemfactory%2Fgvmkit-build-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgolemfactory%2Fgvmkit-build-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgolemfactory%2Fgvmkit-build-rs/lists"}