{"id":19310304,"url":"https://github.com/cedadev/jaspy-manager","last_synced_at":"2025-04-22T13:33:54.358Z","repository":{"id":53107444,"uuid":"170859275","full_name":"cedadev/jaspy-manager","owner":"cedadev","description":"Conda environments for JASMIN (and beyond)","archived":false,"fork":false,"pushed_at":"2024-08-19T12:24:19.000Z","size":280,"stargazers_count":7,"open_issues_count":5,"forks_count":3,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-09-22T00:05:21.316Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cedadev.png","metadata":{"files":{"readme":"README-simple-make-new-env.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-15T12:08:04.000Z","updated_at":"2024-08-19T12:24:24.000Z","dependencies_parsed_at":"2024-08-20T23:03:43.519Z","dependency_job_id":null,"html_url":"https://github.com/cedadev/jaspy-manager","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedadev%2Fjaspy-manager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedadev%2Fjaspy-manager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedadev%2Fjaspy-manager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedadev%2Fjaspy-manager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cedadev","download_url":"https://codeload.github.com/cedadev/jaspy-manager/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223898941,"owners_count":17221855,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T00:23:35.318Z","updated_at":"2024-11-10T00:23:36.147Z","avatar_url":"https://github.com/cedadev.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Making new Jaspy environments\n\n## Overview\n\nThere are 2 different methods that we use for constructing a Jaspy environment.\nThe (1) _quick method_ pulls in the packages from Conda channels on the web.\nThe (2) _full method_ pulls in those packages, caches them on our server, and \ninstalls using only those files.\n\nMethod (2) is more reproducible, but we have found it is less reliable when we\ntry to build environments with many 100s of packages.\n\n## Methods of creating and installing Jaspy environments\n \n### 1. Quick method\n\nThe quick method includes the following steps:\n\n1. Decide on a miniconda version - ensure it is recorded in our config file.\n2. Download and install the miniconda version.\n3. Connect the `ceda-jaspy-envs` repository to the local repo.\n4. Create a name for the new environment.\n5. Create an initial recipe of software packages for the new environment.\n6. Run the `create-env-recipe.sh` script.\n7. Run the `create-env-recipe-jaspy-channel.sh` script.\n8. Push the changes to GitHub\n9. On the server where the env needs installing, run: `install-jaspy-env.sh`\n\n### 2. Full method\n\nThe full method is preferred, because it caches _all packages_ on our system, so that we can\ndefinitely reproduce the same environment over time:\n\n1. Decide on a miniconda version - ensure it is recorded in our config file.\n2. Download and install the miniconda version.\n3. Connect the `ceda-jaspy-envs` repository to the local repo.\n4. Create a name for the new environment.\n5. Create an initial recipe of software packages for the new environment.\n6. Run the `create-env-recipe.sh` script.\n7. Run the `create-env-recipe-jaspy-channel.sh` script.\n8. Push the changes to GitHub\n9. Copy the binaries of the packages to our Jaspy channel server, with `copy-to-jaspy-channel.sh`\n10. On the Jaspy channel server, index the new packages\n11. Adjust the `install-jaspy-env.sh` script to use the file containing our local URLs \n12. On the server where the env needs installing, run: `install-jaspy-env.sh` using our local URLs\n\n\n## Notes (that need tidying)\n\nThese notes outline how the process has been run in the past...kept for reference.\n\n```\nmkdir /tmp/test\ncd /tmp/test/\n\ngit clone https://github.com/cedadev/jaspy-manager\ncd jaspy-manager/\nexport JASPY_BASE_DIR=/tmp/jaspy-base\nmkdir -p $JASPY_BASE_DIR\n\n# Install miniconda\n./bin/install-miniconda.sh py3.7 m3-4.5.11\n\n./bin/clone-initial-env.sh environments/py3.7/m3-4.5.11/isc-env-r20181009 test-env-r20190725\n```\n\nTest we can build an env with a file:\n\nCreate an `initial.yml` file.\n\nBuild the associated files and check it works:\n\n```\n./bin/create-env-recipe.sh environments-ceda-jaspy-envs/r3.6/jasr3.6-m3-4.9.2-r20211105/initial.yml\n\n```\n\nIf it works, you can activate it with:\n\n```\nexport PATH=$JASPY_BASE_DIR/jaspy/miniconda_envs/jaspy3.7/m3-4.5.11/bin:$PATH\nsource activate test-env-r20190725\n```\n\nCreate URLs file that points to our own channel:\n\n```\n./bin/create-env-recipe-jaspy-channel.sh environments-ceda-jaspy-envs/r3.6/jasr3.6-m3-4.9.2-r20211105/packages.txt\n\n\n```\n\nCopy to dist server:\n\n```\n. ./setup-env.sh\n./bin/copy-to-jaspy-channel.sh environments-ceda-jaspy-envs/r3.6/jasr3.6-m3-4.9.2-r20211105/_urls.txt\n\n\n```\n\nOn dist server, index the channels. \n\n```\ncd jaspy-manager/\n./bin/index-channel.sh /datacentre/opshome/dist/htdocs/jaspy/jasr3.6\n\n```\n\n\nInstall on JASMIN:\n\n```\ncd jaspy-manager/\n./bin/install-jaspy-env jasr3.6-m3-4.9.2-r20211105\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcedadev%2Fjaspy-manager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcedadev%2Fjaspy-manager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcedadev%2Fjaspy-manager/lists"}