{"id":23074175,"url":"https://github.com/rhecosystemappeng/agent-morpheus-models","last_synced_at":"2025-06-24T07:40:09.918Z","repository":{"id":267247584,"uuid":"900476221","full_name":"RHEcosystemAppEng/agent-morpheus-models","owner":"RHEcosystemAppEng","description":"This Repository contains an umbrella chart to install nim embedding model and one LLM out of 2 possible LLMs.","archived":false,"fork":false,"pushed_at":"2025-02-24T22:03:41.000Z","size":81,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-02-24T23:19:37.872Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Smarty","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RHEcosystemAppEng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-08T21:26:17.000Z","updated_at":"2025-02-24T22:03:45.000Z","dependencies_parsed_at":"2024-12-09T09:31:04.336Z","dependency_job_id":"912e36a2-f12f-480f-a43d-f0e488592cb2","html_url":"https://github.com/RHEcosystemAppEng/agent-morpheus-models","commit_stats":null,"previous_names":["rhecosystemappeng/agent-morpheus-models"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fagent-morpheus-models","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fagent-morpheus-models/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fagent-morpheus-models/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fagent-morpheus-models/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RHEcosystemAppEng","download_url":"https://codeload.github.com/RHEcosystemAppEng/agent-morpheus-models/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246992496,"owners_count":20865823,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-16T08:22:18.578Z","updated_at":"2025-04-03T11:29:55.450Z","avatar_url":"https://github.com/RHEcosystemAppEng.png","language":"Smarty","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Agent Morpheus Self hosted Models.\n\nThis repo contains umbrella helm chart to install embedding model nim-embed for creating embeddings to store in VDB and one of the\nfollowing LLM:\n\n1. llama3.1-70b-instruct-4bit\n2. nim llama3.1-8b-instruct(16 bit quantization).\n\n## Deploying the chart\n1. Create target namespace to install on it all models.\n```shell\noc new-project agent-morpheus-models\n```\n2. Type in your NGC_API_KEY ( get one [here](https://docs.nvidia.com/ngc/gpu-cloud/ngc-user-guide/index.html#generating-api-key))\n```shell\nexport NGC_API_KEY=your_api_key_goes_here\n```\n\n3. Replace placeholder password with your real API Key\n```shell\nsed -E 's/ \\\u0026ngc-api-key changeme/ \\\u0026ngc-api-key '$NGC_API_KEY'/' agent-morpheus-models/values.yaml  \u003e agent-morpheus-models/yourenv_values.yaml\n```\n\n4. Deploying both LLMs together is not possible, when trying doing so, you'll get an error from  the chart installation:\n```shell\nhelm install --set llama3_1_70b_instruct_4bit.enabled=true --set nim_llm.enabled=true  agent-morpheus-models agent-morpheus-models/ -f agent-morpheus-models/yourenv_values.yaml\n```\nOutput:\n```shell\nError: INSTALLATION FAILED: execution error at (agent-morpheus-models/templates/configmap.yaml:6:3): Only one of models should be deployed!, either llama3_1_70b_instruct_4bit or nim_llm 8b, but not both!\n```\n\n5. Deploy the chart with one of the two possible combinations:\n```shell\n# Deploy with LLM llama3.1-70b-instruct-4bit\n helm install agent-morpheus-models agent-morpheus-models/ -f agent-morpheus-models/yourenv_values.yaml\n# Or Deploy with LLM meta/llama3.1-8b-instruct ( 16bit quantization)\n helm install --set llama3_1_70b_instruct_4bit.enabled=false --set nim_llm.enabled=true  agent-morpheus-models agent-morpheus-models/ -f agent-morpheus-models/yourenv_values.yaml\n```\nOutput:\n```shell\nNAME: agent-morpheus-models\nLAST DEPLOYED: Sun Dec  8 23:05:14 2024\nNAMESPACE: test-models\nSTATUS: deployed\nREVISION: 1\nTEST SUITE: None\nNOTES:\nSend a prompt to the model to test it works:\noc wait --for=condition=ready pod -l component=llama3.1-70b-instruct  --timeout 1000s\ncurl -X POST -H \"Content-Type: application/json\" http://llama3-1-70b-instruct-4bit-agent-morpheus-models.apps.ai-dev03.kni.syseng.devcluster.openshift.com/v1/chat/completions -d @$(git rev-parse --show-toplevel)/agent-morpheus-models/files/70b-4bit-input-example.json | jq .\n```\n\n6. Wait for LLM pod to be ready, and then send an example request to the LLM, in order to get output\n```shell\noc wait --for=condition=ready pod -l component=llama3.1-70b-instruct  --timeout 1000s\ncurl -X POST -H \"Content-Type: application/json\" http://llama3-1-70b-instruct-4bit-agent-morpheus-models.apps.ai-dev03.kni.syseng.devcluster.openshift.com/v1/chat/completions -d @$(git rev-parse --show-toplevel)/agent-morpheus-models/files/70b-4bit-input-example.json | jq .\n```\n\n7. Whenever finishing with models , and wants to free up resources,  you can delete the chart\n```shell\nhelm uninstall agent-morpheus-models\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhecosystemappeng%2Fagent-morpheus-models","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frhecosystemappeng%2Fagent-morpheus-models","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhecosystemappeng%2Fagent-morpheus-models/lists"}