{"id":15925819,"url":"https://github.com/yingding/applyllm","last_synced_at":"2025-08-24T22:33:34.225Z","repository":{"id":220390105,"uuid":"751483992","full_name":"yingding/applyllm","owner":"yingding","description":"A python package for applying LLM with LangChain and Hugging Face on local CUDA/MPS host","archived":false,"fork":false,"pushed_at":"2024-09-04T14:43:49.000Z","size":601,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-10T05:51:47.394Z","etag":null,"topics":["accelerator","batch","cuda","framework","inference","kubeflow","langchain","llm","mps","pipeline","slurm","transformers"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/applyllm/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yingding.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-01T17:44:36.000Z","updated_at":"2024-11-05T13:19:02.000Z","dependencies_parsed_at":"2024-02-05T16:26:11.024Z","dependency_job_id":"3d1d1e77-e104-4a79-b328-e974a7bb1306","html_url":"https://github.com/yingding/applyllm","commit_stats":{"total_commits":274,"total_committers":3,"mean_commits":91.33333333333333,"dds":"0.014598540145985384","last_synced_commit":"ec01316e2f617d4aced440ef0e936a3708d3796b"},"previous_names":["yingding/applyllm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yingding%2Fapplyllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yingding%2Fapplyllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yingding%2Fapplyllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yingding%2Fapplyllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yingding","download_url":"https://codeload.github.com/yingding/applyllm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230848326,"owners_count":18289481,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accelerator","batch","cuda","framework","inference","kubeflow","langchain","llm","mps","pipeline","slurm","transformers"],"created_at":"2024-10-06T22:04:40.904Z","updated_at":"2025-08-24T22:33:34.178Z","avatar_url":"https://github.com/yingding.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# applyllm: python package\nThis repository contains code of [applyllm](https://pypi.org/project/applyllm/) python PyPI package, for loading and training open source llm models e.g. LlaMA2, Mixtral 8x7B, etc.\n\n## Install the package\n```shell\npip install applyllm\n```\n\n## Setup a local venv on Macosx Apple Silicon\n```shell\npython3 -m pip install --upgrade pip\npython3 -m pip install --no-cache-dir -r ./llm-examples/setup/requirements310mac.txt\n```\n\n## Setup on Kubeflow notebook\n```shell\npython3 -m pip install --upgrade pip\npython3 -m pip install --no-cache-dir --user --upgrade -r ./llm-examples/requirements.txt --extra-index-url https://download.pytorch.org/whl/cu118\n```\n\n## Build locally\n```shell\nVENV_NAME=\"llm3.10\";\nVENV_DIR=\"$HOME/VENV\";\nsource ${VENV_DIR}/${VENV_NAME}/bin/activate;\nmake clean \u0026\u0026 make build \u0026\u0026 make reload\n```\n\n## Publish the test pypi package\n```shell\nVENV_NAME=\"llm3.10\";\nVENV_DIR=\"$HOME/VENV\";\nsource ${VENV_DIR}/${VENV_NAME}/bin/activate;\nmake clean \u0026\u0026 make build \u0026\u0026 make testpypi\n```\n* `$HOME/.pypirc` shall be availabe, visit [Build python package docs](./BuildPackage.md) for details to create `$HOME/.pypirc` file to publish PyPI package\n* visit https://test.pypi.org to see the test package published \n\n## Publish the pypi package\n```shell\nVENV_NAME=\"llm3.10\";\nVENV_DIR=\"$HOME/VENV\";\nsource ${VENV_DIR}/${VENV_NAME}/bin/activate;\nmake clean \u0026\u0026 make build \u0026\u0026 make pypi\n```\n* `$HOME/.pypirc` shall be availabe, visit [Build python package docs](./BuildPackage.md) for details to create `$HOME/.pypirc` file to publish PyPI package\n* visit https://pypi.org/ to see the package published\n\n\n## Add a jupyter notebook kernel to VENV\n```shell\nVENV_NAME=\"llm3.10\";\nVENV_DIR=\"$HOME/VENV\";\nsource ${VENV_DIR}/${VENV_NAME}/bin/activate;\npython3 -m pip install --upgrade pip\npython3 -m pip install ipykernel\ndeactivate\n```\n\nWe need to reactivate the venv so that the ipython kernel is available after installation.\n```shell\nVENV_NAME=\"llm3.10\"\nVENV_DIR=\"$HOME/VENV\"\nsource ${VENV_DIR}/${VENV_NAME}/bin/activate;\npython3 -m ipykernel install --user --name=${VENV_NAME} --display-name ${VENV_NAME}\n```\nNote: \n* restart the vs code, to select the venv as jupyter notebook kernel\n\nReference:\n* https://ipython.readthedocs.io/en/stable/install/kernel_install.html\n* https://anbasile.github.io/posts/2017-06-25-jupyter-venv/\n\n## Remove ipykernel\n```shell\nVENV_NAME=\"llm3.10\"\njupyter kernelspec uninstall -y ${VENV_NAME}\n```\n\n## Remove all package from venv\n```\npython3 -m pip freeze | xargs pip uninstall -y\npython3 -m pip list\n```\n\n## Sync AIMLflow\n```shell\naim_repo_path=./aimruns\nmlflow_uri=./mlruns\nmkdir ${aim_repo_path}\ncd ${aim_repo_path}\naim init\ncd ..\n# start a watch process\naimlflow sync --mlflow-tracking-uri=${mlflow_uri} --aim-repo=${aim_repo_path}\n# start a second terminal\naim up --repo=${aim_repo_path}\n```\n\n## Restart MLflow UI\nIf the default port 5000 is used\n```shell\nps -A | grep gunicorn\npkill -f gunicorn\nps -A | grep gunicorn\n# use relative path to set the backend-store-uri, full path with file:///root/sub/mlruns\nmlflow ui --backend-store-uri llm-examples/mlruns\n# mlflow server --host 127.0.0.1 --port 8080\n```\n\nReference:\n* https://stackoverflow.com/questions/60531166/how-to-safely-shutdown-mlflow-ui/63141642#63141642\n* mlflow backend store uri https://stackoverflow.com/questions/63525498/how-do-i-set-a-different-local-directory-for-mlflow\n\n\n# Relevant Tech Info\n\n* [LLM tech docs](./LLM.md)\n* [Build python package docs](./BuildPackage.md)\n* mlflow https://mlflow.org/docs/latest/llms/langchain/guide/index.html\n* mlflow langchain flavour https://mlflow.org/docs/latest/llms/langchain/index.html\n* AIM https://github.com/aimhubio/aim\n* AIM, Weights And Biases https://medium.com/aimstack/aim-tutorial-for-weights-and-biases-users-bfbdde76f21e\n\n\n## Makefile **space 4 issue\nMakefile need tab as indent, not space 4\n* https://stackoverflow.com/questions/16931770/makefile4-missing-separator-stop/72198029#72198029\n\n\n## Reference:\n* build python resource package .whl https://docs.gitlab.com/ee/user/packages/workflows/build_packages.html#pypi\n* upload .whl to the package registry https://docs.gitlab.com/ee/user/packages/pypi_repository/#install-from-the-group-level\n* python package release name convention https://peps.python.org/pep-0440/\n* build package with sub package https://stackoverflow.com/questions/44834/what-does-all-mean-in-python/35710527#35710527\n* Merge dictionary https://stackoverflow.com/questions/38987/how-do-i-merge-two-dictionaries-in-a-single-expression-in-python","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyingding%2Fapplyllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyingding%2Fapplyllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyingding%2Fapplyllm/lists"}