Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/yingding/applyllm

A python package for applying LLM with LangChain and Hugging Face on local CUDA/MPS host
https://github.com/yingding/applyllm

accelerator batch cuda framework inference kubeflow langchain llm mps pipeline slurm transformers

Last synced: about 2 months ago
JSON representation

A python package for applying LLM with LangChain and Hugging Face on local CUDA/MPS host

Awesome Lists containing this project

README 

        
# applyllm: python package

This repository contains code of [applyllm](https://pypi.org/project/applyllm/) python PyPI package, for loading and training open source llm models e.g. LlaMA2, Mixtral 8x7B, etc.



## Install the package

```shell

pip install applyllm

```



## Setup a local venv on Macosx Apple Silicon

```shell

python3 -m pip install --upgrade pip

python3 -m pip install --no-cache-dir -r ./llm-examples/setup/requirements310mac.txt

```



## Setup on Kubeflow notebook

```shell

python3 -m pip install --upgrade pip

python3 -m pip install --no-cache-dir --user --upgrade -r ./llm-examples/requirements.txt --extra-index-url https://download.pytorch.org/whl/cu118

```



## Build locally

```shell

VENV_NAME="llm3.10"

VENV_DIR="$HOME/VENV"

source ${VENV_DIR}/${VENV_NAME}/bin/activate;

make clean && make build && make reload

```



## Publish the test pypi package

```shell

VENV_NAME="llm3.10"

VENV_DIR="$HOME/VENV"

source ${VENV_DIR}/${VENV_NAME}/bin/activate;

make clean && make build && make testpypi

```

* `$HOME/.pypirc` shall be availabe, visit [Build python package docs](./BuildPackage.md) for details to create `$HOME/.pypirc` file to publish PyPI package

* visit https://test.pypi.org to see the test package published 



## Publish the pypi package

```shell

VENV_NAME="llm3.10"

VENV_DIR="$HOME/VENV"

source ${VENV_DIR}/${VENV_NAME}/bin/activate;

make clean && make build && make pypi

```

* `$HOME/.pypirc` shall be availabe, visit [Build python package docs](./BuildPackage.md) for details to create `$HOME/.pypirc` file to publish PyPI package

* visit https://pypi.org/ to see the package published



## Add a jupyter notebook kernel to VENV

```shell

VENV_NAME="llm3.10"

VENV_DIR="$HOME/VENV"

source ${VENV_DIR}/${VENV_NAME}/bin/activate;

python3 -m pip install --upgrade pip

python3 -m pip install ipykernel

deactivate

```



We need to reactivate the venv so that the ipython kernel is available after installation.

```shell

VENV_NAME="llm3.10"

VENV_DIR="$HOME/VENV"

source ${VENV_DIR}/${VENV_NAME}/bin/activate;

python3 -m ipykernel install --user --name=${VENV_NAME} --display-name ${VENV_NAME}

```

Note: 

* restart the vs code, to select the venv as jupyter notebook kernel



Reference:

* https://ipython.readthedocs.io/en/stable/install/kernel_install.html

* https://anbasile.github.io/posts/2017-06-25-jupyter-venv/



## Remove ipykernel

```shell

VENV_NAME="llm3.10"

jupyter kernelspec uninstall -y ${VENV_NAME}

```



## Remove all package from venv

```

python3 -m pip freeze | xargs pip uninstall -y

python3 -m pip list

```



## Sync AIMLflow

```shell

aim_repo_path=./aimruns

mlflow_uri=./mlruns

mkdir ${aim_repo_path}

cd ${aim_repo_path}

aim init

cd ..

# start a watch process

aimlflow sync --mlflow-tracking-uri=${mlflow_uri} --aim-repo=${aim_repo_path}

# start a second terminal

aim up --repo=${aim_repo_path}

```



## Restart MLflow UI

If the default port 5000 is used

```shell

ps -A | grep gunicorn

pkill -f gunicorn

ps -A | grep gunicorn

# use relative path to set the backend-store-uri, full path with file:///root/sub/mlruns

mlflow ui --backend-store-uri llm-examples/mlruns

# mlflow server --host 127.0.0.1 --port 8080

```



Reference:

* https://stackoverflow.com/questions/60531166/how-to-safely-shutdown-mlflow-ui/63141642#63141642

* mlflow backend store uri https://stackoverflow.com/questions/63525498/how-do-i-set-a-different-local-directory-for-mlflow



# Relevant Tech Info



* [LLM tech docs](./LLM.md)

* [Build python package docs](./BuildPackage.md)

* mlflow https://mlflow.org/docs/latest/llms/langchain/guide/index.html

* mlflow langchain flavour https://mlflow.org/docs/latest/llms/langchain/index.html

* AIM https://github.com/aimhubio/aim

* AIM, Weights And Biases https://medium.com/aimstack/aim-tutorial-for-weights-and-biases-users-bfbdde76f21e



## Makefile **space 4 issue

Makefile need tab as indent, not space 4

* https://stackoverflow.com/questions/16931770/makefile4-missing-separator-stop/72198029#72198029



## Reference:

* build python resource package .whl https://docs.gitlab.com/ee/user/packages/workflows/build_packages.html#pypi

* upload .whl to the package registry https://docs.gitlab.com/ee/user/packages/pypi_repository/#install-from-the-group-level

* python package release name convention https://peps.python.org/pep-0440/

* build package with sub package https://stackoverflow.com/questions/44834/what-does-all-mean-in-python/35710527#35710527

* Merge dictionary https://stackoverflow.com/questions/38987/how-do-i-merge-two-dictionaries-in-a-single-expression-in-python