https://github.com/j3soon/llm-tutorial
LLM tutorial materials include but not limited to NVIDIA NeMo, TensorRT-LLM, Triton Inference Server, and NeMo Guardrails.
https://github.com/j3soon/llm-tutorial
llm nemo nemo-guardrails nvidia-nemo tensorrt-llm
Last synced: 6 months ago
JSON representation
LLM tutorial materials include but not limited to NVIDIA NeMo, TensorRT-LLM, Triton Inference Server, and NeMo Guardrails.
- Host: GitHub
- URL: https://github.com/j3soon/llm-tutorial
- Owner: j3soon
- License: apache-2.0
- Created: 2024-07-19T17:04:10.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2024-09-19T02:21:48.000Z (about 1 year ago)
- Last Synced: 2025-03-24T10:21:19.300Z (7 months ago)
- Topics: llm, nemo, nemo-guardrails, nvidia-nemo, tensorrt-llm
- Language: Jupyter Notebook
- Homepage:
- Size: 3.07 MB
- Stars: 2
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# LLM Tutorial
LLM tutorial materials include but not limited to NVIDIA NeMo, TensorRT-LLM, Triton Inference Server, and NeMo Guardrails.
This material is used in the [NCHC LLM Bootcamp](https://www.openhackathons.org/s/siteevent/a0CUP00000L45Bx2AJ/se000345).
## Running on [TWCC](https://www.twcc.ai/)
Please follow this [TWCC README](./README_TWCC.md) to run the tutorials on TWCC.
## Running Locally
Install [docker](https://docs.docker.com/engine/install/ubuntu/) and [nvidia container toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). Then add your user to the docker group and re-login/restart.
```sh
git clone https://github.com/j3soon/LLM-Tutorial.git
cd LLM-Tutorial
# (a) NeMo
docker run --rm -it --gpus=all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -v $(pwd)/workspace:/workspace --network=host nvcr.io/nvidia/nemo:24.05
# in the container
jupyter lab
# open the notebook URL in your browser
# (b) TensorRT-LLM
docker run --rm -it --gpus=all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -v $(pwd)/workspace:/workspace --network=host nvcr.io/nvidia/tritonserver:24.05-trtllm-python-py3
# in the container
jupyter lab
# open the notebook URL in your browser
```
## Contributing
Make sure to run the following before committing:
```sh
pip install nb-clean
nb-clean clean workspace/NeMo_Training_TinyLlama.ipynb
nb-clean clean workspace/TensorRT-LLM.ipynb
nb-clean clean workspace/NeMo_Guardrails.ipynb
```
## Contributors
The code was primarily written by [Cliff](https://github.com/wcks13589), with assistance from others listed in the [contributor list](https://github.com/j3soon/LLM-Tutorial/graphs/contributors).
## Acknowledgements
We would like to thank [NVIDIA](http://www.nvidia.com/), [OpenACC](http://www.openacc.org/), and [NCHC](https://www.nchc.org.tw/?langid=2) (National Center for High-performance Computing) for making this bootcamp happen.
## References
- [openhackathons-org/End-to-End-LLM](https://github.com/openhackathons-org/End-to-End-LLM)