https://github.com/agencypmg/self-hosting-large-language-models

https://github.com/agencypmg/self-hosting-large-language-models

Last synced: 3 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/agencypmg/self-hosting-large-language-models
• Owner: AgencyPMG
• Created: 2024-04-03T14:15:16.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-04-24T10:33:03.000Z (about 1 year ago)
• Last Synced: 2025-01-25T04:11:54.400Z (5 months ago)
• Language: HCL
• Size: 672 KB
• Stars: 8
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Self Hosting Large Langauge Models

This is some code for a talk at the AI Camp meetups in Dallas in Austin, TX.

## Getting Started

Run `./bin/dev/up` to setup the environment and download the GPT2 model.

Then run `source .venv/bin/activate` to activate the virtual environment.

## Pytorch is Big

That's why there's this bit in `requirements.txt`

```

--extra-index-url https://download.pytorch.org/whl/cpu

```

Which puts torch in CPU only mode, which is fine for this demo. With a GPU

backed instance, you may want to include CUDA.