https://github.com/rjmacarthy/llamallamallama
Llamallamallama is a chat solution that allows users to chat with "Llama" fine tuned with LoRa on the "Alpaca" dataset.
https://github.com/rjmacarthy/llamallamallama
ai alpaca artificial-intelligence chatbot chatgpt huggingface-transformers llama machine-learning nlp python sveltekit typescript
Last synced: 3 months ago
JSON representation
Llamallamallama is a chat solution that allows users to chat with "Llama" fine tuned with LoRa on the "Alpaca" dataset.
- Host: GitHub
- URL: https://github.com/rjmacarthy/llamallamallama
- Owner: rjmacarthy
- License: mit
- Created: 2023-03-26T19:41:35.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2023-04-06T07:32:58.000Z (about 3 years ago)
- Last Synced: 2025-09-03T22:41:11.765Z (10 months ago)
- Topics: ai, alpaca, artificial-intelligence, chatbot, chatgpt, huggingface-transformers, llama, machine-learning, nlp, python, sveltekit, typescript
- Language: TypeScript
- Homepage:
- Size: 533 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# llamallamallama
llamallamallama is a chat solution that allows users to chat with "Llama" fine tuned with LoRa on the "Alpaca" dataset. The solution uses SvelteKit to stream responses from a Python API to the browser.
The AI model was trained on the alpaca dataset [Alpaca Dataset](https://github.com/tatsu-lab/stanford_alpaca).
The model streaming logic was inspired by [Alpaca-LoRA-Serve](https://github.com/deep-diver/Alpaca-LoRA-Serve).
The project uses sqllite to store chat history.
## Installation
Copy `backend/config.example.yml` to `config.yml` and edit for personal settings.
### Backend
To install the backend dependencies, run the following command:
`pip install -r requirements.txt` inside the backend directory.
You will need to add your models inside the `backend/models` folder if using locally to match `config.yml` or use Hugging Face.
```
.
├── llama-7b
└── llama-7b-ft
```
The fine tuned folder should have the `adapter_model.bin` and `adapter_config.json` files.
### Setup script
Run `./scripts/setup.sh` to copy the configuration file and install the front-end.
### Frontend
To install the frontend dependencies, run the following command:
`npm install`
### Docker
```
docker compose up -d
```
## Usage
### Backend
To start the backend, run the following command:
`uvicorn main:app --reload`
This will start the backend server and make it available at http://localhost:8000.
### Frontend
To start the frontend in development.mode, run the following command:
`npm run dev`
This will start the frontend server and make it available at http://localhost:5173. You can then open this URL in a web browser to access the chat interface.
## Contributing
We welcome contributions to the llamallamallama project!
## License
llamallamallama is licensed under the MIT. See the [LICENSE](LICENSE) file for details.
## Credits
llamallamallama was created by rjmacarthy with help from other repositories.
I will update this soon with all the credits.
## Todo
- Get model and weights from huggingface
- Docker
- Setup scripts
- Better UI
- Chat context
- Refactor
- Add tests
- Maintenance
- Better configuration options