https://github.com/load1n9/chat
leverage llama3.2 and other large language models to generate responses to your questions locally with no installation
https://github.com/load1n9/chat
chatgpt deno gpt huggingface huggingface-transformers javascript large-language-models llama llama3 llama3-1 llama3-2 llm transformer typescript
Last synced: 5 months ago
JSON representation
leverage llama3.2 and other large language models to generate responses to your questions locally with no installation
- Host: GitHub
- URL: https://github.com/load1n9/chat
- Owner: load1n9
- License: mit
- Created: 2024-10-01T06:46:37.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-10-07T01:55:52.000Z (over 1 year ago)
- Last Synced: 2025-04-05T12:07:26.557Z (about 1 year ago)
- Topics: chatgpt, deno, gpt, huggingface, huggingface-transformers, javascript, large-language-models, llama, llama3, llama3-1, llama3-2, llm, transformer, typescript
- Language: TypeScript
- Homepage: https://jsr.io/@loading/chat
- Size: 143 KB
- Stars: 2
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# [Chat](https://jsr.io/@loading/chat)
## Simply run the following command and thats it
```sh
deno run -A jsr:@loading/chat
```
## [Optional] create a `chat-config.toml` file in the active directory to configure the chat
```toml
"$schema" = 'https://jsr.io/@loading/chat/0.1.16/config-schema.json'
[config]
model = "onnx-community/Llama-3.2-1B-Instruct"
system = [
"You are an assistant designed to help with any questions the user might have."
]
max_new_tokens = 128
max_length = 20
temperature = 1.0
top_p = 1.0
repetition_penalty = 1.2
```
## Run the server to kinda match a similar api to the openai chat api
```sh
deno serve -A jsr:@loading/chat/server
```
### Try it out
```sh
curl -X POST http://localhost:8000/v1/completions \ -H "Content-Type: application/json" \ -d '{ "prompt": "Once upon a time", "max_tokens": 50, "temperature": 0.7 }'
```
## (New) Code Companion
With the new code companion you can generate new projects and edit them and
stuff
```sh
deno run -A jsr:@loading/chat/companion
```
type /help to get a list of commands
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
for details.