Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/nuhmanpk/quick-llama
Run Ollama models anywhere easily
https://github.com/nuhmanpk/quick-llama
colab langchain-python llama llm llm-agents llm-serving ollama ollama-api ollama-client ollama-python open-ai pypi
Last synced: about 7 hours ago
JSON representation
Run Ollama models anywhere easily
- Host: GitHub
- URL: https://github.com/nuhmanpk/quick-llama
- Owner: nuhmanpk
- License: mit
- Created: 2024-12-12T10:11:13.000Z (10 days ago)
- Default Branch: main
- Last Pushed: 2024-12-14T14:13:11.000Z (8 days ago)
- Last Synced: 2024-12-14T14:18:22.758Z (8 days ago)
- Topics: colab, langchain-python, llama, llm, llm-agents, llm-serving, ollama, ollama-api, ollama-client, ollama-python, open-ai, pypi
- Language: Python
- Homepage: https://pypi.org/project/quick-llama
- Size: 312 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Quick Llama
[](https://badge.fury.io/py/quick-llama)
[](https://pepy.tech/project/quick-llama)
[](https://opensource.org/licenses/MIT)
A Python wrapper for Ollama that simplifies managing and interacting with LLMs.
QuickLlama automates server setup, model management, and seamless interaction with LLMs, providing an effortless developer experience.
🚀 Colab-Ready: Easily run and experiment with QuickLlama on Google Colab for hassle-free, cloud-based development!
> **Note**: Don’t forget to use a GPU if you actually want it to perform well!
## Installtion
```py
pip install quick-llama
```
```py
from quick_llama import QuickLlama
from ollama import chat
from ollama import ChatResponse
# Defaults to mistral
quick_llama = QuickLlama(model_name="llama3.2:1b",verbose=False)
quick_llama.init()
response: ChatResponse = chat(model='llama3.2:1b', messages=[
{
'role': 'user',
'content': 'Why is the sky blue?',
},
])
print(response['message']['content'])
# or access fields directly from the response object
print(response.message.content)
quick_llama.stop_server()
```
```py
from quick_llama import QuickLlama
from ollama import chat
from ollama import ChatResponse
# Defaults to mistral
quick_llama = QuickLlama(model_name="llama3.2:1b")
quick_llama.init()
response: ChatResponse = chat(model='llama3.2:1b', messages=[
{
'role': 'user',
'content': 'what is 6 times 5?',
},
])
print(response['message']['content'])
print(response.message.content)
```
## Use with Langchain
```py
from quick_llama import QuickLlama
from langchain_ollama import OllamaLLM
model_name = "llama3.2:1b"
quick_llama = QuickLlama(model_name=model_name,verbose=True)
quick_llama.init()
model = OllamaLLM(model=model_name)
model.invoke("Come up with 10 names for a song about parrots")
```
## Use custom Models
```py
quick_llama = QuickLlama() # Defaults to mistral
quick_llama.init()
# Custom Model
# Supports all models from https://ollama.com/search
quick_llama = QuickLlama(model_name="custom-model-name")
quick_llama.init()
```
## List Models
```py
quick_llama.list_models()
```
## Stop Model
```py
quick_llama.stop_model("llama3.2:1b")
```
## Stop Server
```py
quick_llama.stop_server()
```
Made with ❤️ by [Nuhman](https://github.com/nuhmanpk). Happy Coding 🚀