https://github.com/shreyanmitra/candyllm
A simple, easy-to-use framework for HuggingFace and OpenAI text-generation models focused on explainability.
https://github.com/shreyanmitra/candyllm
alpaca falcon gpt-4 huggingface llama3 llm-training llms mistral openai transformers vicuna
Last synced: 6 months ago
JSON representation
A simple, easy-to-use framework for HuggingFace and OpenAI text-generation models focused on explainability.
- Host: GitHub
- URL: https://github.com/shreyanmitra/candyllm
- Owner: shreyanmitra
- License: bsd-3-clause
- Created: 2024-07-09T08:16:45.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2024-09-10T01:49:47.000Z (9 months ago)
- Last Synced: 2024-12-19T06:42:18.443Z (6 months ago)
- Topics: alpaca, falcon, gpt-4, huggingface, llama3, llm-training, llms, mistral, openai, transformers, vicuna
- Language: Python
- Homepage:
- Size: 89.8 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
Awesome Lists containing this project
README
# CandyLLM 🍬
A simple, easy-to-use framework for HuggingFace and OpenAI text-generation models. The goal is to eventually integrate other sources such as custom large language models (LLMs) as well to create a coherent UI.
Unlike other libraries like Langchain, the implementation of the wrapper is *all in one page*, making it easy to read and modify! There is also a focus towards explainability inherently built-in to the UI and the code.
This is a work-in-progress, so pull-requests and issues are welcome! We try to keep it as stable as possible though, so people installing this library do not have any problems.
NOTE: Claude and Gemini are not supported yet.
If you use this library, please cite Shreyan Mitra. The library is also forked and contributed to by members of AIEA Lab at the University of California, Santa Cruz.
With all the administrivia out of the way, here are some examples of how to use the library. We are still setting up the official documentation. The following examples show some use cases, or tasks, and how an user of llm-wrapper would invoke the model of their choice.
## Install package
```
pip install CandyLLM
```
## Task: Fetch Llama3-8b and run it with default parameters on a simple QA Prompt without retrieval augmented generation
```python
from CandyLLM import*
myLLM = LLMWrapper("MY_HF_TOKEN", testing=False)
myLLM.answer("What is the capital of Uzbekistan?") #Returns Tashkent
```
This behavior is due to the fact that the default model is Llama3-8b
## Task: Fetch Llama2-7b and run it with tempereature = 0.6 on an QA Prompt with retrieval augmented generation
```python
from CandyLLM import*
myLLM = LLMWrapper("MY_HF_TOKEN", testing=False, modelName = "Llama2-7b") #or myLLM = LLMWrapper("MY_HF_TOKEN", testing=False, modelName = "meta-llama/Llama-2-7b-chat-hf", modelNameType="path")
myLLM.answer("What is the capital of Funlandia?", task="QAWithRAG", "The capital of Funlandia is Funtown", temperature=0.6) #Returns Funtown
```
## Task: Fetch GPT-4 and run it with presence_penalty = 0.5 on an Open-Ended Prompt
```python
from CandyLLM import*
myLLM = LLMWrapper("MY_OPENAI_TOKEN", testing=False, source="OpenAI", modelName = "gpt-4-turbo", modelNameType="path")
myLLM.answer("Write a creative essay about sustainability", task="Open-ended", presence_penalty=0.5)
```
## Log out of HuggingFace and OpenAI and remove my API keys from the environment
```python
myLLM = LLMWrapper(...) #Create some LLM wrapper
myLLM.answer(...) #Do something with the LLM
myLLM.logout()
```
## Check for malicious input prompts
```python
LLMWrapper.promptSafetyCheck("Is 1010 John Doe's social security number?") #Returns false to indicate unsafe prompt
```
## Change Config
Want to use a different model? No need to create another wrapper.
```python
myLLM = LLMWrapper(...) #Create some LLM wrapper
myLLM.setConfig("MY_TOKEN", testing = False, source="HuggingFace", modelName = "Mistral", modelNameType = "alias") #Tada: a changed LLM wrapper
```
## Dummy LLM
Sometimes, you don't want to spend the time and money to make api calls to an actual LLM, especially if you are testing an UI or an integration of a chat service. Dummy LLMs to the rescue! Our dummy LLM is called "Useless" and it will return answers immediately with very little computation spent (granted, the results it gives are useless - but, hey, what did you expect? 😃)
## CandyUI
CandyUI is the user interface of CandyLLM. It provides a chatbot, a dropdown for choosing the LLM to use, parameter configs for the LLM, and the option to apply post-hoc and pre-hoc methods to the user prompt and LLM output. CandyUI can be integrated into and communicate with a larger UI with custom functions, or you can use the ``selfOutput`` option for the custom post-hoc metrics to be displayed within CandyUI itself.
For example, running
```python
def postprocess(message, response):
#Sample postprocessor_fn which just returns the difference in length between LLM response and user prompt
return len(response) - len(message)
x = LLMWrapper.getUI(postprocessor_fn = postprocess, selfOutput = True, selfOutputLabel = "Length Difference")
```
deploys the following webpage:
You can also change how the output is shown. For example, for explainability purposes, you might want to set ```selfOutputType = "HighlightedText"```:
```python
def postprocess(message, response):
#Randomly assigns importance scores to words in the user prompt
importantWords = []
for word in message.split():
importantWords.append((word, "important")) if len(word) > 3 else importantWords.append((word, "unimportant"))
return importantWords
x = LLMWrapper.getUI(postprocessor_fn = postprocess, selfOutput = True, selfOutputLabel = "Important Words", selfOutputType = "HighlightedText")
```
The UI now looks like this:
