Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/bean980310/ai-companion

AI companions including generative AI such as chatbots, image generation, text generation, and audio generation.
https://github.com/bean980310/ai-companion

ai ai-translate chatbot generative-ai image-generation llm local-ai nlp stable-diffusion storytelling-generators text-generation text-to-image

Last synced: about 2 months ago
JSON representation

AI companions including generative AI such as chatbots, image generation, text generation, and audio generation.

Awesome Lists containing this project

README 

          
# AI Companion for Local Machines



![AI Companion](assets/images/ai-companion-2.png)



A Gradio-based AI companion app leveraging generative AI models (LLMs, Diffusion models, etc.) running in a local environment.



[English](README.md) [한국어](README_ko.md) [日本語](README_ja.md) [简体中文](README_zh_cn.md) [繁體中文](README_zh_tw.md)



## Overview



The AI Companion offers chatbot services along with AI-powered generation of images, videos, and audio content. Utilizing user personas and character personas, the app introduces persona-based chatbot interactions, transforming AI beyond mere tools into genuine friends and partners. Users can engage in conversations, collaborative tasks, and recreational activities with AI characters.



While the user persona mainly targets general users, the image, video, music, and audio generation services also cater to professional users.



## Side Project



[LLM Backend](https://github.com/bean980310/ai-companion-llm-backend.git)

[Image Backend](https://github.com/bean980310/ai-companion-image-backend.git)

[Langchain Integrator](https://github.com/bean980310/ai-companion-langchain-integrator.git)



## Main Features



### Chatbot



Interact with AI using Large Language Models (LLMs).



**Supported Models**  



* **API**  



|Provider|Model Name|

|-------|----|

|OpenAI|gpt-3.5-turbo 
 gpt-4 
 gpt-4-turbo 
 gpt-4o-mini 
 gpt-4o 
 gpt-4.1 
 gpt-4.1-mini 
 gpt-4.1-nano |

|Anthropic|claude-3-haiku-20240307 
 claude-3-sonnet-20240229 
 claude-3-opus-latest 
 claude-3-5-sonnet-latest 
 claude-3-5-haiku-latest 
 claude-3-7-sonnet-latest|

|Google GenAI|gemini-1.5-flash 
 gemini-1.5-flash-8b 
 gemini-1.5-pro 
 gemini-2.0-flash|



* **Local**: Transformers, GGUF, MLX (Apple Silicon Macs only)  

Pre-downloaded Transformer models are available via the Download Center.  



|Provider|Model Name|

|--------|-----|

|meta-llama|meta-llama/Llama-3.1-8B 
 meta-llama/Llama-3.1-8B-Instruct 
 meta-llama/Llama-3.2-11B-Vision 
 meta-llama/Llama-3.2-11B-Vision-Instruct|

|google|google/gemma-2-9b 
 google/gemma-2-9b-it 
 google/gemma-3-12b-pt 
 google/gemma-3-12b-it|

|Qwen|Qwen/Qwen2.5-7B 
 Qwen/Qwen2.5-7B-Instruct 
 Qwen/Qwen2.5-14B 
 Qwen/Qwen2.5-14B-Instruct|

|mistralai|mistralai/Mistral-7B-v0.3 
 mistralai/Mistral-7B-Instruct-v0.3 
 mistralai/Mistral-Small-24B-Base-2501 
 mistralai/Mistral-Small-24B-Instruct-2501|



**Character Persona Settings**  



* **System Prompt(System Message)**: Instructions that define the chatbot’s role or behavior based on user needs. (Default: You are an useful AI assistant.) Presets or characters automatically apply predefined prompts.

* **Character**: Choose your AI conversation partner.

* **Preset**: Applies user-defined system prompts. Presets automatically update when switching characters.



**Hyperparameter Settings**  

If you aren’t familiar with hyperparameters, it’s recommended to stick to the default values.



* **Seed**: Initial random number seed used in generation (default: 42).

* **Temperature**: Controls creativity/randomness of responses. Higher values yield more creative answers; lower values result in deterministic responses (default: 0.6).

* **Top K**: Limits the number of word options, choosing from the most probable words for high-quality outputs (default: 20).

* **Top P**: Adjusts randomness by setting a probability threshold, sampling from top tokens until the threshold is reached (default: 0.9).

* **Repetition Penalty**: Controls repetition of words; higher values reduce repetition (default: 1.1).



### Image Generation



Create images using models like Stable Diffusion and Flux. Images are generated via ComfyUI backend.



**Supported Models**  



* **API**

Currently, API image generation models have limited support.



|Developer|Model repo id|

|-------|----|

|OpenAI|dall-e-3|

|Google GenAI|imagen-3.0-generate-002|



* **Local**: Diffusers, Checkpoints

  * **Diffusers**: Currently, Diffusers models can be scanned and selected but cannot yet generate images (feature coming soon).

  * **Checkpoints**: Image generation via ComfyUI. Model files should be placed in models/checkpoints within the ComfyUI directory.



Supported Local Base Models:



* Stable Diffusion 1.5

* Stable Diffusion 2.x

* Stable Diffusion XL 1.0

* Stable Diffusion 3 Medium

* FLUX.1 Schnell

* FLUX.1 Dev

* Stable Diffusion 3.5 Large

* Stable Diffusion 3.5 Medium

* Illustrious XL 1.0



**Detailed options**  



* **LoRA**: Supports up to 10 LoRA models per base model. Compatible LoRAs must be applied.

* **VAE**: User-defined VAE. Defaults to embedded VAE within the Checkpoints.

* **Embedding**: Use via syntax like embedding:name.

* **ControlNet**: Interface not yet implemented; coming soon.

* **Refiner**: For Stable Diffusion XL 1.0, select the start step for refiner sampling.



**Generation Options**  



* **Positive Prompt**: Generate images based on the input text.

* **Negative Prompt**: Exclude unwanted features from generated images.

* **Width, Height**: Adjust image dimensions.

* **Recommended Resolutions**



|Base Model|Recommended Resolution|

|----------|----------|

|Stable Diffusion 1.5 
 Stable Diffusion 2.x|512x512 
 512x768 
 768x512|

|Stable Diffusion XL 1.0 
 Stable Diffusion 3 Medium 
 FLUX.1 Schnell 
 FLUX.1 Dev 
 Stable Diffusion 3.5 Large 
 Stable Diffusion 3.5 Medium|1024x1024 
 896x1152 
 1152x896 
 832x1216 
 1216x832 
 768x1344 
 1344x768 
 640x1536 
 1536x640|

|Illustrious XL 1.0|1536x1536 
 1248x1824 
 1824x1248|



* **Generation Steps**: Higher values result in longer generation times but potentially better quality.

* **Denoise Strength**: Adjusts the level of noise.



**Advanced Settings**  



* **Sampler**: Different samplers yield varying outputs.

* **Scheduler**: Affects output even with the same sampler/prompt.

* **CFG Scale**: Higher CFG values closely follow prompts; lower values are more creative.

* **Seed**: Initial random number seed.

* **Clip Skip**: Skips certain steps during image creation.

* **Batch Size**: Number of images generated per run.



**Image to Image**  

Modify existing images or use Inpaint for selective alterations.



### Storyteller



Utilizes LLMs optimized for text creation, especially for storytelling. (Currently under development.)



**Supported Models**  

Same as Chatbot.



**Hyperparameters**  

Same as Chatbot.  

If you are not familiar with how hyperparameters work, we recommend leaving them at the default values.



### Video Generation



Coming Soon



### Audio Generation



Coming Soon



### Translator



Multi-language translation using translation models. Supports text extraction from uploaded images or PDF files for translation.



## Installation



**Clone a repository**  



```shell

# Clone a repository only

git clone https://github.com/bean980310/ai-companion.git

# Clone a repository with submodules

git clone --recursive https://github.com/bean980310/ai-companion.git

# Init and Update a submodules

git submodule init

git submodule update

```



* **Virtual Environment Setup**



```shell

# conda (Recommended!)

# Python 3.10

conda create -n ai-companion python=3.10

# Python 3.11

conda create -n ai-companion python=3.11

# Python 3.12

conda create -n ai-companion python=3.12

conda activate ai-companion

# Other virtual environments

cd ai-companion

# venv

python3 -m venv venv

# uv

uv venv --python 3.10 

uv venv --python 3.11

uv venv --python 3.12

# MacOS/Linux/Windows WSL2

source venv/bin/activate # venv

source .venv/bin/activate # uv

# Windows

.\venv\Scripts\activate.bat # venv

.\.venv\Scripts\activate.bat # uv

```



**Install dependencies**  



* **Windows**



```shell

# on Windows

.\installer_windows_amd64.bat

```



```pwsh

# on Windows (Powershell)

.\installer_windows_amd64.ps1

```



```bash

# on Windows Subsystem for Linux 2

bash installer_windows_amd64_wsl2.sh

# or

./installer_windows_amd64_wsl2.sh

```



* **macOS(Apple Silicon Mac)**



```zsh

zsh installer_macos_arm64.sh

# or

./installer_macos_arm64.sh

```



* **Linux**



```bash

bash installer_linux_amd64_cuda.sh

# or

./installer_linux_amd64_cuda.sh

```



* **MeloTTS(Optional)**



```shell

pip install git+https://github.com/myshell-ai/MeloTTS.git --no-deps

```



### Installing xformers on Mac



```zsh

brew update

brew install gcc cmake llvm libomp

```



```zsh

# Case 1

export PATH="/opt/homebrew/opt/llvm/bin:$PATH"



export LDFLAGS="-L/opt/homebrew/opt/libomp/lib"

export CPPFLAGS="-I/opt/homebrew/opt/libomp/include"

export CXXFLAGS="-Xpreprocessor -fopenmp"



export CC=/opt/homebrew/opt/llvm/bin/clang

export CXX=/opt/homebrew/opt/llvm/bin/clang++

export LDFLAGS="-L/opt/homebrew/opt/llvm/lib"

export CPPFLAGS="-I/opt/homebrew/opt/llvm/include"

pip install --upgrade xformers



# Case 2

PATH="/opt/homebrew/opt/llvm/bin:$PATH" LDFLAGS="-L/opt/homebrew/opt/libomp/lib" CPPFLAGS="-I/opt/homebrew/opt/libomp/include" CXXFLAGS="-Xpreprocessor -fopenmp" CC=/opt/homebrew/opt/llvm/bin/clang CXX=/opt/homebrew/opt/llvm/bin/clang++ LDFLAGS="-L/opt/homebrew/opt/llvm/lib" CPPFLAGS="-I/opt/homebrew/opt/llvm/include" pip install --upgrade xformers

```



## Run



```zsh

python app.py

```



### Support for Intel Macs



AI Companion for Local Machines no longer supports Intel CPU-based Macs.  

If you are using an Intel CPU-based Macs, we recommend that you consider migrating to an Apple Silicon Based Macs or a Windows PC or Linux machine with an Nvidia GPU environment. If you have difficulty migrating from an Intel CPU-based Macs, you can use a companion application that supports Intel CPU-based Macs instead.



### Guidance for Windows Systems with GPUs That Do Not Support CUDA 12.4 or Higher



AI Companion for Local Machines provides support for dependency installation to ensure compatibility between PyTorch and xformers.  

On Windows, versions of CUDA lower than 12.4 cannot install the latest xformers. Therefore, for CUDA 11.8 and 12.1 environments, the PyTorch version is fixed at **2.4.0** to ensure compatibility with the latest supported xformers version, **xformers 0.0.27.post2**.  

Please note that future support for Windows systems with CUDA versions below 12.4 will be discontinued.  

If you are using Windows and have a CUDA version below 12.4, we recommend upgrading to CUDA 12.4 or higher and reinstalling PyTorch and xformers accordingly. If your GPU does not support CUDA 12.4 or higher, we recommend upgrading to a compatible GPU or considering migration to a newer PC.  



### Support for Windows



AI Companion for Local Machines is optimized for UNIX/Linux kernel-based operating systems. While it can be used on Windows, GPU acceleration is unavailable when running directly on Windows. To properly use AI Companion for Local Machines on Windows, we recommend using it within a WSL2 environment.