https://github.com/bitswired/semantic-caching

Explore semantic caching to reduce your OpenAI/LLM API bill
https://github.com/bitswired/semantic-caching

Last synced: about 1 month ago
JSON representation

Explore semantic caching to reduce your OpenAI/LLM API bill

• Host: GitHub
• URL: https://github.com/bitswired/semantic-caching
• Owner: bitswired
• Created: 2023-06-16T21:51:08.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-07-21T19:46:44.000Z (about 1 year ago)
• Last Synced: 2024-06-03T09:48:03.213Z (4 months ago)
• Language: Python
• Size: 70.3 KB
• Stars: 10
• Watchers: 1
• Forks: 2
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• jimsghstars - bitswired/semantic-caching - Explore semantic caching to reduce your OpenAI/LLM API bill (Python)

README

        
# Semantic Caching Evaluation 🚀

Explore semantic caching to reduce your OpenAI/LLM API bill

This repository contains a Python application that demonstrates the use of semantic caching in searching for similar questions in a cache. It compares the performance of two different embedding methods: OpenAI and ONNX.

## Features 🌟

- Streamlit web application to test and evaluate semantic caching.

- CLI for testing exact, semantic, and no cache.

- ONNX and OpenAI embeddings.

- FAISS search for fast similarity search.

## Installation 🛠️

To install this project, you need to have Python 3.10 installed. Then, follow these steps:

1. Clone the repository

2. Enter the project directory

3. Install the project: `poetry install`

4. Set up your OpenAI API key in the `.env` file.

## Usage 🎮

### CLI

To run the CLI, use the following command:

```bash

poetry run cli run 

```

Replace `` with `no_cache`, `semantic_cache`.

### Streamlit Web App

To run the Streamlit web app, use the following command:

```bash

poetry run webapp

```

The app will be available at `localhost:8501`.

## Project Structure 📁

- `pyproject.toml`: TOML file that contains the project metadata and dependencies.

- `scripts/`: Folder containing the Streamlit app and CLI scripts.

- `semantic_caching/`: Folder containing the core caching logic.

- `cache/`: Folder to store cache files (FAISS indices and SQLite databases).

## Dependencies 📚

- langchain

- openai

- streamlit

- python-dotenv

- gptcache

- tiktoken

- rich

- torch

- typer

## Contributing 🤝

We welcome contributions to this project! Please feel free to submit issues or pull requests.

## License ⚖️

This project is licensed under the MIT License.