Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/brianlesko/embedding-ai
This repo implements text embedding that allows similarity search, clustering, and AI models to interpret embedded content
https://github.com/brianlesko/embedding-ai
Last synced: about 20 hours ago
JSON representation
This repo implements text embedding that allows similarity search, clustering, and AI models to interpret embedded content
- Host: GitHub
- URL: https://github.com/brianlesko/embedding-ai
- Owner: BrianLesko
- License: mit
- Created: 2023-11-21T00:19:49.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2024-01-17T23:15:53.000Z (10 months ago)
- Last Synced: 2024-01-18T06:06:46.094Z (10 months ago)
- Language: Python
- Size: 7.51 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
- License: LICENSE.md
Awesome Lists containing this project
README
# Embedding AI
This code implements a text embedding algorithm. Embedding is a method of turning data into a standard format that can be used for search, clustering, and training nueral networks. This implementation is written in [Pure Python](https://github.com/BrianLesko/embedding-ai/blob/main/app.py) - created for Learning Purposes.
## Dependencies
This code uses the following libraries:
- `streamlit`: for building the user interface.
- `numpy`: for creating arrays.
- `pandas`: for creating dataframes.
- `scikit-learn`: for comparing the similarity of two embeddings
- `openai`: for creating the text embeddings
- `plotly`: for plotting
- `tiktoken`: for tokenizing text, needed for OpenAI embeddings
## Usage
Run the following commands:
```
pip install --upgrade streamlit numpy pandas scikit-learn openai plotly
streamlit run https://github.com/BrianLesko/embedding-ai/blob/main/app.py
```
This will start the local Streamlit server, and you can access the chatbot by opening a web browser and navigating to `http://localhost:8501`.
## How it Works
The app as follows:
1. The user enters some text in the input field.
2. OpenAI is used to embed the text into a vector of numbers.
3. Scikit-learn is used to compare the embedding to an embedding created from a description of soccer and a description of math.
4. The app displays the similarity on a Plotly chart.
5. The user can enter more text to see more similarities.
## Repository Structure
```
repository/
├── app.py # the code and UI integrated together live here
├── customize_gui # class for adding gui elements
├── requirements.txt # the python packages needed to run locally
├── .gitignore # includes the local virtual environment named my_env
├── .streamlit/
│ └── config.toml # theme info for the UI
└── docs/
└── preview.png # preview photo for Github
```
## Topics
```
Python | Streamlit | Git | Low Code UI
Chat interface | text embedding | OpenAI | textual data | similarity search
Self taught coding | Mechanical engineer | Robotics engineer
```
╭━━╮╭━━━┳━━┳━━━┳━╮╱╭╮ ╭╮╱╱╭━━━┳━━━┳╮╭━┳━━━╮
┃╭╮┃┃╭━╮┣┫┣┫╭━╮┃┃╰╮┃┃ ┃┃╱╱┃╭━━┫╭━╮┃┃┃╭┫╭━╮┃
┃╰╯╰┫╰━╯┃┃┃┃┃╱┃┃╭╮╰╯┃ ┃┃╱╱┃╰━━┫╰━━┫╰╯╯┃┃╱┃┃
┃╭━╮┃╭╮╭╯┃┃┃╰━╯┃┃╰╮┃┃ ┃┃╱╭┫╭━━┻━━╮┃╭╮┃┃┃╱┃┃
┃╰━╯┃┃┃╰┳┫┣┫╭━╮┃┃╱┃┃┃ ┃╰━╯┃╰━━┫╰━╯┃┃┃╰┫╰━╯┃
╰━━━┻╯╰━┻━━┻╯╱╰┻╯╱╰━╯ ╰━━━┻━━━┻━━━┻╯╰━┻━━━╯
follow all of these or i will kick you