Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/romiconez/konspecto-llm
LLM agent that provides tools for convenient work with personal documents using voice or text.
https://github.com/romiconez/konspecto-llm
agent backend docker docx frontend google langchain llamaindex llm managment nlp rag whisper
Last synced: 3 days ago
JSON representation
LLM agent that provides tools for convenient work with personal documents using voice or text.
- Host: GitHub
- URL: https://github.com/romiconez/konspecto-llm
- Owner: RomiconEZ
- License: apache-2.0
- Created: 2024-12-05T13:43:00.000Z (about 1 month ago)
- Default Branch: develop
- Last Pushed: 2025-01-01T13:56:58.000Z (6 days ago)
- Last Synced: 2025-01-01T14:29:29.411Z (6 days ago)
- Topics: agent, backend, docker, docx, frontend, google, langchain, llamaindex, llm, managment, nlp, rag, whisper
- Language: Python
- Homepage:
- Size: 5.3 MB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# 🎓 KONSPECTO - LLM Agent for Note Management
## 👥 Authors
- Neronov Roman
- Fazlyev Albert
## 📋 Project Description
KONSPECTO is an intelligent agent based on a local LLM model, offering the following capabilities:
🔍 **Search Through Notes**
- Semantic search across the notes database
- Generation of structured responses based on the retrieved information
- Ability to view original documents
🎥 **Video Processing**
- Extraction of keyframes from YouTube videos
- Creation of DOCX documents with images
- Filtering of similar frames
🎤 **Voice Input**
- Transcription of voice messages using Whisper
- Support for the Russian language
- Ability to combine voice and text input
## 📽️ Presentation
[Presentation KONSPECTO](https://github.com/RomiconEZ/KONSPECTO-LLM/blob/develop/presentation/KONSPECTO_LLM_base.pdf)
## 🛠 Tech Stack
### Frontend
- ⚛️ React + Vite
- 🎨 TailwindCSS
- 🔄 React Router
- ✨ React Icons
### Backend
- 🚀 FastAPI
- 🤖 LangChain
- 🔍 LlamaIndex
- 📝 Whisper
- 🎥 OpenCV
- 🗄️ Redis Stack
## 📦 Installation
### Prerequisites
- Docker and Docker Compose
- Node.js 18+
- Python 3.11+
- Poetry
- pre-commit
- LM Studio - Download from [https://lmstudio.ai](https://lmstudio.ai/)
### LM Studio Setup
1. Download and install LM Studio from the official website
2. In LM Studio:
- Go to "Search" tab
- Find and download `IlyaGusev/saiga_nemo_12b_gguf/saiga_nemo_12b.Q8_0.gguf` model
- Go to "Local Server" tab
- Select the downloaded model from the dropdown menu
- Start the server (it will run on http://localhost:1234/v1)
- Keep the server running while using KONSPECTO
> ⚠️ Note: Make sure the LM Studio server is running before starting the application, as KONSPECTO relies on it for text generation.
### 1️⃣ Clone the Repository
```bash
git clone https://github.com/RomiconEZ/KONSPECTO
cd KONSPECTO
```
### 2️⃣ Configure Settings
Create configuration files in the `backend/app/config/` directory:
**.env**
```env
FOLDER_ID=your_google_drive_folder_id
GOOGLE_SERVICE_ACCOUNT_KEY_PATH=config/service_account_key.json
TRANSCRIPTION_MODEL=whisper
WHISPER_MODEL_SIZE=large-v3
LLM_STUDIO_BASE_URL=http://localhost:1234/v1
EMBEDDING_MODEL_NAME="intfloat/multilingual-e5-large"
EMBEDDING_BATCH_SIZE=16
EMBEDDING_DIMENSION=1024
```
**service_account_key.json**
```json
{
// Your Google service account credentials
// Obtain them from the Google Cloud Console
}
```
### 3️⃣ Install Dependencies
Frontend:
```bash
cd frontend
npm install
```
Backend:
```bash
cd backend
poetry install
```
### 4️⃣ Set Up pre-commit Hooks
```bash
pre-commit install --install-hooks
pre-commit run --all-files
```
### 5️⃣ Run Tests
Frontend tests:
```bash
cd frontend
npm run test
```
Backend tests:
```bash
cd backend
bash tests/run_tests.sh
```
### 6️⃣ Launch the Application
```bash
docker compose up --build
```
The application will be available at the following addresses:
- Frontend: [http://localhost:80](http://localhost:80)
- Backend API: [http://localhost:8000](http://localhost:8000)
- Redis Stack: [http://localhost:8001](http://localhost:8001)
## 🔄 Workflow
1. **Information Search**
- The user sends a request through the UI
- The agent analyzes the request and determines the necessary tools
- A search is performed across the knowledge base and a response is generated
2. **Video Processing**
- Uploading a YouTube video
- Extracting frames every 5 seconds
- Filtering similar images
- Creating a DOCX document
3. **Voice Input**
- Recording audio via the browser
- Transcription using Whisper
- Adding the text to the current query
## ✅ Validation
It is not possible to produce a deterministic assessment of the agent’s performance because its effectiveness depends on the unique data serving as its knowledge base. In our case, this knowledge base consists of user-generated notes, which are different for every individual. Consequently, any quality measurement will vary significantly from one user’s environment to another.
In this project, we tested the agent on two specific documents: one explaining gradient descent and another explaining stochastic gradient descent. The system demonstrated consistent accuracy in retrieving relevant information from these documents during the queries shown in the demo video. However, because user notes can differ in style, depth, and content, the same agent might show varied results when applied to an entirely different set of documents.
This inherent reliance on specialized, user-specific data makes it impossible to generalize the agent’s quality or establish a uniform benchmark. The system’s performance is inseparable from the nuances of the data it is provided with, preventing any deterministic evaluation of its capabilities.
## 📜 License
Apache License
## ⭐️ Support the Project
If you like the project, give it a star on GitHub!