https://github.com/dhammon/security-assistant
The Security Assistant is an Ollama based large language model (LLM) retrieval-augmented generation (RAG) project that summarizes PDF and XLSX documents and responds to customer and auditor security questions. Everything runs locally and does not rely on third party/online models.
https://github.com/dhammon/security-assistant
ai llm ollama retrieval-a security-audit security-automation
Last synced: 5 months ago
JSON representation
The Security Assistant is an Ollama based large language model (LLM) retrieval-augmented generation (RAG) project that summarizes PDF and XLSX documents and responds to customer and auditor security questions. Everything runs locally and does not rely on third party/online models.
- Host: GitHub
- URL: https://github.com/dhammon/security-assistant
- Owner: dhammon
- License: gpl-2.0
- Created: 2024-11-24T17:51:34.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-11-24T20:36:02.000Z (over 1 year ago)
- Last Synced: 2025-04-06T11:12:13.223Z (about 1 year ago)
- Topics: ai, llm, ollama, retrieval-a, security-audit, security-automation
- Language: Python
- Homepage:
- Size: 3.43 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# About
The Security Assistant is an Ollama based large language model (LLM) retrieval-augmented generation (RAG) project that summarizes PDF and XLSX documents and responds to customer and auditor security questions. Everything runs locally and does not rely on third party/online models.
# Why
Many organizations are hesitant to provide internal documentation to web based LLMs, such as ChatGPT, because of legal and privacy risks. Therefore, a solution is needed to run a high quality on-premise LLM+RAG that does not rely on a third party to process data. I was inspired to create an LLM that could use existing support documentation to answer security questions customers and/or auditors might have. While the configured prompt is tailored to answering security questions, this project's prompts and files could be easily adjusted to be used for any LLM + RAG purpose.
# How it works
Langchain unstructured document parsers iterate through each document within the `files` folder, then chunks the content before inserting it into a local vector database using the `nomic-embed-text` model.
The agent runs two stages when asked a question using the Ollama3 model. During the first stage, using only the content from the vector database, the agent analyzes available material based on the question and returns a few responses. The second stage uses the output of the first stage to produce a final response.
# Requirements
- Tested on Debian and Ubuntu
- Storage: 25 GB
- CPU: 8 cores, w/o GPU will take ~5 min for LLM to respond
- GPU: optional, but 1000x faster
- Python3, pip, python3-venv (`apt install python3-venv`)
# Install
1. Setup python virtual env
```bash
python3 -m venv env
source env/bin/activate
```
2. Setup environment
```bash
chmod +x setup.sh
./setup.sh
```
# Setup
> Make sure you are in the Python Virtual Environment (`source env/bin/activate`)
1. Upload PDF, XLSX, CSV files to files folder
2. Create vector dababase
```bash
python3 vector.py
```
# Use
> Make sure you are in the Python Virtual Environment (`source env/bin/activate`)
1. Use the agent to ask questions
```bash
python3 agent.py
#ask your questions
#quit/exit to exit
```
# Notes
## Inspiration from:
1. https://www.youtube.com/watch?v=WfFpeBNfaeQ&list=PLB6yN0iA43p69t3613Q6i2pqq90Ps3ttb
2. https://github.com/tonykipkemboi/ollama_pdf_rag/tree/main
## References
1. https://github.com/hwchase17/chroma-langchain/blob/master/persistent-qa.ipynb
2. https://docs.aws.amazon.com/systems-manager/latest/userguide/walkthrough-cli.html