{"id":19517304,"url":"https://github.com/armaggheddon/paperllama","last_synced_at":"2026-02-03T11:05:15.017Z","repository":{"id":259966214,"uuid":"879879765","full_name":"Armaggheddon/PaperLlama","owner":"Armaggheddon","description":"📄🦙 PaperLlama transforms your PDFs into a powerful, fully local Q\u0026A assistant—just upload, ask, and get instant answers!","archived":false,"fork":false,"pushed_at":"2024-11-21T18:49:33.000Z","size":1296,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-15T21:33:14.666Z","etag":null,"topics":["docker-compose","faiss","fastapi","ollama","python3","rag","sqlite3","streamlit"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Armaggheddon.png","metadata":{"files":{"readme":".github/README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-10-28T17:53:37.000Z","updated_at":"2024-11-21T18:49:37.000Z","dependencies_parsed_at":"2025-04-21T04:33:55.572Z","dependency_job_id":null,"html_url":"https://github.com/Armaggheddon/PaperLlama","commit_stats":null,"previous_names":["armaggheddon/paperllama"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Armaggheddon/PaperLlama","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Armaggheddon%2FPaperLlama","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Armaggheddon%2FPaperLlama/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Armaggheddon%2FPaperLlama/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Armaggheddon%2FPaperLlama/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Armaggheddon","download_url":"https://codeload.github.com/Armaggheddon/PaperLlama/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Armaggheddon%2FPaperLlama/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29043789,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-03T10:09:22.136Z","status":"ssl_error","status_checked_at":"2026-02-03T10:09:16.814Z","response_time":96,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker-compose","faiss","fastapi","ollama","python3","rag","sqlite3","streamlit"],"created_at":"2024-11-11T00:01:34.897Z","updated_at":"2026-02-03T11:05:14.983Z","avatar_url":"https://github.com/Armaggheddon.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv id=\"top\"\u003e\u003c/div\u003e\n\u003cbr/\u003e\n\u003cbr/\u003e\n\u003cbr/\u003e\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/paper_llama_nobg_border.png\" width=\"180\"\u003e\n\u003c/p\u003e\n\u003ch1 align=\"center\"\u003e\n    \u003ca href=\"https://github.com/Armaggheddon/PaperLlama\"\u003ePaperLlama\u003c/a\u003e\n\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/Armaggheddon/PaperLlama/commits/master\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/last-commit/Armaggheddon/PaperLlama\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/Armaggheddon/PaperLlama\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Maintained-yes-green.svg\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/Armaggheddon/PaperLlama/issues\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/issues/Armaggheddon/PaperLlama\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/Armaggheddon/PaperLlama/blob/master/LICENSE\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/license/Armaggheddon/PaperLlama\"\u003e\n    \u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n    Your fully local, AI-powered Q\u0026A assistant—meet PaperLlama!\n    \u003cbr/\u003e\n    \u003cbr/\u003e\n    \u003ca href=\"https://github.com/Armaggheddon/PaperLlama/issues\"\u003eReport Bug\u003c/a\u003e\n    •\n    \u003ca href=\"https://github.com/Armaggheddon/PaperLlama/issues\"\u003eRequest Feature\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\nWelcome to **PaperLlama**! 🦙🎓 Your personal academic assistant that’s powered up and ready to help you sift through stacks of papers, PDFs, and academic docs for those *must-have* insights! Using a combo of state-of-the-art AI magic (thanks to Ollama's wide model support) and custom-built tech, PaperLlama makes document-based Q\u0026A a breeze.\n\n\n## 🚀 Getting Started\nWith **Docker Compose**, you can spin up the whole PaperLlama suite in one go! Here's how:\n1. Clone the repository:\n    ```bash\n    git clone https://github.com/Armaggheddon/PaperLlama.git\n    cd PaperLlama\n    ```\n\n1. Chose either the `docker-compose.yml` for cpu-only setups or `gpu-docker-compose.yml` if you have an Nvidia GPU. First build and then launch the containers with\n    ```bash\n    docker compose -f \u003ccompose_file\u003e build\n    docker compose -f \u003ccompose_file\u003e up -d\n    ```\n\n1. Once everything's up, navigate to `http://localhost:80` to start using PaperLlama through the web ui. \n\n1. Start Exploring! Upload a PDF, ask a question, and let PaperLlama pull the info you need in seconds!\n\n\u003e [!NOTE]\n\u003e GPU support requires the NVIDIA Container Toolkit. Look [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for the installation guide.\n\n\u003e [!IMPORTANT]\n\u003e The first startup, as well as the first file upload, might take a while since the required models are downloaded. This is a one time operation. \n\n\n## 🖼️ User Interface Overview\nHere's a quick tour of what's on the PaperLlama dashboard!\n\n### 🤖 Chat\nThe heart of PaperLlama! Ask questions directly here, and get answers powered by your uploaded documents. Perfect for digging into those long reports and finding what you need fast. This page pulls data from all the uploaded files. \n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/global_chat_ui.png\" width=\"650\"\u003e\n\u003c/p\u003e\n\n### 📃 Document chat\nSelect a document and chat exclusively with its contents. This mode only queries data from the selected document uuid (which can be found in the knowledge manager)\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/document_chat_ui.png\" width=\"650\"\u003e\n\u003c/p\u003e\n\n### 🧠 Knowledge Manager\nThis page is your overview of everything PaperLlama has indexed. You’ll see all the uploaded documents, metadata, and the data that can be used for generating answers.\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/knowledge_manager_ui.png\" width=\"650\"\u003e\n\u003c/p\u003e\n\n### ⚡ Upload new\nGot more PDFs? Head here to add them to PaperLlama. Each upload is automatically embedded and indexed so it’s ready for action. If is the first time uploading a file, it might take a while due to the models being downloaded. Go take a coffee or think about that bug you introduced last friday 🤔.\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/upload_new_ui.png\" width=\"650\"\u003e\n\u003c/p\u003e\n\n\u003e [!NOTE]\n\u003e For now, we only support PDFs. But we’re always working on expanding to more document types! 📄✨\n\n### 🛠️ Status\nSee the current status of the services that allow PaperLlama to work. Hopefully everything is 🟢. 😁\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/service_status_ui.png\" width=\"650\"\u003e\n\u003c/p\u003e\n\n\n## 🎨 Customization\nChose which model to use for the document summarization, chat and embedding. By default the following models are used. However feel free to try different ones by setting the following environment variables in the docker compose file:\n```dockerfile\nenvironment:\n    - EMBEDDING_MODEL_NAME=nomic-embed-text\n    - EMBEDDING_MODEL_OUTPUT_SIZE=768\n    - CHAT_MODEL_NAME=llama3.2:1b\n    - INSTRUCT_MODEL_NAME=llama3.2:1b-instruct-q4_0\n```\n\nYou can use any model that is supported by Ollama, assuming it can run on your hardware 🫣. Look [**here**](https://ollama.com/search) for a complete list of supported models.\n\nThe prompts used for the different tasks are not the greatest, I will give that. So feel free to change the prompts used in [`prompts.py`](../backend/src/ollama_proxy/prompts.py) to better match the model you choose to use. Remember to rerun [`Step 2 of Getting started`](#-getting-started) if the prompts are changed while the services are running.\n\n\u003e [!NOTE]\n\u003e Using Llama 3.2 1B, while being lightweight to run, will not yield the best results. Try with a larger model since it generally has better understanding capabilities and adherence to the prompts.\n\n\u003e [!WARNING]\n\u003e When you change the text embedding model and there are already embedded documents, if the embedding size is different, querying or adding new documents will crash the application. Delete the currently stored documents and restart.\n\n\n## 🧐 How it works?\nPaperLlama is a project born from a desire to experiment with LLMs and RAG systems. The goal was to create a straightforward implementation that balances simplicity with flexibility, allowing users to easily understand the system while also enabling them to swap components of customize the pipeline to fit their needs.\n\nAt its core, PaperLlama provides a seamless experience for managing and querying documents by organizing its functionality into modular services. Each service exposes its functionality through a FastAPI web server, making it easy to replace or extend components as long as they adhere to the same API interface.\n\nThe **datastore** is built using Faiss and SQLite, and I considered it as a personal excercise. If you prefer other vector indexes, which might also provide more functionalities you can simply replace it! There are many open source alternatives, for example you prefer ChromaDB? No problem! Replace the **datastore** service with one that implements the same API interface, and you're good to go.\n\nHere’s a brief overview of how the project works and the main services involved:\n\n- **web_ui**: handles the user-facing web interface, providing a way to interact with the system.\n\n- **backend**: the main service that orchestrates the interaction between different services, effectively acting as a proxy for **datastore** and **document_converter**.\n\n- **datastore**: manages data storage and retrieval, including document metadata and vector embeddings.\n\n- **document_converter**: converts PDF documents into a markdown format that can be easily processed by the AI model.\n\nThis modular approach ensures flexibility, enabling experimentation with different LLMs, storage systems, or workflows. By keeping the design minimal yet extensible, PaperLlama is an ideal playground for anyone curious about building RAG pipelines or exploring document-based AI applications.\n\nFor details on API specifications, refer to:\n- [**backend**](API_BACKEND.md)\n- [**datastore**](API_DATASTORE.md)\n- [**document_converter**](API_DOCUMENT_CONVERTER.md)\n\n\n## 🛠️ Development\nIf you’re working on PaperLlama and want to focus on developing or debugging a specific service, you can enable individual services by modifying the provided [`dev-docker-compose.yml`](../dev-docker-compose.yml) file. Here’s how you can adjust the Compose file for development purposes:\n\n1. For each service you want to to work with uncomment the following lines \n    ```yaml\n      stdin_open: true\n      tty: true\n      command:\n        - /bin/bash\n    ```\n\n1. Launch the containers:\n    ```bash\n    docker compose -f dev-docker-compose.yml up\n    ```\n\n1. Connect to the container/s that you want to develop with. A development folder will be mounted at `/dev_\u003cservice_name\u003e` where `\u003cservice_name\u003e` is the name of the service you are currently connected to; for example for the datastore will be `/dev_datastore`. All the changes made to the contents of this folder will be reflected on the host `/\u003cservice_name\u003e` folder.\n\n1. Once you finished simply terminate the containers with either `ctrl+c` or:\n    ```bash\n    docker compose -f dev-docker-compose down\n    ```\n    add option `-v` to also clear the volumes. This will also cause the services to redownload all the models on the next startup.\n\n1. Chose between `gpu-docker-compose.yml` or `docker-compose.yml` and run:\n    ```bash\n    docker compose -f \u003ccompose_file\u003e build\n    docker compose -f \u003ccompose_file\u003e up -d\n    ```\n\n\u003e [!WARNING]\n\u003e The `dev-docker-compose.yml` assumes that the host has an Nvidia GPU. If this is not the case, simply remove the following lines from the services `ollama` and `document_converter`:\n\u003e```yaml\n\u003e    deploy:\n\u003e    resources:\n\u003e        reservations:\n\u003e        devices:\n\u003e            -driver: nvidia\n\u003e            count: 1\n\u003e            capabilities: [gpu]\n\u003e```\n\n\u003e [!NOTE]\n\u003e When using `dev-docker-compose.yml` all the services will expose the OpenAPI documentation on the following ports:\n\u003e- backend: `http://localhost:8100/docs`\n\u003e- datastore: `http://localhost:8102/docs`\n\u003e- document_converter: `http://localhost:8101/docs`\n\n## 📮 Responsible Disclosure\n We assume no responsibility for an improper use of this code and everything related to it. We do not assume any responsibility for damage caused to people and / or objects in the use of the code.\n\n**By using this code even in a small part, the developers are declined from any responsibility.**\n\n More informations is available at the following link: [**License**](../LICENSE)\n\n\n## 🐛 Bug and Feature \nTo **report a bug** or to request the implementation of **new features**, it is strongly recommended to use the [**ISSUES tool from Github »**](https://github.com/Armaggheddon/PaperLlama/issues).\n\nHere you may already find the answer to the problem you have encountered, in case it has already happened to other people. Otherwise you can report the bugs found.\n\n**ATTENTION: To speed up the resolution of problems, it is recommended to answer all the questions present in the request phase in an exhaustive manner.**\n\n(Even in the phase of requests for the implementation of new functions, we ask you to better specify the reasons for the request and what final result you want to obtain).\n\n\n## 🔍 License\n**MIT LICENSE**\n\n*Permission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including...*\n\n\n[**License Documentation »**](../LICENSE)\n\n\n## 📚 Libraries used\nThis project makes use of the following third party libraries:\n- [**Ollama**](https://ollama.com/): for the LLM inference.\n- [**FastAPI**](https://fastapi.tiangolo.com/): for the webserver of each service.\n- [**Faiss**](https://github.com/facebookresearch/faiss): for the vector database.\n- [**Docling**](https://github.com/DS4SD/docling): for processing the PDF files in such a way that is easily understandable by AI models.\n- [**Streamlit**](https://streamlit.io/): for the web ui.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farmaggheddon%2Fpaperllama","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farmaggheddon%2Fpaperllama","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farmaggheddon%2Fpaperllama/lists"}