{"id":13588461,"url":"https://github.com/shreyaskarnik/DistiLlama","last_synced_at":"2025-04-08T06:32:04.234Z","repository":{"id":203870641,"uuid":"710127591","full_name":"shreyaskarnik/DistiLlama","owner":"shreyaskarnik","description":"Chrome Extension to Summarize or Chat with Web Pages/Local Documents Using locally running LLMs. Keep all of your data and conversations private. 🔐","archived":false,"fork":false,"pushed_at":"2024-09-02T23:50:12.000Z","size":10151,"stargazers_count":290,"open_issues_count":9,"forks_count":33,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-06T11:11:10.474Z","etag":null,"topics":["chrome-extension","langchain","llama2","llms","local-llm","mistral-7b","ollama","privacy-preserving-computing","private","readability","retrieval-augmented-generation","summarization","zephyr"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shreyaskarnik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-26T04:34:12.000Z","updated_at":"2025-04-04T23:11:44.000Z","dependencies_parsed_at":"2023-11-12T23:23:24.786Z","dependency_job_id":"af1d9563-838e-418f-9e3d-32e7fc31a3c9","html_url":"https://github.com/shreyaskarnik/DistiLlama","commit_stats":{"total_commits":195,"total_committers":3,"mean_commits":65.0,"dds":0.2102564102564103,"last_synced_commit":"d0bff7ae6a310d910488a87016e0f14be286c31d"},"previous_names":["shreyaskarnik/distillama"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shreyaskarnik%2FDistiLlama","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shreyaskarnik%2FDistiLlama/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shreyaskarnik%2FDistiLlama/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shreyaskarnik%2FDistiLlama/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shreyaskarnik","download_url":"https://codeload.github.com/shreyaskarnik/DistiLlama/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247792077,"owners_count":20996876,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chrome-extension","langchain","llama2","llms","local-llm","mistral-7b","ollama","privacy-preserving-computing","private","readability","retrieval-augmented-generation","summarization","zephyr"],"created_at":"2024-08-01T15:06:43.789Z","updated_at":"2025-04-08T06:32:04.169Z","avatar_url":"https://github.com/shreyaskarnik.png","language":"TypeScript","funding_links":[],"categories":["TypeScript","chrome-extension"],"sub_categories":[],"readme":"# DistiLlama\n\n![image](public/icon-128.png)\n\n## What is DistiLlama?\n\nDistiLlama is a Chrome extension that leverages locally running LLM perform following  tasks.\n\n![Overview](./Overview.png)\n\nOne of the things that I was experimenting with is how to use a locally running LLM instance for various tasks and summarization (tl;dr) was on the top of my list. It was key to have all calls to LLM be local and all the data to stay private.\n\nThis project utilizes [Ollama](https://ollama.ai/) as the locally running LLM instance. Ollama is a great project that is easy to setup and use. I highly recommend checking it out.\n\nTo generate the summary I am using the following approach:\n\n- Grab the current active tab id\n- Use [Readability](https://github.com/mozilla/readability) to extract the text content from the page. In my experiments it was clear that the quality of the summary was much better when using Readability as it removed a lot of un-necessary content from the page.\n- Use [LangChain (LangChain.js)](https://js.langchain.com/docs/get_started/introduction/) to summarize the text content.\n- Display the summary in a popup window.\n\n## How to use DistiLlama?\n\n- Prerequisites:\n  - Install [Ollama](https://ollama.ai/download) you can also choose to run Ollama in a [Docker container](https://ollama.ai/blog/ollama-is-now-available-as-an-official-docker-image).\n  - Start Ollama using the following command: `OLLAMA_ORIGINS=* OLLAMA_HOST=127.0.0.1:11435 ollama serve`\n  - In another terminal you can run `ollama pull llama2:latest` or `ollama pull mistral:latest`\n  - Choice of model depends on your use case. Here are the models supported by Ollama \u003chttps://ollama.ai/library\u003e\n  - Make sure you set OLLAMA_ORIGINS=* for the Ollama environment by following instructions [here](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-configure-ollama-server)\n\n- Clone this repo\n  - Install pnpm `npm install -g pnpm`\n  - run `pnpm install`\n  - run `pnpm dev`\n  - Open Chrome and navigate to `chrome://extensions/`\n    - Enable developer mode (if not already enabled)\n    - Click on `Load unpacked` and select the `dist` folder from the base of the cloned project.\n    - You should see the DistiLlama added to your Chrome extensions.\n    - You may want to pin the extension to your Chrome toolbar for easy access.\n\n## Demo\n\n### Chat with LLM\n\n![Chat](./Chat.gif)\n\n### Chat with Documents (PDF)\n\n![ChatWithDocs](./ChatWithDocs.gif)\n\n### Chat with Web Page\n\n![ChatWithPage](./ChatWithPage.gif)\n\n### Summarization\n\n![Summary](./Summary.gif)\n\n## TODOS\n\n- [ ] Make the summarization chain configurable\n- [x] Make LLM model configurable\n- [ ] Save summary in local storage\n- [ ] Improve the UI (not an expert in this area but will try to learn)\n- [ ] Add TTS support\n- [ ] Check out performance with different tuned prompts\n- [x] Extend to chat with the page (use embeddings and LLMs for RAG)\n- [x] Use [transformers.js](https://github.com/xenova/transformers.js) for local in browser embeddings and [Voy](https://github.com/tantaraio/voy) for the storage similar to this [Building LLM-Powered Web Apps with Client-Side Technology](https://ollama.ai/blog/building-llm-powered-web-apps)\n- [ ] Focus on improving the quality of the summarization and chat\n- [ ] Multimodal support\n\n## References and Inspiration\n\n- [LangChain](https://github.com/langchain-ai/langchainjs)\n- [Ollama](https://ollama.ai/)\n- [Building LLM-Powered Web Apps with Client-Side Technology](https://ollama.ai/blog/building-llm-powered-web-apps)\n- [Chrome Extension Template](https://github.com/Jonghakseo/chrome-extension-boilerplate-react-vite)\n- Artwork generated using [DALL·E 3](https://openai.com/dall-e-3)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshreyaskarnik%2FDistiLlama","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshreyaskarnik%2FDistiLlama","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshreyaskarnik%2FDistiLlama/lists"}