{"id":24560825,"url":"https://github.com/nathanielknight/llm-questioncache","last_synced_at":"2025-04-19T14:44:17.041Z","repository":{"id":272996209,"uuid":"918416127","full_name":"nathanielknight/llm-questioncache","owner":"nathanielknight","description":"An LLM plugin to efficiently pose questions to LLMs, cache the answers, and quickly retrieve answers to questions that you've already posed.","archived":false,"fork":false,"pushed_at":"2025-02-09T05:50:22.000Z","size":81,"stargazers_count":9,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-11T17:10:38.664Z","etag":null,"topics":["cli","embeddings","llm","sqlite"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nathanielknight.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-17T22:15:43.000Z","updated_at":"2025-02-21T02:46:30.000Z","dependencies_parsed_at":"2025-01-17T23:36:44.846Z","dependency_job_id":null,"html_url":"https://github.com/nathanielknight/llm-questioncache","commit_stats":null,"previous_names":["nathanielknight/llm-questioncache"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nathanielknight%2Fllm-questioncache","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nathanielknight%2Fllm-questioncache/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nathanielknight%2Fllm-questioncache/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nathanielknight%2Fllm-questioncache/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nathanielknight","download_url":"https://codeload.github.com/nathanielknight/llm-questioncache/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249716842,"owners_count":21315068,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","embeddings","llm","sqlite"],"created_at":"2025-01-23T07:18:55.879Z","updated_at":"2025-04-19T14:44:17.029Z","avatar_url":"https://github.com/nathanielknight.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# llm-questioncache\n\nA plugin for [llm](https://llm.datasette.io/) for sending questions\nto LLMs and getting succinct answers. It also saves answers in a\nSQLite database along with embeddings of the corresponding questions\nand will answer future, similar questions from the cache rather than\nthe LLM.\n\n## Installation\n\n```bash\nllm install llm-questioncache\n```\n\n## Usage\n\nThe plugin adds a new `questioncache` command group to `llm`. See `llm questioncache --help` for the full list of subcommands.\n\n### Ask a Question\n\n```bash\nllm questioncache ask \"What is the capital of France?\"\n```\n\nThis will:\n1. Check if similar questions exist in the cache\n2. If found, show the cached answers\n3. If not found, ask the LLM and cache the response\n\nYou can also pipe questions through stdin:\n```bash\necho \"What is the capital of France?\" | llm questioncache ask -\n```\n\n### Send Last Question Directly to LLM\n\nTo bypass the cache and send the last asked question directly to the LLM:\n\n```bash\nllm questioncache send\n```\n\nYou might have to do this if you've previously asked a similar-but-distinct\nquestion\n\n### Import Previous Answers\n\nYou can import a collection of previous questions and answers from a JSON file:\n\n```bash\nllm questioncache importanswers answers.json\n```\n\nThe JSON file should contain an array of objects with `question` and `answer` fields.\n\nIf you've been using LLM in this way already you might have some useful answers already.\nTo retrieve and format all the LLM responses with a particular system prompt, use `sqlite-utils`:\n\n```sh\nuvx sqlite-utils \"$(llm logs path)\" \"select prompt as question, response as answer from responses where system = 'Answer in as few words as possible. Use a brief style with short replies.'\"\n```\n\n### Clear the Cache\n\nTo delete all cached questions and answers:\n\n```bash\nllm questioncache clearcache\n```\n\n## Configuration\n\nThe plugin uses your default LLM and embedding models as configured in `llm`. No additional configuration is required.\n\nKey parameters (configured in the code):\n- Relevance cutoff for similar questions: 0.8\n- Number of similar answers to show: 3\n- System prompt for brief answers: \"Answer in as few words as possible. Use a brief style with short replies.\"\n\n\n## Shell integration\n\nYou might find it useful to create a shell script to succinctly invoke `llm questioncache`:\n\nFor example, save this as `~/.local/bin/q`:\n\n```\n#!/usr/bin/env sh\nllm questioncache $*\n```\n\nYou can now pose questions with:\n\n```sh\nq how do you exit vim\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnathanielknight%2Fllm-questioncache","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnathanielknight%2Fllm-questioncache","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnathanielknight%2Fllm-questioncache/lists"}