{"id":28858013,"url":"https://github.com/parthapray/readability_ollama_llm","last_synced_at":"2026-05-06T06:37:49.401Z","repository":{"id":299983277,"uuid":"1004741463","full_name":"ParthaPRay/Readability_Ollama_LLM","owner":"ParthaPRay","description":"This repo shows the coding of readability analysis of response from lcoalized small LLMs from Raspberry Pi using Ollama","archived":false,"fork":false,"pushed_at":"2025-06-19T08:27:26.000Z","size":36,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-19T09:31:30.747Z","etag":null,"topics":["edge","gradio","iot","large-language-models","localization","ollama","raspberry-pi","readability-metrics","spacy","textdescriptives","textstat"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ParthaPRay.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-19T05:38:56.000Z","updated_at":"2025-06-19T08:28:44.000Z","dependencies_parsed_at":"2025-06-19T09:44:32.742Z","dependency_job_id":null,"html_url":"https://github.com/ParthaPRay/Readability_Ollama_LLM","commit_stats":null,"previous_names":["parthapray/readability_ollama_llm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ParthaPRay/Readability_Ollama_LLM","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthaPRay%2FReadability_Ollama_LLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthaPRay%2FReadability_Ollama_LLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthaPRay%2FReadability_Ollama_LLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthaPRay%2FReadability_Ollama_LLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ParthaPRay","download_url":"https://codeload.github.com/ParthaPRay/Readability_Ollama_LLM/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ParthaPRay%2FReadability_Ollama_LLM/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260862918,"owners_count":23074182,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["edge","gradio","iot","large-language-models","localization","ollama","raspberry-pi","readability-metrics","spacy","textdescriptives","textstat"],"created_at":"2025-06-20T02:04:25.635Z","updated_at":"2026-05-06T06:37:49.396Z","avatar_url":"https://github.com/ParthaPRay.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ReadLLMAnOLLmAPI\n\nReadability LLM Analyzer for Ollama API on Localized Resource Constained Edge (Raspberry Pi)\n\n\u003e **Analyze the readability and structural metrics of responses generated by small, quantized LLMs running locally via [Ollama](https://github.com/ollama/ollama) and view runtime metrics interactively with Gradio.**\n\n---\n\n## ✨ Project Overview\n\nThis tool enables researchers and practitioners to **evaluate the readability, complexity, and linguistic structure** of outputs from localized small-scale LLMs (quantized for edge/low-resource devices) using the Ollama inference engine.  \nMetrics are calculated using [TextStat](https://github.com/textstat/textstat) and [TextDescriptives](https://hlasse.github.io/TextDescriptives/index.html) and are stored in a local SQLite database for further analysis.  \nA [Gradio](https://gradio.app/) web interface allows for interactive prompt-response analysis and metrics exploration.\n\n---\n\n## 🚀 Features\n\n- **Supports Small Quantized LLMs**: Works with any local model served by [Ollama](https://github.com/ollama/ollama) API.\n- **Comprehensive Metrics**: Extracts classic readability indices, spaCy linguistic features, TextDescriptives metrics (including POS proportions, coherence, information theory, and more).\n- **Interactive Gradio UI**: Type a prompt, select a model, and view LLM output plus all computed metrics in a clear dashboard.\n- **Metrics Logging**: Every inference is logged into a local SQLite database for reproducible, longitudinal, or comparative studies.\n- **Runtime Statistics**: Captures Ollama-specific runtime stats (token/s, duration, etc.) alongside text metrics.\n- **Designed for Edge**: Runs efficiently on a Raspberry Pi 4B or similar edge device, and any system supported by Ollama.\n\n---\n\n## 🛠️ Installation\n\n1. **Clone this repo**  \n    ```bash\n    git clone https://github.com/YOUR-USERNAME/llm-readability-analyzer.git\n    cd llm-readability-analyzer\n    ```\n\n2. **Install dependencies**  \n    We recommend using a virtual environment (e.g., `venv` or `conda`):\n\n    ```bash\n    pip install -r requirements.txt\n    ```\n\n    **`requirements.txt` contents:**\n    ```\n    gradio\n    requests\n    textstat==0.7.7\n    spacy\n    textdescriptives==2.8.4\n    ```\n\n3. **Download the spaCy language model** (run once):\n    ```bash\n    python -m spacy download en_core_web_sm\n    ```\n\n4. **Start the Ollama server and pull any quantized model you want to use**  \n   (see [Ollama documentation](https://ollama.com/library)).\n\n---\n\n## ⚡ Usage\n\n1. **Launch the app:**\n    ```bash\n    python ReadLLMAnOLLmAPI.py\n    ```\n\n2. **Visit the Gradio UI:**  \n   Open [http://localhost:7860](http://localhost:7860) in your browser.\n\n3. **Workflow:**\n   - Select an available LLM model (auto-discovered from Ollama).\n   - Enter a prompt.\n   - Click \"Analyze\".\n   - View the model's response, readability indices, TextDescriptives metrics, and Ollama runtime statistics.\n   - All results are also logged in `llm_readability.db` (SQLite).\n\n---\n\n## 📊 What Metrics Are Measured?\n\n- **Readability:** Flesch Reading Ease, SMOG, Coleman-Liau, Dale-Chall, etc.\n- **Complexity \u0026 Structure:** Syllable/word/sentence counts, polysyllabic/monosyllabic word ratios, word length stats.\n- **POS Proportions:** Fraction of nouns, verbs, adjectives, etc.\n- **TextDescriptives:** Coherence, information theory (entropy, perplexity), quality metrics (repetition, bullets, ellipsis, OOV ratio), dependency distance, and more.\n- **Ollama Runtime:** Model loading time, prompt evaluation stats, tokens per second.\n\n---\n\n## 📝 Example Output\n\n- The UI displays all metrics in a tabular format, with headings.\n- The SQLite DB (`llm_readability.db`) can be analyzed further with any SQL tool or exported as CSV for research.\n\n---\n\n## 🧑‍💻 Code Organization\n\n- **`ReadLLMAnOLLmAPI.py`** — Main application logic (Gradio UI, metrics computation, DB logging)\n- **`requirements.txt`** — List of dependencies\n\n---\n\n## 🔗 References\n\n- [Ollama Documentation](https://ollama.com/)\n- [TextStat](https://github.com/textstat/textstat)\n- [spaCy](https://spacy.io/)\n- [TextDescriptives](https://hlasse.github.io/TextDescriptives/index.html)\n- [Gradio](https://gradio.app/)\n\n---\n\n## 📝 Notes\n\n- This project is optimized for **localized, resource-constrained environments** (Raspberry Pi, Jetson, low-end cloud, etc.), but will run on any Python 3.8+ system.\n- Make sure the Ollama server is running before starting the app.\n- All metrics are calculated **locally** for data privacy and research reproducibility.\n\n---\n\n## 👤 Author\n\nMaintained by [Partha Pratim Ray].  \nPull requests and issues welcome!\n\n---\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparthapray%2Freadability_ollama_llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparthapray%2Freadability_ollama_llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparthapray%2Freadability_ollama_llm/lists"}