{"id":28138956,"url":"https://github.com/darshan-vijay/plant-doc","last_synced_at":"2026-05-02T02:34:41.229Z","repository":{"id":290611005,"uuid":"966393474","full_name":"darshan-vijay/Plant-Doc","owner":"darshan-vijay","description":"Detects plant diseases from leaf images using a fine-tuned ResNet model, then uses a domain-adapted Flan-T5 language model to answer user questions about the disease, including symptoms and treatments, based on curated knowledge.","archived":false,"fork":false,"pushed_at":"2025-04-29T17:20:09.000Z","size":98765,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-14T17:16:05.147Z","etag":null,"topics":["agriculture","computer-vision","flan-t5","image-classification","lora","plant-disease-detection","question-answering","resnet50","streamlit","transformers"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/darshan-vijay.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-14T21:17:48.000Z","updated_at":"2025-04-29T17:23:48.000Z","dependencies_parsed_at":"2025-04-29T18:41:37.437Z","dependency_job_id":null,"html_url":"https://github.com/darshan-vijay/Plant-Doc","commit_stats":null,"previous_names":["darshan-vijay/plant-doc"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/darshan-vijay/Plant-Doc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/darshan-vijay%2FPlant-Doc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/darshan-vijay%2FPlant-Doc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/darshan-vijay%2FPlant-Doc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/darshan-vijay%2FPlant-Doc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/darshan-vijay","download_url":"https://codeload.github.com/darshan-vijay/Plant-Doc/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/darshan-vijay%2FPlant-Doc/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32521108,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-02T01:12:54.858Z","status":"online","status_checked_at":"2026-05-02T02:00:05.923Z","response_time":132,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agriculture","computer-vision","flan-t5","image-classification","lora","plant-disease-detection","question-answering","resnet50","streamlit","transformers"],"created_at":"2025-05-14T17:15:40.926Z","updated_at":"2026-05-02T02:34:41.205Z","avatar_url":"https://github.com/darshan-vijay.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Plant Disease Identification and QA System\n\n## Motivation\n\nTimely identification and treatment of plant diseases is **critical to preventing crop loss** and improving agricultural productivity. Most existing systems stop at **disease classification** — leaving users without actionable insights.\n\nThis project aims to solve that gap by not only classifying plant diseases from images, but also generating **natural language answers** to user questions using a **domain-specific fine-tuned language model**.\n\n\u003e “Less Guessing... More Doing.\n\u003e Ask the Expert – because even plants need a little therapy!”\n\n## Watch our Video Walkthrough\n\nYOUTUBE Link Here -\u003e [YouTube Video](https://www.youtube.com/watch?v=2WJns_VitCA)\n\n## Features\n\n- Upload leaf image for disease prediction\n- ResNet50-based image classifier trained on PlantVillage dataset\n- Flan-T5 Large QA model fine-tuned using LoRA on 130+ domain-specific Q\u0026A pairs\n- Integrated chat interface to ask questions about symptoms, treatments, etc.\n- Domain-tuned answers from curated disease contexts\n\n## Poster\n\n\u003cimg src=\"assets/Poster.png\" alt=\"Poster\" width=\"100%\" /\u003e\n\n### Functional Pipeline\n\n\u003cimg src=\"assets/Application%20flow.png\" alt=\"Pipeline\" width=\"80%\" /\u003e\n\n| Step | Component                       | Description                                        |\n| ---- | ------------------------------- | -------------------------------------------------- |\n| 1    | Image Upload                    | User uploads leaf image via UI                     |\n| 2    | ResNet Classifier               | Predicts disease class with 99.2% accuracy         |\n| 3    | User Question                   | User types a natural-language query                |\n| 4    | QA Model (Flan-T5 Large + LoRA) | Generates context-rich answers using prompt tuning |\n| 5    | Output                          | Returns an actionable answer to the user           |\n\n## Our Approach\n\n| Component            | Details                                                             |\n| -------------------- | ------------------------------------------------------------------- |\n| Image Classification | Fine-tuned**ResNet-50**, frozen first 3 blocks, trained final block |\n| Context Curation     | Manual context generation for 26 plant diseases + 1 healthy case    |\n| QA Dataset           | ~130 Q\u0026A pairs collected per disease (symptoms, causes, treatment)  |\n| Fine-tuning          | Used**LoRA** with Flan-T5-Large via HuggingFace `peft`              |\n| Evaluation           | Metrics:**ROUGE-1**, **ROUGE-2**, **ROUGE-L**, **BLEU**             |\n| Deployment           | Streamlit app, real-time interaction and inference                  |\n\n## Results\n\n### Classification\n\n| Metric                  | Value                   |\n| ----------------------- | ----------------------- |\n| **Training Accuracy**   | 99.5%                   |\n| **Validation Accuracy** | 99.2%                   |\n| **Overfitting**         | No overfitting observed |\n\n### QA Model (Flan-T5-Large + LoRA)\n\n| Metric      | Before Fine-Tune | After Fine-Tune |\n| ----------- | ---------------- | --------------- |\n| **ROUGE-1** | ~22.4            | **59.29**       |\n| **ROUGE-2** | ~11.3            | **36.23**       |\n| **ROUGE-L** | ~20.7            | **57.93**       |\n| **BLEU**    | ~14.0            | **32.06**       |\n\n** Insight:** After fine-tuning, the model adapts significantly to domain-specific phrasing, symptoms, and treatments — yielding more relevant and fluent answers.\n\n## Installation\n\n```bash\ngit clone \u003cyour-repo-url\u003e\ncd \u003cyour-project-folder\u003e\n\n# Create environment\npython3 -m venv venv\nsource venv/bin/activate\n\n# Install dependencies\npip install -r requirements.txt\n\n# Run Streamlit app\nstreamlit run app.py\n```\n\n## Requirements\n\nSee [`requirements.txt`](./requirements.txt) – includes:\n\n```\nstreamlit\ntorch\ntorchvision\nPillow\ntransformers\npeft\ndatasets\nevaluate\nnumpy\n```\n\n## Credits\n\nBuilt by **Darshan \u0026 Akshara**\nCU Boulder – Deep Learning Course\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdarshan-vijay%2Fplant-doc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdarshan-vijay%2Fplant-doc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdarshan-vijay%2Fplant-doc/lists"}