{"id":20619910,"url":"https://github.com/mafda/knee_oa_dl_app","last_synced_at":"2025-10-25T20:06:46.294Z","repository":{"id":81393533,"uuid":"587967006","full_name":"mafda/knee_OA_dl_app","owner":"mafda","description":"Web app to predict knee osteoarthritis grade using Deep Learning and Streamlit","archived":false,"fork":false,"pushed_at":"2023-07-25T16:19:20.000Z","size":6740,"stargazers_count":11,"open_issues_count":0,"forks_count":7,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-15T12:12:36.204Z","etag":null,"topics":["convolutional-neural-networks","data-science","deep-neural-networks","knee-osteoarthritis","knee-osteoarthritis-analysis","ml-app","ml-application","streamlit","x-ray-images"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mafda.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-12T02:10:28.000Z","updated_at":"2024-06-28T13:57:27.000Z","dependencies_parsed_at":"2024-11-20T11:02:55.132Z","dependency_job_id":null,"html_url":"https://github.com/mafda/knee_OA_dl_app","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafda%2Fknee_OA_dl_app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafda%2Fknee_OA_dl_app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafda%2Fknee_OA_dl_app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mafda%2Fknee_OA_dl_app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mafda","download_url":"https://codeload.github.com/mafda/knee_OA_dl_app/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249067779,"owners_count":21207396,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-networks","data-science","deep-neural-networks","knee-osteoarthritis","knee-osteoarthritis-analysis","ml-app","ml-application","streamlit","x-ray-images"],"created_at":"2024-11-16T12:12:48.752Z","updated_at":"2025-10-25T20:06:46.243Z","avatar_url":"https://github.com/mafda.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Knee Osteoarthritis Analysis with X-ray Images using Deep Convolutional Neural Networks\n\nThis code repository is the final project of the Complex Data Mining course at\nUnicamp (MDC013).\n\n## Knee Osteoarthritis\n\n[Knee osteoarthritis](https://en.wikipedia.org/wiki/Knee_arthritis) is a\npathology that occurs due to wear on the cartilage that protects the bones in\nthis region from friction and impacts.\n\nSome medical procedures are necessary to identify this pathology, such as\n**X-rays** or magnetic resonance imaging, in which it is possible to assess the\nloss in joint spacing, thus indicating the severity of the disease.\n\nThe severity of osteoarthritis was classified into 5 levels based on [KL\nscore](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4925407/), from the healthy\nlevel to the severe level, where the greater the degree of severity, the smaller\nthe spacing of the joint.\n\n- KL score:\n    - Grade 0: Healthy\n    - Grade 1: Doubtful \n    - Grade 2: Minimal \n    - Grade 3: Moderate \n    - Grade 4: Severe\n\nThe following image shows the different levels from [Knee Osteoarthritis Dataset\nwith Severity\nGrading](https://www.kaggle.com/datasets/shashwatwork/knee-osteoarthritis-dataset-with-severity).\n\n![KL Score](assets/kl-score.png)\n\n## Purpose\n\nThe purpose of this project is to correctly classify the severity of\nosteoarthritis based on X-ray images.\n\n![streamlit app - knee_dl_app](assets/streamlit_knee_ss.png)\n\n## Project Structure\n\n```shell\n.\n├── README.md\n├── app\n│   ├── app.py\n│   └── img\n├── assets\n├── dataset\n│   ├── test\n│   ├── train\n│   └── val\n├── environment.yml\n└── src\n    ├── 01_data_preparation.ipynb\n    ├── 02_ensemble_models.ipynb\n    ├── 02_model_inception_resnet_v2.ipynb\n    ├── 02_model_resnet50.ipynb\n    ├── 02_model_xception.ipynb\n    ├── 03_best_model_on_test_xception.ipynb\n    └── models\n        └── model_Xception_ft.hdf5\n```\n\n## Project Setup\n\nHow does the Web App to **predict** knee osteoarthritis grade using **Deep\nLearning** and **Streamlit** works?\n\n### Clone this repository\n\n```shell\n(base)$: git clone git@github.com:mafda/knee_OA_dl_app.git\n(base)$: cd knee_OA_dl_app\n```\n\n### Configure environment\n\n- Create the conda environment\n\n    ```shell\n    (base)$: conda env create -f environment.yml\n    ```\n\n- Activate the environment\n\n    ```shell\n    (base)$: conda activate knee_dl_app\n    ```\n\n- Download `model_Xception_ft.hdf5` model from\n   [here](https://drive.google.com/file/d/1vPS_j2AW3M1W8GydREEDw6CDENSgVwCy/view?usp=share_link)\n   to run the application. Create `models` folder and copy the model here.\n\n    ```shell\n    (knee_dl_app)$: mkdir src/models\n    ```\n\n- Download the dataset from [Knee Osteoarthritis Dataset with Severity\n   Grading](https://www.kaggle.com/datasets/shashwatwork/knee-osteoarthritis-dataset-with-severity)\n   to train the model and test the application. Create `dataset` folder and copy\n   the data here.\n\n    ```shell\n    (knee_dl_app)$: mkdir dataset\n    ```\n\n- Run it:\n\n    ```shell\n    (knee_dl_app)$: streamlit run app/app.py\n    ```\n\n    As soon as you run the script, a local Streamlit server will spin up, and\n    your app will open in a new tab in your default web browser.\n\n    Or you can navigate to `http://localhost:8501`.\n\n## Methodology\n\nThe following methodology has been proposed to correctly classify the degree of\nosteoarthritis based on X-ray images:\n\n1. Data preparation\n2. Model training\n3. Model evaluation\n4. Web application development with the best model\n\n### Data preparation\n\nThe dataset consisting of 8000 X-ray images, approximately, of the knee obtained\nfrom the [Knee Osteoarthritis\nDataset](https://www.kaggle.com/datasets/shashwatwork/knee-osteoarthritis-dataset-with-severity).\n\n![data](assets/data.png)\n\nIn the *bar chart* we can see the image distribution of the 5 grades (classes),\nfor each of the training, validation and test datasets, and in the *pie chart*\nwe can see the average percentage of data we have for each class. So we have an\n**unbalanced** dataset.\n\nThree strategies were implemented to reduce the impact that the unbalanced base\ncan have on the models:\n\n- class weight\n- data augmentation (horizontal_flip, brightness_range, width_shift_range,\n  zoom_range)\n- preprocessing features of pre-trained networks\n\n\u003e See [01_data_preparation.ipynb](src/01_data_preparation.ipynb) notebook.\n\n### Model training\n\n#### Pre-trained Networks\n\nThree pre-trained networks were chosen:\n[Xception](https://arxiv.org/abs/1610.02357),\n[ResNet-50](https://arxiv.org/abs/1512.03385) e [Inception Resnet\nv2](https://arxiv.org/abs/1602.07261v2). \n\nThe following table presents the summary of the partial results obtained in the\nimplementation of the different pre-trained networks with fine tuning. Our\nmetrics is Balanced Accuracy. Models were trained on Apple M1 Pro chip with\n8-core CPU, 14-core GPU and 16-core Neural Engine.\n\n| Model                           | Balanced Accuracy | Time Execution |\n| ------------------------------- | ----------------- | -------------- |\n| Xception fine tuning            | 67%               | 68min          |\n| ResNet50 fine tuning            | 65%               | 80min          |\n| Inception_resnet_v2 fine tuning | 64%               | 56min          |\n\n![Confusion Matrix](assets/confusion_matrix_3_models.png)\n\nHighlighting, the highest success rate of each model by class, we have:\n\n- Inception ResNet classified the minimal class better\n- Xception classified the doubtful and severe classes better\n- ResNet50 classified the healthy and moderate classes better\n\n\u003e See [02_model_inception_resnet_v2.ipynb](src/02_model_inception_resnet_v2.ipynb)\n\n\u003e See [02_model_xception.ipynb](src/02_model_xception.ipynb)\n\n\u003e See [02_model_resnet50.ipynb](src/02_model_resnet50.ipynb)\n   \n#### Ensemble\n\nThree ensemble approaches were performed with the previous results:\n\n- One calculating the average, and two calculating the weighted average of the\n  accuracy and f1 of each class.\n\n| Model             | Balanced Accuracy | Time Execution |\n| ----------------- | ----------------- | -------------- |\n| Ensemble mean     | 68.63%            | 16seg          |\n| Ensemble accuracy | 68.48%            | 16seg          |\n| Ensemble f1       | 68.69%            | 16seg          |\n\n![ensemble](assets/ensemble.png)\n\nThe three models had similar results, but we selected the **ensemble with f1 model**.\n\n\u003e See [02_ensemble_models.ipynb](src/02_ensemble_models.ipynb)\n\n### Model evaluation\n\nWe evaluated the [best model](src/02_ensemble_models.ipynb) in the test set, a\nbalanced accuracy of 71% was obtained, and in the confusion matrix we can\nobserve the highlight of the moderate and severe classes.\n\n![ensemble test](assets/ensemble_test.png)\n\n#### Explainability\n\nWe implemented the [Grad-CAM](https://arxiv.org/abs/1610.02391) explainability\ntechnique to better understand how classes are classified. The Grad-CAM\nindicates the parts of the image that most impact the classification score.\n\nWe can see in the images that for the healthy, doubtful and minimal classes, the\nmost prominent areas are located in the center of the knee, and the moderate and\nsevere classes are most prominent on the right or left edges of the knee.\n\nGrad-CAM results were obtained from the last convolutional layer of the\n[Xception](src/02_model_xception.ipynb) model.\n\n![Healthy](assets/Healthy.png)\n![Doubtful](assets/Doubtful.png)\n![Minimal](assets/Minimal.png)\n![Moderate](assets/Moderate.png)\n![Severe](assets/Severe.png)\n\n\u003e See [03_best_model_on_test_xception.ipynb](src/03_best_model_on_test_xception.ipynb)\n\n### Web application development with the best model\n\nThe web application allows you to select and load an X-Ray image, to later\npredict and evaluate the loss in joint spacing, and indicate the probability of\ndisease severity, as well as the area that most impacted the classification\nscore.\n\n![streamlit app - knee_dl_app](assets/streamlit_knee_low.gif)\n\n#### [pt-br] YouTube\n\n[![Análise da Gravidade de Artrose no Joelho](https://img.youtube.com/vi/gl28zQAs_rk/0.jpg)](https://www.youtube.com/watch?v=gl28zQAs_rk \"Análise da Gravidade de Artrose no Joelho\")\n\n## References\n\n- [Complex data mining](https://www.ic.unicamp.br/~mdc/) at\n  [unicamp](https://www.unicamp.br/unicamp/).\n- [Knee Osteoarthritis Dataset with Severity\n  Grading](https://www.kaggle.com/datasets/shashwatwork/knee-osteoarthritis-dataset-with-severity)\n\n---\n\nmade with 💙 by [mafda](https://mafda.github.io/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmafda%2Fknee_oa_dl_app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmafda%2Fknee_oa_dl_app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmafda%2Fknee_oa_dl_app/lists"}