{"id":16494822,"url":"https://github.com/danifranco/timise","last_synced_at":"2025-10-27T21:31:25.713Z","repository":{"id":43772603,"uuid":"490112106","full_name":"danifranco/TIMISE","owner":"danifranco","description":"Toolbox for Identifying Mitochondria Instance Segmentation Errors","archived":false,"fork":false,"pushed_at":"2023-10-15T08:52:30.000Z","size":18283,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-01T08:41:41.270Z","etag":null,"topics":["computer-vision","deep-learning","instance-segmentation","machine-learning","medical-imaging","mitochondria","statistics"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/danifranco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-05-09T02:41:41.000Z","updated_at":"2023-10-08T15:15:14.000Z","dependencies_parsed_at":"2023-02-05T23:17:04.123Z","dependency_job_id":null,"html_url":"https://github.com/danifranco/TIMISE","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danifranco%2FTIMISE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danifranco%2FTIMISE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danifranco%2FTIMISE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/danifranco%2FTIMISE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/danifranco","download_url":"https://codeload.github.com/danifranco/TIMISE/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238560957,"owners_count":19492613,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","deep-learning","instance-segmentation","machine-learning","medical-imaging","mitochondria","statistics"],"created_at":"2024-10-11T14:15:47.509Z","updated_at":"2025-10-27T21:31:20.091Z","avatar_url":"https://github.com/danifranco.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A **T**oolbox for **I**dentifying **M**itochondria **I**nstance **S**egmentation **E**rrors\n\nThe toolbox provides a set of size and morphological measures from the prediction and ground truth, including volume,\nskeleton length and cable length; and a set of evaluation metrics, including the AP used from MitoEM, association\nand matching metrics. Thus, you could analize the errors between these two volumes:\n\n\u003ctable\u003e\n  \u003ctr\u003e\n     \u003ctd\u003eGround Truth\u003c/td\u003e\n     \u003ctd\u003ePrediction\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\u003cimg src=\"https://github.com/danifranco/TIMISE/blob/main/examples/img/toy_gt.gif\" width=200\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cimg src=\"https://github.com/danifranco/TIMISE/blob/main/examples/img/toy_pred.gif\" width=200 \u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n## Installation\nCreate a conda environment with all the dependencies:\n```shell\nconda create -n timise python=3.9\npip install -r requirements.txt\n```\n\n## Usage\n\n```python\nfrom timise import TIMISE\n\ntimise = TIMISE()\n\n# Compute metrics\ntimise.evaluate(\"/home/user/model_xx_prediction_folder\",\"/home/user/gt_folder\", \n                \"/home/user/output_folder\", data_resolution=[30,8,8])\n\n# Summarize the results as tables in the console\ntimise.summarize() \n\n# Plot errors and save in '/home/user/output'\ntimise.plot() \n\n# You can also create neuroglancer scripts to visualize them easily. \n# More info in 'examples/neuroglancer' folder\ntimise.create_neuroglancer_file(\"gt\", categories=['large'])\n```\n\nExample of the tables printed in the console:\n\n```\nStats in /home/user/analysis/output/model_xx_prediction\n\n\n              Average Precision (AP)              \n+---------------+----------+----------+----------+\n| IoU=0.50:0.95 | IoU=0.50 | IoU=0.75 | IoU=0.90 |\n+---------------+----------+----------+----------+\n|     0.087     |  0.369   |  0.035   |   0.0    |\n+---------------+----------+----------+----------+\n\n                                                                                                                   Prediction   \n                                         Ground truth associations                                              false positives \n+----------+-------+------------+---------+-------------------+--------------------+--------------+-------+------+------------+\n| category |       | one-to-one | missing | over-segmentation | under-segmentation | many-to-many | Total |      | background |\n+----------+-------+------------+---------+-------------------+--------------------+--------------+-------+------+------------+\n|  small   | Count |     37     |    7    |         5         |         1          |      0       |   50  |      |     25     |\n|          |   %   |    74.0    |   14.0  |        10.0       |        2.0         |     0.0      |       |      |            |\n|  medium  | Count |     29     |    0    |         3         |         3          |      0       |   35  |      |     -      |\n|          |   %   |   82.86    |   0.0   |        8.57       |        8.57        |     0.0      |       |      |            |\n|  large   | Count |     13     |    0    |         5         |         1          |      0       |   19  |      |     -      |\n|          |   %   |   68.42    |   0.0   |       26.32       |        5.26        |     0.0      |       |      |            |\n|          |       |            |         |                   |                    |              |       |      |            |\n|  TOTAL   | Count |     79     |    7    |         13        |         5          |      0       |  104  |      |     25     |\n|          |   %   |   75.96    |   6.73  |        12.5       |        4.81        |     0.0      |  100  |      |   16.03    |\n+----------+-------+------------+---------+-------------------+--------------------+--------------+-------+------+------------+\n\n                                                                     Matching metrics                                                                    \n+----------+--------+-----+----+----+-----------+--------+----------+-------+--------+--------+-----------------+--------------------+------------------+\n| category | thresh |  fp | tp | fn | precision | recall | accuracy |   f1  | n_true | n_pred | mean_true_score | mean_matched_score | panoptic_quality |\n+----------+--------+-----+----+----+-----------+--------+----------+-------+--------+--------+-----------------+--------------------+------------------+\n|  small   |  0.3   |  41 | 36 | 14 |   0.468   |  0.72  |  0.396   | 0.567 |   50   |   77   |      0.362      |       0.503        |      0.285       |\n|  small   |  0.5   |  57 | 20 | 30 |    0.26   |  0.4   |  0.187   | 0.315 |   50   |   77   |      0.233      |       0.582        |      0.183       |\n|  small   |  0.75  |  77 | 0  | 50 |    0.0    |  0.0   |   0.0    |  0.0  |   50   |   77   |       0.0       |        0.0         |       0.0        |\n|  medium  |  0.3   |  8  | 30 | 5  |   0.789   | 0.857  |  0.698   | 0.822 |   35   |   38   |       0.49      |       0.571        |       0.47       |\n|  medium  |  0.5   |  16 | 22 | 13 |   0.579   | 0.629  |  0.431   | 0.603 |   35   |   38   |      0.391      |       0.622        |      0.375       |\n|  medium  |  0.75  |  34 | 4  | 31 |   0.105   | 0.114  |  0.058   |  0.11 |   35   |   38   |      0.089      |       0.783        |      0.086       |\n|  large   |  0.3   |  24 | 17 | 2  |   0.415   | 0.895  |  0.395   | 0.567 |   19   |   41   |      0.495      |       0.554        |      0.314       |\n|  large   |  0.5   |  30 | 11 | 8  |   0.268   | 0.579  |  0.224   | 0.367 |   19   |   41   |      0.359      |        0.62        |      0.227       |\n|  large   |  0.75  |  40 | 1  | 18 |   0.024   | 0.053  |  0.017   | 0.033 |   19   |   41   |      0.044      |       0.829        |      0.028       |\n|  total   |  0.3   |  73 | 83 | 21 |   0.532   | 0.798  |  0.469   | 0.638 |  104   |  156   |      0.429      |       0.537        |      0.343       |\n|  total   |  0.5   | 103 | 53 | 51 |    0.34   |  0.51  |  0.256   | 0.408 |  104   |  156   |      0.309      |       0.607        |      0.247       |\n|  total   |  0.75  | 151 | 5  | 99 |   0.032   | 0.048  |   0.02   | 0.038 |  104   |  156   |      0.038      |       0.792        |       0.03       |\n+----------+--------+-----+----+----+-----------+--------+----------+-------+--------+--------+-----------------+--------------------+------------------+\n```\n\nAnd a summary plot for your model's errors:                                                                                                                                  \n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/danifranco/TIMISE/blob/main/examples/img/toy_summary.png\" alt=\"summary_plot\" width=\"500\"/\u003e\n\u003c/p\u003e\n\n## Details\nTwo different workflows are implemented:\n- When no more folders are found inside the input path, e.g. ``/home/user/model_xx_prediction_folder`` in this example, the file ``.h5`` or ``.tif`` inside that folder will be evaluated against gt file in ``/home/user/gt_folder``. Find an example in this notebook: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/danifranco/TIMISE/blob/main/examples/TIMISE_one_method_evaluation_example.ipynb)\n\n- When more folders are found inside the input path e.g. ``/home/user/model_xx_prediction_folder`` in this example. In this case every folder will be processed as if it were a method to be evaluated, so each folder must contain its own ``.h5``/``.tif`` file. This option is usefull when multiple models' predictions need to be evaluated at once. Apart for individual plots this workflow also allows the creation of a general plot gathering the results of all methods. Find an example in this notebook: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/danifranco/TIMISE/blob/main/examples/TIMISE_multiple_method_evaluation_example.ipynb)\n\n## Jupyter Notebook\nCheck out the jupyter notebooks in [examples folder](https://github.com/danifranco/TIMISE/blob/main/examples) for every type of plot than can be generated with TIMISE for the two workflows described above. Please note that some graphics do not display correctly on Github, so we suggest you open the Colab links above to see them correctly. \n\n## Understanding output folder files\nThese are the files that the toolbox will create:\n\n```shell\n/home/user/output_folder/model_xx_prediction_folder/\n├─ gt_group_file.txt (file)\n├─ gt_stats.csv (file)\n├─ *.svg (files) \n└─ model_xx_prediction_folder/\n   ├─ associations.csv (file)\n   ├─ associations_stats.csv (file)\n   ├─ gt_final.csv (file)\n   ├─ map_map.txt (file)\n   |─ map_match_fn.txt (file)\n   ├─ map_match_p.txt (file)\n   ├─ matching_metrics.csv (file)\n   ├─ prediction_stats.csv (file)\n   ├─ pred_gt_matching_info.csv (file)\n   ├─ pred_group_file.txt (file)\n   ├─ map_aux_files(folder)\n   └─ plots (folder)\n```\n- ``gt_group_file.txt``: each GT instance category database. Currently done by skeleton length.\n- ``gt_stats.csv``: statistics of GT instances. \n- ``*.svg``: optional .svg plots generated when multiple methods are available.\n- ``associations.csv``: associations between predicted and gt instances.\n- ``associations_stats.csv``: summary of the associations to print them easily.\n- ``gt_final.csv``: gt statistics mixed with the association errors. Used to generate the final plots easily.\n- ``map_map.txt``: mAP results.\n- ``map_match_fn.txt``: rest of matchings between prediction and GT. False negatives are here.\n- ``map_match_p.txt``: IoU matching between prediction and GT (per category). False positives are here too.\n- ``matching_metrics.csv``: summary of the matching metrics to print them easily.\n- ``pred_group_file.txt``: each prediction instance category database. Currently done by skeleton length.\n- ``pred_gt_matching_info.csv``: IoU matching between prediction and GT as in map_match_p.txt but in csv format so it can be loaded easily into a dataframe. \n- ``prediction_stats.csv``: statistics of predicted instances.   \n- ``map_aux_files``: directory where some auxiliary files of the metrics evaluation will be store so the calculation goes faster.\n- ``plots``: folder where all the plots are stored.\n\n## Citation\nUnder construction . . .\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanifranco%2Ftimise","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdanifranco%2Ftimise","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdanifranco%2Ftimise/lists"}