{"id":13565765,"url":"https://github.com/nyuvis/explanation_explorer","last_synced_at":"2026-01-25T18:12:35.194Z","repository":{"id":86217958,"uuid":"88910201","full_name":"nyuvis/explanation_explorer","owner":"nyuvis","description":"A user interface to interpret machine learning models.","archived":false,"fork":false,"pushed_at":"2020-01-14T16:04:59.000Z","size":15138,"stargazers_count":71,"open_issues_count":0,"forks_count":4,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-11-04T19:42:10.605Z","etag":null,"topics":["interpretability","machine-learning","visual-interface","visualization-application"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nyuvis.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-04-20T21:02:06.000Z","updated_at":"2024-05-20T17:00:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"f303cae4-df9d-4294-91e0-bd2f20ab28bb","html_url":"https://github.com/nyuvis/explanation_explorer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nyuvis%2Fexplanation_explorer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nyuvis%2Fexplanation_explorer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nyuvis%2Fexplanation_explorer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nyuvis%2Fexplanation_explorer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nyuvis","download_url":"https://codeload.github.com/nyuvis/explanation_explorer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247097586,"owners_count":20883121,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["interpretability","machine-learning","visual-interface","visualization-application"],"created_at":"2024-08-01T13:01:55.065Z","updated_at":"2026-01-25T18:12:35.158Z","avatar_url":"https://github.com/nyuvis.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# Explanation Explorer\n\nThe **Explanation Explorer** is a visual interface to explore similarly explained data items.\nHaving a trained machine learning model it is possible to create explanations for\ndata items by probing model inputs.\nThe visual analytics interface groups similar explanations together and provides\nan interactive way of exploring the significants and quantity of those explanations\nin a given data set, i.e., a validation data set.\n\nYou can find a [live demo here](http://nyuvis-web.poly.edu/projects/explanation_explorer/)\nwhich uses an example data set [below](#airbnb).\n\n[![Overview of the Explanation Explorer](explainer.png)](https://vimeo.com/235631465)\n\n## Publications\n\n[A Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanations](https://arxiv.org/abs/1705.01968);\nJosua Krause, Aritra Dasgupta, Jordan Swartz, Yindalon Aphinyanaphongs, Enrico Bertini;\nPublished at IEEE VAST 2017.\n[[slides]](explainer.key?raw=true)\n[[talk]](https://vimeo.com/238852188)\n[[video]](https://vimeo.com/235631465)\n\n## Getting started\n\nThe project consists of a `python` server with a web front-end.\nTo get started clone the repository and run:\n\n```bash\ngit submodule update --init --recursive\npip install -r requirements.txt\n```\n\nAfter that the project is ready to run.\n\n```bash\n./server.py input.csv expl.json\n```\n\nwhere `input.csv` and `expl.json` are files as described [below](#input-format)\nor as created with [`create_explanations.py`](#creating-explanations).\nOnce the server is started navigate to the URL as prompted in the server output.\n\nRun `./server.py -h` to get a list of all input arguments.\n\n## Creating explanations\n\nIn order to create explanations you can implement a subclass of the `Model`\nclass in `defs.py`. Then you can call `create_explanations.py` with:\n\n```bash\n./create_explanations.py yourfile YourModel output\n```\n\nwhere `yourfile` is a relative python module path (ie., `yourfile.py` is a\nfile in the current folder) which contains the definition of `YourModel` which\nis a subclass of `defs.Model`. `output` is the folder where the two output\nfiles (the input to `./server.py`) are written to.\n\n`./create_explanations.py` uses an adaption of the [LIME algorithm](https://arxiv.org/abs/1602.04938) by default.\n\n### Airbnb\n\n`example_airbnb.py` contains an example implementation for a text data set\n(whether a place has good ratings based on its description --\nyou can find the jupyter notebook that was used for creating the data set in `example/airbnb/`)\nand can be used like this:\n\n```bash\n./create_explanations.py example_airbnb AirbnbModel output\n```\n\nAnd the server can then be started via:\n\n```bash\n./server.py output/airbnbmodel.csr output/airbnbmodel.lime.json\n```\n\n### Mushrooms\n\n`example_mushroom.py` contains an example implementation for a categorical data set\n(whether mushrooms are edible given certain physical features)\nand can be used like this:\n\n```bash\n./create_explanations.py example_mushroom MushroomModel output\n```\n\nAnd the server can then be started via:\n\n```bash\n./server.py output/mushroommodel.csv output/mushroommodel.lime.json\n```\n\n## Input format\n\nAs of now the project requires two input files.\nThe input data and the explanation description.\n\nThe input data has to be a CSV file representing a binary data matrix.\nThe first row contains the column names.\nThere is one special columns\n`label` (containing the ground truth label `0` or `1`) and\nthe file should only contain rows of the validation data set.\nOptionally, the input data can be stored as CSR file (`*.csr`) which is a\nCSV file where the first row is `label` followed by the feature names and\nthe following rows contain the label as first element and then the indices\nof the features that are 1 (the data has to be binary for that).\n\nThe explanation description is a JSON file of the following format:\n\n```javascript\n{\n  \"test_auc\": 0.85, // area under ROC curve for the explained set\n  \"train_auc\": 0.9, // area under ROC curve for the training set\n  \"total_rows\": 135, // number of rows in the input data that belong to the explained set (used for integrity check)\n  \"total_true\": 71, // number of rows with a `1` label in the input data that belong to the explained set (used for integrity check)\n  \"threshold\": 0.6, // the optimal threshold minimizing incorrectly predicted training instances\n  \"features\": [ \"foo\", \"bar\", ... ], // names of the features corresponding to the input data\n  \"expls\": [ // array of explanations for each data item in order of the ixs array\n    {\n      \"ix\": 0, // the index of the current item starting at 0\n      \"label\": 0, // the ground truth label 0 or 1\n      \"pred\": 0.1, // prediction score\n      \"pred_label\": 0, // the predicted label 0 or 1 using the threshold from above (used for integrity check)\n      \"expl\": [ // the explanation\n        [ 12, \"+\" ], // one step: feature index in features array, feature prefix (can be \"\")\n        // ...\n      ],\n      \"postfixes\": [ // postfixes to put after feature names -- corresponds to the features array\n        \"=[0, 10)\",\n        \"=5\",\n        \"\", // empty string if no postfix should be added\n        None, // ignore feature when listing\n        // ...\n      ],\n    },\n    // ...\n  ],\n}\n```\n\n### Older formats\n\nUsing `--protocol 0` as additional command line argument enables using the legacy\nformat.\nHere, the CSV file has an additional column\n`pred` (containing the prediction score of the current row between `0.0` and `1.0`).\nAlso, ideally, the file should contain both rows of the training\ndata set (optional) and validation data set (needed).\n\nThe explanation description is a JSON file of the following format:\n\n```javascript\n{\n  \"auc\": 0.85, // area under ROC curve for the explained set\n  \"train_auc\": 0.9, // area under ROC curve for the training set\n  \"ixs\": [ 0, 1, 2, ... ], // indices of rows in the input data that belong to the explained set\n  \"train_ixs\": [ 135, 136, 137, ... ], // indices of rows in the input data that belong to the training set\n  \"total_features\": 1400, // number of features (used for integrity check)\n  \"total_rows\": 135, // number of rows in the input data that belong to the explained set (used for integrity check)\n  \"total_true\": 71, // number of rows with a `1` label in the input data that belong to the explained set (used for integrity check)\n  \"train_preds\": [ 0.2, 0.113, ... ], // prediction scores for training data in order of the train_ixs array\n  \"features\": [ \"foo\", \"bar\", ... ], // names of the features corresponding to the input data\n  \"expls\": [ // array of explanations for each data item in order of the ixs array\n    {\n      \"file\": null, // unused\n      \"meta\": \"\", // unused\n      \"ix\": 0, // the index of the current item (must correspond to the ixs array)\n      \"label\": 0, // the ground truth label 0 or 1\n      \"pred\": 0.1, // prediction score\n      \"down\": [ // removing features (aka. setting features to 0) to reduce the prediction score\n        [ 12, 0.09, [] ], // one step: feature index in features array, new prediction score, re-addable feature list\n        // ...\n      ],\n      \"up\": [ // removing features to increase the prediction score\n        [ 24, 0.11, [] ], // one step: feature index in features array, new prediction score, re-addable feature list\n        // ...\n      ],\n    },\n    // ...\n  ],\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnyuvis%2Fexplanation_explorer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnyuvis%2Fexplanation_explorer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnyuvis%2Fexplanation_explorer/lists"}