{"id":20453163,"url":"https://github.com/pietrobarbiero/logic_explained_networks","last_synced_at":"2025-10-10T04:40:07.232Z","repository":{"id":62567428,"uuid":"323039527","full_name":"pietrobarbiero/logic_explained_networks","owner":"pietrobarbiero","description":"Logic Explained Networks is a python repository implementing explainable-by-design deep learning models.","archived":false,"fork":false,"pushed_at":"2023-06-23T08:10:09.000Z","size":12543,"stargazers_count":49,"open_issues_count":0,"forks_count":10,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-21T15:49:49.000Z","etag":null,"topics":["deep-learning","explainable-ai","logic","machine-learning","neural-networks","xai"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pietrobarbiero.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-12-20T09:51:19.000Z","updated_at":"2025-03-19T08:49:32.000Z","dependencies_parsed_at":"2023-01-29T04:01:05.018Z","dependency_job_id":null,"html_url":"https://github.com/pietrobarbiero/logic_explained_networks","commit_stats":null,"previous_names":["pietrobarbiero/deep-logic"],"tags_count":2,"template":false,"template_full_name":"pietrobarbiero/project-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pietrobarbiero%2Flogic_explained_networks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pietrobarbiero%2Flogic_explained_networks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pietrobarbiero%2Flogic_explained_networks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pietrobarbiero%2Flogic_explained_networks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pietrobarbiero","download_url":"https://codeload.github.com/pietrobarbiero/logic_explained_networks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248659780,"owners_count":21141181,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","explainable-ai","logic","machine-learning","neural-networks","xai"],"created_at":"2024-11-15T11:11:55.466Z","updated_at":"2025-10-10T04:40:07.125Z","avatar_url":"https://github.com/pietrobarbiero.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Welcome to the Logic Explained Networks (LENs)\n=======\n\n![](len.jpg)\n\nThe Logic Explained Network is a python repository providing a set of utilities and modules to\nbuild deep learning models that are explainable by design.\nThis library provides both already implemented LENs classes and APIs classes to get First-Order Logic (FOL) explanations from neural networks.\nPaper\n\nThe idea proposed in this library has been published in the [Logic\nExplained\nNetworks](https://www.sciencedirect.com/science/article/pii/S000437022200162X)\npaper, published in the Artificial Intelligence journal.\nA publicly available version can be downloaded from\n[ArXiv](https://arxiv.org/abs/2108.05149).\n\nExample\n=======\n\n### Import LENs\n\n``` {.python}\nimport lens\nimport torch\n```\n\n### Create train, validation and test datasets\n\nLet's create a XOR-like datasets with 2 redundant features (the 3rd and\n4th).\n\n``` {.python}\nlens.utils.base.set_seed(0)\nx = torch.rand([100, 4])\ny = (x[:, 0] \u003e 0.5) \u0026 (x[:, 1] \u003c 0.5) | \\\n    (x[:, 0] \u003c 0.5) \u0026 (x[:, 1] \u003e 0.5)\n\ndata = torch.utils.data.TensorDataset(x, y)\n\ntrain_data, val_data, test_data = torch.utils.data.random_split(data, [80, 10, 10])\nx_train, y_train = data[train_data.indices]\nx_val, y_val = data[val_data.indices]\nx_test, y_test = data[test_data.indices]\n```\n\n### Instantiate a MuNN \n\n``` {.python}\nmodel = lens.models.XMuNN(n_classes=2, n_features=4,\n                          hidden_neurons=[10], loss=torch.nn.CrossEntropyLoss())\n```\n\n### Train the model \n``` {.python}\nmodel.fit(train_data, val_data, epochs=100, l_r=0.1)\n\n## get accuracy on test samples\ntest_acc = model.evaluate(test_data)\nprint(\"Test accuracy:\", test_acc)\n```\n\n    Epoch: 1/100, Loss: 0.733, Tr_acc: 46.88, Val_acc: 53.00, best_e: -1\n    Epoch: 2/100, Loss: 0.733, Tr_acc: 53.12, Val_acc: 53.00, best_e: -1\n    Epoch: 3/100, Loss: 0.706, Tr_acc: 53.12, Val_acc: 65.00, best_e: -1\n    Epoch: 4/100, Loss: 0.696, Tr_acc: 58.50, Val_acc: 46.00, best_e: -1\n    ...\n    Epoch: 43/100, Loss: 0.178, Tr_acc: 94.50, Val_acc: 91.00, best_e: -1\n    Epoch: 44/100, Loss: 0.169, Tr_acc: 94.12, Val_acc: 92.00, best_e: -1\n    Epoch: 45/100, Loss: 0.164, Tr_acc: 94.62, Val_acc: 91.00, best_e: -1\n    Epoch: 46/100, Loss: 0.156, Tr_acc: 95.88, Val_acc: 92.00, best_e: -1\n    Epoch: 47/100, Loss: 0.149, Tr_acc: 96.00, Val_acc: 92.00, best_e: -1\n    Epoch: 48/100, Loss: 0.144, Tr_acc: 97.12, Val_acc: 93.00, best_e: -1\n    Epoch: 49/100, Loss: 0.139, Tr_acc: 97.12, Val_acc: 93.00, best_e: -1\n    Pruned 2/4 features\n    Pruned 2/4 features\n    Pruned features\n    Epoch: 50/100, Loss: 0.133, Tr_acc: 97.62, Val_acc: 93.00, best_e: -1\n    Epoch: 51/100, Loss: 0.140, Tr_acc: 94.62, Val_acc: 78.00, best_e: 51\n    Epoch: 52/100, Loss: 0.363, Tr_acc: 81.88, Val_acc: 90.00, best_e: 52\n    Epoch: 53/100, Loss: 0.146, Tr_acc: 95.62, Val_acc: 92.00, best_e: 53\n    ...\n    Epoch: 54/100, Loss: 0.165, Tr_acc: 92.00, Val_acc: 88.00, best_e: 53\n    Epoch: 96/100, Loss: 0.099, Tr_acc: 98.12, Val_acc: 98.00, best_e: 89\n    Epoch: 97/100, Loss: 0.099, Tr_acc: 98.12, Val_acc: 98.00, best_e: 89\n    Epoch: 98/100, Loss: 0.098, Tr_acc: 98.38, Val_acc: 98.00, best_e: 89\n    Epoch: 99/100, Loss: 0.097, Tr_acc: 98.88, Val_acc: 97.00, best_e: 89\n    Epoch: 100/100, Loss: 0.096, Tr_acc: 98.75, Val_acc: 97.00, best_e: 89\n    Test accuracy: 99.0\n\n### Extract and evaluate global explanation\n\n``` {.python}\n## get first order logic explanations for a specific target class\ntarget_class = 1\nconcept_names = ['x1', 'x2', 'x3', 'x4']\nformula = model.get_global_explanation(x_train, y_train, target_class,\n                                       top_k_explanations=2, concept_names=concept_names)\nprint(f\"{formula} \u003c-\u003e f{target_class}\")\n\n## compute explanation accuracy\nexp_accuracy, _ = lens.logic.test_explanation(formula, target_class, x_test, y_test,\n                                              concept_names=concept_names)\nprint(\"Logic Test Accuracy:\", exp_accuracy)\n```\n\n    (x1 \u0026 ~x2) | (x2 \u0026 ~x1) \u003c-\u003e f1\n    Logic Test Accuracy: 100.0\n\n### Plot decision boundaries and explanations\n\n![](examples/output_12_0.png)\n\n\nStructure of the repository \n=============\n\n``` {.}\nLENs\n├── data\n├── examples\n│     ├── example.py\n|     └── api_examples\n├── experiments\n├── lens\n|     ├── models\n│     ├── logic\n│         ├── mu_nn\n│         ├── psi_nn\n│         └── relu_nn\n|     └── utils\n└── tests\n    ├── test_models\n    ├── test_logic_base\n    └── test_concept_extractors\n```\n### data\nIt contains some functions to download and structure the datasets as used in the experiments\n\n### examples\nIt contains the simple ``xor_example`` reported above, explaining how to train and extract FOL explanations of a LEN.  \nSeveral examples on how to extract explanations from any models are reported in ``api_examples``\n\n### experiments\nHere are reported the notebooks used in the paper to test the models. each notebook contains the comparison on one dataset.\nAlso you can find the exp. against adversarial attacks on cub ``cub_adv.ipynb``, \nthe exp. on explaining a black-box models on mimic ``mimic_ebb.ipynb``, and the experiment on interpretable clustering ``mnist_iclu.ipynb``.\n\n### lens.models\nThe most important folder is of course ``lens.models`` where you can find all the proposed Logic Explained Networks: \n*   The $\\psi$ Network \n*   The $\\mu$ Network \n*   The ReLu Network. \n\nThese models are explainable by design and they all offer two methods to extract explanations:\n*   ``.get_local_explanation()`` which returns the local explanation for one sample prediction\n*   ``.get_global_explanation()`` which returns the global explanation for an entire class\n\nAlso, in models are implemented a number of white-box classifiers and provide the same functions. \nAmong others, *BRL*, *Decision tree*, *Logistic regression*, \na black box model explained with *Anchors*, and *Deep Red* (the implementation of the latter has still some issues).\n\nIn the same folder are also reported two black box models *Random forest* and a *Black-box* NN. \n\nFinally a *Robust CNN* is reported. It employs the explanations of a LEN to defend from adversarial attacks. \n\n### lens.logic\nIn this folder we implemented all the API required to extract FOL explanations of a model. \nMost of the functions are not-strcitly related with any model and therefore can be also used in general as reported in ``examples.api_examples``.\n\n### lens.utils\nHere are reported several utilities functions used in the models and in the experiments. \nAn important utility file is ``lens.utils.dataset``. \nIt contains all the dataset classes used to implement the different concept bottleneck architectures.\n\n### tests\nTesting utilities of the main functions and classes. \n\n\nCitations \n============= \n### Logic Explained Networks\nTo cite the Logic Explained Network paper use the following bibtex:\n\n    @article{ciravegna2022logic,\n      title={Logic explained networks},\n      author={Ciravegna, Gabriele and Barbiero, Pietro and Giannini, Francesco and Gori, Marco and Li{\\'o}, Pietro and Maggini, Marco and Melacci, Stefano},\n      journal={Artificial Intelligence},\n      pages={103822},\n      year={2022},\n      publisher={Elsevier}\n    }\n\n### Entropy LEN: *Entropy-based logic explanations of neural networks*\n    @inproceedings{barbiero2022entropy,\n      title={Entropy-based logic explanations of neural networks},\n      author={Barbiero, Pietro and Ciravegna, Gabriele and Giannini, Francesco and Li{\\'o}, Pietro and Gori, Marco and Melacci, Stefano},\n      booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},\n      volume={36},\n      number={6},\n      pages={6046--6054},\n      year={2022}\n    }\n\n### Extending Logic Explained Networks to Text Classification\n    @misc{https://doi.org/10.48550/arxiv.2211.09732,\n      doi = {10.48550/ARXIV.2211.09732},\n      url = {https://arxiv.org/abs/2211.09732},\n      author = {Jain, Rishabh and Ciravegna, Gabriele and Barbiero, Pietro and Giannini, Francesco and Buffelli, Davide and Lio, Pietro},\n      keywords = {Computation and Language (cs.CL), Artificial Intelligence (cs.AI), FOS: Computer and information sciences, FOS: Computer and information sciences},\n      title = {Extending Logic Explained Networks to Text Classification},\n      publisher = {arXiv},\n      year = {2022},\n      copyright = {Creative Commons Attribution Non Commercial Share Alike 4.0 International}\n    }\n\n\n## Theoretical foundations\nTheoretical foundations of this work can be found in the following papers:\n\n### Explaining a network with another network: *Human-Driven FOL Explanations of Deep Learning*\n    @inproceedings{ciravegna2020human,\n      title={Human-Driven FOL Explanations of Deep Learning.},\n      author={Ciravegna, Gabriele and Giannini, Francesco and Gori, Marco and Maggini, Marco and Melacci, Stefano},\n      booktitle={IJCAI},\n      pages={2234--2240},\n      year={2020}\n    }\n\n### Learning of constraints: *A Constraint-based Approach to Learning and Explanation*\n\n    @inproceedings{ciravegna2020constraint,\n      title={A Constraint-Based Approach to Learning and Explanation.},\n      author={Ciravegna, Gabriele and Giannini, Francesco and Melacci, Stefano and Maggini, Marco and Gori, Marco},\n      booktitle={AAAI},\n      pages={3658--3665},\n      year={2020}\n    }\n\n\nAuthors\n-------\n\n-   [Gabriele\n    Ciravegna](https://team.inria.fr/maasai/gabriele-ciravegna/),\n    Université Cote d\\'Azur, FR.\n-   [Pietro Barbiero](http://www.pietrobarbiero.eu/), University of\n    Cambridge, UK.\n-   [Francesco\n    Giannini](http://sailab.diism.unisi.it/people/francesco-giannini/),\n    University of Siena, IT.\n\nLicence\n-------\n\nCopyright 2022 Gabriele Ciravegna, Pietro Barbiero, Francesco Giannini.\n\nLicensed under the Apache License, Version 2.0 (the \\\"License\\\"); you\nmay not use this file except in compliance with the License. You may\nobtain a copy of the License at:\n\u003chttp://www.apache.org/licenses/LICENSE-2.0\u003e.\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \\\"AS IS\\\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n\nSee the License for the specific language governing permissions and\nlimitations under the License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpietrobarbiero%2Flogic_explained_networks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpietrobarbiero%2Flogic_explained_networks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpietrobarbiero%2Flogic_explained_networks/lists"}