{"id":20194483,"url":"https://github.com/airtai/monotonic-nn","last_synced_at":"2025-04-09T07:06:21.507Z","repository":{"id":135549245,"uuid":"494411585","full_name":"airtai/monotonic-nn","owner":"airtai","description":"Keras implementation of the constrained monotonic neural networks","archived":false,"fork":false,"pushed_at":"2025-03-27T06:57:55.000Z","size":22870,"stargazers_count":39,"open_issues_count":5,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-02T06:09:05.616Z","etag":null,"topics":["keras","monotonic","neural-networks","tensorflow"],"latest_commit_sha":null,"homepage":"https://monotonic.airt.ai","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/airtai.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-05-20T09:58:39.000Z","updated_at":"2025-03-14T06:05:58.000Z","dependencies_parsed_at":"2024-03-08T14:46:07.195Z","dependency_job_id":null,"html_url":"https://github.com/airtai/monotonic-nn","commit_stats":null,"previous_names":["airtai/monotonic-nn"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airtai%2Fmonotonic-nn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airtai%2Fmonotonic-nn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airtai%2Fmonotonic-nn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/airtai%2Fmonotonic-nn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/airtai","download_url":"https://codeload.github.com/airtai/monotonic-nn/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247994119,"owners_count":21030050,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["keras","monotonic","neural-networks","tensorflow"],"created_at":"2024-11-14T04:09:42.353Z","updated_at":"2025-04-09T07:06:21.486Z","avatar_url":"https://github.com/airtai.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"Constrained Monotonic Neural Networks\n================\n\n\u003c!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! --\u003e\n\n## Running in Google Colab\n\nYou can execute this interactive tutorial in Google Colab by clicking\nthe button below:\n\n\u003ca href=\"https://colab.research.google.com/github/airtai/monotonic-nn/blob/main/nbs/index.ipynb\" target=”_blank”\u003e\n\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\" /\u003e\n\u003c/a\u003e\n\n## Summary\n\nThis Python library implements Constrained Monotonic Neural Networks as\ndescribed in:\n\nDavor Runje, Sharath M. Shankaranarayana, “Constrained Monotonic Neural\nNetworks”, in Proceedings of the 40th International Conference on\nMachine Learning, 2023. \\[[PDF](https://arxiv.org/pdf/2205.11775.pdf)\\].\n\n#### Abstract\n\nWider adoption of neural networks in many critical domains such as\nfinance and healthcare is being hindered by the need to explain their\npredictions and to impose additional constraints on them. Monotonicity\nconstraint is one of the most requested properties in real-world\nscenarios and is the focus of this paper. One of the oldest ways to\nconstruct a monotonic fully connected neural network is to constrain\nsigns on its weights. Unfortunately, this construction does not work\nwith popular non-saturated activation functions as it can only\napproximate convex functions. We show this shortcoming can be fixed by\nconstructing two additional activation functions from a typical\nunsaturated monotonic activation function and employing each of them on\nthe part of neurons. Our experiments show this approach of building\nmonotonic neural networks has better accuracy when compared to other\nstate-of-the-art methods, while being the simplest one in the sense of\nhaving the least number of parameters, and not requiring any\nmodifications to the learning procedure or post-learning steps. Finally,\nwe prove it can approximate any continuous monotone function on a\ncompact subset of $\\mathbb{R}^n$.\n\n#### Citation\n\nIf you use this library, please cite:\n\n``` title=\"bibtex\"\n@inproceedings{runje2023,\n  title={Constrained Monotonic Neural Networks},\n  author={Davor Runje and Sharath M. Shankaranarayana},\n  booktitle={Proceedings of the 40th {International Conference on Machine Learning}},\n  year={2023}\n}\n```\n\n## Python package\n\nThis package contains an implementation of our Monotonic Dense Layer\n[`MonoDense`](https://monotonic.airt.ai/latest/api/airt/keras/layers/MonoDense/#airt.keras.layers.MonoDense)\n(Constrained Monotonic Fully Connected Layer). Below is the figure from\nthe paper for reference.\n\nIn the code, the variable `monotonicity_indicator` corresponds to **t**\nin the figure and parameters `is_convex`, `is_concave` and\n`activation_weights` are used to calculate the activation selector **s**\nas follows:\n\n- if `is_convex` or `is_concave` is **True**, then the activation\n  selector **s** will be (`units`, 0, 0) and (0, `units`, 0),\n  respecively.\n\n- if both `is_convex` or `is_concave` is **False**, then the\n  `activation_weights` represent ratios between $\\breve{s}$, $\\hat{s}$\n  and $\\tilde{s}$, respecively. E.g. if `activation_weights = (2, 2, 1)`\n  and `units = 10`, then\n\n$$\n(\\breve{s}, \\hat{s}, \\tilde{s}) = (4, 4, 2)\n$$\n\n![mono-dense-layer-diagram](https://github.com/airtai/monotonic-nn/raw/main/nbs/images/mono-dense-layer-diagram.png)\n\n### Install\n\n``` sh\npip install monotonic-nn\n```\n\n### How to use\n\nIn this example, we’ll assume we have a simple dataset with three inputs\nvalues $x_1$, $x_2$ and $x_3$ sampled from the normal distribution,\nwhile the output value $y$ is calculated according to the following\nformula before adding Gaussian noise to it:\n\n$y = x_1^3 + \\sin\\left(\\frac{x_2}{2 \\pi}\\right) + e^{-x_3}$\n\n\u003ctable id=\"T_37b51\"\u003e\n  \u003cthead\u003e\n    \u003ctr\u003e\n      \u003cth id=\"T_37b51_level0_col0\" class=\"col_heading level0 col0\" \u003ex0\u003c/th\u003e\n      \u003cth id=\"T_37b51_level0_col1\" class=\"col_heading level0 col1\" \u003ex1\u003c/th\u003e\n      \u003cth id=\"T_37b51_level0_col2\" class=\"col_heading level0 col2\" \u003ex2\u003c/th\u003e\n      \u003cth id=\"T_37b51_level0_col3\" class=\"col_heading level0 col3\" \u003ey\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd id=\"T_37b51_row0_col0\" class=\"data row0 col0\" \u003e0.304717\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row0_col1\" class=\"data row0 col1\" \u003e-1.039984\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row0_col2\" class=\"data row0 col2\" \u003e0.750451\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row0_col3\" class=\"data row0 col3\" \u003e0.234541\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd id=\"T_37b51_row1_col0\" class=\"data row1 col0\" \u003e0.940565\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row1_col1\" class=\"data row1 col1\" \u003e-1.951035\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row1_col2\" class=\"data row1 col2\" \u003e-1.302180\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row1_col3\" class=\"data row1 col3\" \u003e4.199094\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd id=\"T_37b51_row2_col0\" class=\"data row2 col0\" \u003e0.127840\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row2_col1\" class=\"data row2 col1\" \u003e-0.316243\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row2_col2\" class=\"data row2 col2\" \u003e-0.016801\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row2_col3\" class=\"data row2 col3\" \u003e0.834086\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd id=\"T_37b51_row3_col0\" class=\"data row3 col0\" \u003e-0.853044\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row3_col1\" class=\"data row3 col1\" \u003e0.879398\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row3_col2\" class=\"data row3 col2\" \u003e0.777792\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row3_col3\" class=\"data row3 col3\" \u003e-0.093359\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd id=\"T_37b51_row4_col0\" class=\"data row4 col0\" \u003e0.066031\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row4_col1\" class=\"data row4 col1\" \u003e1.127241\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row4_col2\" class=\"data row4 col2\" \u003e0.467509\u003c/td\u003e\n      \u003ctd id=\"T_37b51_row4_col3\" class=\"data row4 col3\" \u003e0.780875\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\nNow, we’ll use the\n[`MonoDense`](https://monotonic.airt.ai/latest/api/airt/keras/layers/MonoDense/#airt.keras.layers.MonoDense)\nlayer instead of `Dense` layer to build a simple monotonic network. By\ndefault, the\n[`MonoDense`](https://monotonic.airt.ai/latest/api/airt/keras/layers/MonoDense/#airt.keras.layers.MonoDense)\nlayer assumes the output of the layer is monotonically increasing with\nall inputs. This assumtion is always true for all layers except possibly\nthe first one. For the first layer, we use `monotonicity_indicator` to\nspecify which input parameters are monotonic and to specify are they\nincreasingly or decreasingly monotonic:\n\n- set 1 for increasingly monotonic parameter,\n\n- set -1 for decreasingly monotonic parameter, and\n\n- set 0 otherwise.\n\nIn our case, the `monotonicity_indicator` is `[1, 0, -1]` because $y$\nis:\n\n- monotonically increasing w.r.t. $x_1$\n  $\\left(\\frac{\\partial y}{x_1} = 3 {x_1}^2 \\geq 0\\right)$, and\n\n- monotonically decreasing w.r.t. $x_3$\n  $\\left(\\frac{\\partial y}{x_3} = - e^{-x_2} \\leq 0\\right)$.\n\n``` python\nfrom tensorflow.keras import Sequential\nfrom tensorflow.keras.layers import Dense, Input\n\nfrom airt.keras.layers import MonoDense\n\nmodel = Sequential()\n\nmodel.add(Input(shape=(3,)))\nmonotonicity_indicator = [1, 0, -1]\nmodel.add(\n    MonoDense(128, activation=\"elu\", monotonicity_indicator=monotonicity_indicator)\n)\nmodel.add(MonoDense(128, activation=\"elu\"))\nmodel.add(MonoDense(1))\n\nmodel.summary()\n```\n\n    Model: \"sequential\"\n    _________________________________________________________________\n     Layer (type)                Output Shape              Param #   \n    =================================================================\n     mono_dense (MonoDense)      (None, 128)               512       \n                                                                     \n     mono_dense_1 (MonoDense)    (None, 128)               16512     \n                                                                     \n     mono_dense_2 (MonoDense)    (None, 1)                 129       \n                                                                     \n    =================================================================\n    Total params: 17,153\n    Trainable params: 17,153\n    Non-trainable params: 0\n    _________________________________________________________________\n\nNow we can train the model as usual using `Model.fit`:\n\n``` python\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.optimizers.schedules import ExponentialDecay\n\nlr_schedule = ExponentialDecay(\n    initial_learning_rate=0.01,\n    decay_steps=10_000 // 32,\n    decay_rate=0.9,\n)\noptimizer = Adam(learning_rate=lr_schedule)\nmodel.compile(optimizer=optimizer, loss=\"mse\")\n\nmodel.fit(\n    x=x_train, y=y_train, batch_size=32, validation_data=(x_val, y_val), epochs=10\n)\n```\n\n    Epoch 1/10\n    313/313 [==============================] - 3s 5ms/step - loss: 9.4221 - val_loss: 6.1277\n    Epoch 2/10\n    313/313 [==============================] - 1s 4ms/step - loss: 4.6001 - val_loss: 2.7813\n    Epoch 3/10\n    313/313 [==============================] - 1s 4ms/step - loss: 1.6221 - val_loss: 2.1111\n    Epoch 4/10\n    313/313 [==============================] - 1s 4ms/step - loss: 0.9479 - val_loss: 0.2976\n    Epoch 5/10\n    313/313 [==============================] - 1s 4ms/step - loss: 0.9008 - val_loss: 0.3240\n    Epoch 6/10\n    313/313 [==============================] - 1s 4ms/step - loss: 0.5027 - val_loss: 0.1455\n    Epoch 7/10\n    313/313 [==============================] - 1s 4ms/step - loss: 0.4360 - val_loss: 0.1144\n    Epoch 8/10\n    313/313 [==============================] - 1s 4ms/step - loss: 0.4993 - val_loss: 0.1211\n    Epoch 9/10\n    313/313 [==============================] - 1s 4ms/step - loss: 0.3162 - val_loss: 1.0021\n    Epoch 10/10\n    313/313 [==============================] - 1s 4ms/step - loss: 0.2640 - val_loss: 0.2522\n\n    \u003ckeras.callbacks.History\u003e\n\n## License\n\n\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003e\u003cimg alt=\"Creative Commons Licence\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" /\u003e\u003c/a\u003e\u003cbr /\u003eThis\nwork is licensed under a\n\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"\u003eCreative\nCommons Attribution-NonCommercial-ShareAlike 4.0 International\nLicense\u003c/a\u003e.\n\nYou are free to:\n\n- Share — copy and redistribute the material in any\nmedium or format\n\n- Adapt — remix, transform, and build upon the material\n\nThe licensor cannot revoke these freedoms as long as you follow the\nlicense terms.\n\nUnder the following terms: \n\n- Attribution — You must give appropriate\ncredit, provide a link to the license, and indicate if changes were\nmade. You may do so in any reasonable manner, but not in any way that\nsuggests the licensor endorses you or your use.\n\n- NonCommercial — You may not use the material for commercial purposes.\n\n- ShareAlike — If you remix, transform, or build upon the material, you\n  must distribute your contributions under the same license as the\n  original.\n\n- No additional restrictions — You may not apply legal terms or\n  technological measures that legally restrict others from doing\n  anything the license permits.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fairtai%2Fmonotonic-nn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fairtai%2Fmonotonic-nn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fairtai%2Fmonotonic-nn/lists"}