{"id":24300574,"url":"https://github.com/okitouni/mode","last_synced_at":"2026-04-28T16:36:32.782Z","repository":{"id":57442656,"uuid":"288483831","full_name":"okitouni/MoDe","owner":"okitouni","description":"Moment Decorrelation for constrained NN outputs with respect to protected attributes. (Decorrelated (flat), linear, quadratic, etc.)","archived":false,"fork":false,"pushed_at":"2020-11-10T17:06:45.000Z","size":4759,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-03T09:36:54.780Z","etag":null,"topics":["decorrelation","fairness-ml","hep-ex","hep-ph","pytorch","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/okitouni.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-18T14:48:13.000Z","updated_at":"2022-05-25T14:40:29.000Z","dependencies_parsed_at":"2022-09-26T17:21:17.692Z","dependency_job_id":null,"html_url":"https://github.com/okitouni/MoDe","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okitouni%2FMoDe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okitouni%2FMoDe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okitouni%2FMoDe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/okitouni%2FMoDe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/okitouni","download_url":"https://codeload.github.com/okitouni/MoDe/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242213757,"owners_count":20090700,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["decorrelation","fairness-ml","hep-ex","hep-ph","pytorch","tensorflow"],"created_at":"2025-01-16T23:14:43.018Z","updated_at":"2025-12-05T16:04:28.075Z","avatar_url":"https://github.com/okitouni.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MoDe\nMoment Decorrelation (MoDe) is a tool that can enforce decorrelation between some nuisance parameter (or protected attribute in ML fairness lingo) and the response of some model with gradient-based optimization (a neural network for example.) It can force trained models to have the same response across different values of the protected attribute but it can also go beyond simple decorrelation. For example, MoDe can constrain the response function to be linear or quadratic in the protected attribute. This can increase performance tremendously if the independence constraint is not necessary but instead one only cares about the \"smoothness\" of the dependence of the response on the protected attribute (which is a weaker constraint.)\n\nFor more details please see our article at [2010.09745](https://arxiv.org/abs/2010.09745). (Please use [INSPIRE-HEP](https://inspirehep.net/literature?sort=mostrecent\u0026size=25\u0026page=1\u0026q=find%20eprint%202010.09745) for citations.)\n\nAn implementation is available in each of TensorFlow and PyTorch. The notebooks in [examples/](examples/) illustrate how MoDe is used and show how one can obtain different response functions on a toy example and on a W-tagging dataset.\n\n\n## Installation \n```\npip install modeloss\n```\nThe PyTorch implementation requires PyTorch 1.6.0 or newer. \nThe TensorFlow implementation requires TensorFlow 2.2.0 or newer. \n\n## Usage \nFor PyTorch:\n```\nfrom modeloss.pytorch import MoDeLoss\n\nflatness_loss = MoDeLoss(order=0)\nloss = lambda pred,target,m,weights: lambd * flatness_loss(pred,target,m,weights)+\\\n                                            classification_loss(pred,target,weights)\n```\n\nFor TensorFlow, replace `modeloss.pytorch` above with `modeloss.tf`.\n## Example \nThis is a toy example (located in [examples/ToyExampleTF.ipynb](examples/ToyExampleTF.ipynb)) in which the signal (samples with label 1) is localized near the value 0.2 of the protected attribute *m*. While backgrounds (samples with label 0) are uniform in this feature, we note that this bias is introduced into any naive classifer. Indeed, we see that an unconstrained classifier (in the sense that it has no additional fairness regularization) has a large false positive rate for backgrounds near *m* = 0.2. Here we show how different MoDe regularizations (MoDe\\[n\\]) mitigate this bias by flattening the false positive rate as a function of the protected attribute *m* into its n'th legendre decomposition (where n is the highest moment allowed; 0 is flat, 1 is linear, etc) . \n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"100%\" src=\"https://raw.githubusercontent.com/okitouni/MoDe/master/docs/source/_plots/roc.svg?sanitize=true\" /\u003e\n\u003c/p\u003e\n\n\n\n**Left**: The false positive rate versus mass (*m*) for various models at signal efficiencies (or true positive rates (TPR)) ε = 80, 50, 20% (each set of 3 identically colored and stylized lines correspond to the same model but with selection thresholds chosen to achieve the 3 desired TPRs). The bottom panel\nshows that MoDe\\[1\\] and MoDe\\[2\\] completely overlap with the m-agnostic model for this simple\nexample, which is expected because the optimal classifier here has linear dependence on mass (see paper). **Right**: ROC curves for MoDe\\[0\\], MoDe\\[1\\], and MoDe\\[2\\] compared to the *m*-agnostic\nmodel and a model with unconstrained mass dependence. As in the left panel, we see that `MoDe[1]`,\nMoDe\\[2\\], and the *m*-agnostic ROC curves are nearly identical because the optimal classifier has\nlinear mass dependence in this simple example. \nFor more details see [2010.09745](https://arxiv.org/pdf/2010.09745.pdf).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fokitouni%2Fmode","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fokitouni%2Fmode","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fokitouni%2Fmode/lists"}