{"id":15176698,"url":"https://github.com/wearepal/ethicml","last_synced_at":"2025-08-21T08:08:37.603Z","repository":{"id":37090940,"uuid":"170703124","full_name":"wearepal/EthicML","owner":"wearepal","description":"Package for evaluating the performance of methods which aim to increase fairness, accountability and/or transparency","archived":false,"fork":false,"pushed_at":"2024-12-07T17:08:41.000Z","size":92025,"stargazers_count":23,"open_issues_count":43,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-28T14:08:57.274Z","etag":null,"topics":["algorithmic-fairness","computer-vision","data-science","ethical-artificial-intelligence","ethical-data-science","fairness","fairness-ai","fairness-assessment","fairness-awareness-model","fairness-comparison","fairness-ml","machine-bias","machine-learning","pytorch","responsible-ai","toolkit"],"latest_commit_sha":null,"homepage":"https://wearepal.ai/EthicML/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wearepal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-02-14T14:21:29.000Z","updated_at":"2024-12-05T21:58:51.000Z","dependencies_parsed_at":"2023-09-25T22:02:59.983Z","dependency_job_id":"a1ea70f0-55c3-4e2f-a437-d74abb905648","html_url":"https://github.com/wearepal/EthicML","commit_stats":{"total_commits":1764,"total_committers":13,"mean_commits":"135.69230769230768","dds":0.7772108843537415,"last_synced_commit":"054d91dbfe528cbb10ad2297473f5c046dbcf44a"},"previous_names":[],"tags_count":49,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wearepal%2FEthicML","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wearepal%2FEthicML/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wearepal%2FEthicML/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wearepal%2FEthicML/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wearepal","download_url":"https://codeload.github.com/wearepal/EthicML/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251326839,"owners_count":21571636,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithmic-fairness","computer-vision","data-science","ethical-artificial-intelligence","ethical-data-science","fairness","fairness-ai","fairness-assessment","fairness-awareness-model","fairness-comparison","fairness-ml","machine-bias","machine-learning","pytorch","responsible-ai","toolkit"],"created_at":"2024-09-27T13:40:26.448Z","updated_at":"2025-04-28T14:09:21.247Z","avatar_url":"https://github.com/wearepal.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# EthicML: A featureful framework for developing fair algorithms\n\n[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)\n![](https://github.com/predictive-analytics-lab/EthicML/workflows/EthicML%20CI/badge.svg)\n\n\n\nEthicML is a library for performing and assessing __algorithmic fairness__.\nUnlike other libraries, EthicML isn't an education tool, but rather a __researcher__'s toolkit.\n\nOther algorthimic fairness packages are useful, but given that we primarily do research,\na lot of the work we do doesn't fit into some nice box.\nFor example, we might want to use a 'fair' pre-processing method on the data before training a classifier on it.\nWe may still be experimenting and only want part of the framework to execute,\nor we may want to do hyper-parameter optimization.\nWhilst other frameworks can be modified to do these tasks,\nyou end up with hacked-together approaches that don't lend themselves to be built on in the future.\nBecause of this, we built EthicML, a fairness toolkit for researchers.\n\nFeatures include:\n- Support for multiple sensitive attributes\n- Vision datasets\n- Codebase typed with mypy\n- Tested code\n- Reproducible results\n\n### Why not use XXX?\n\nThere are an increasing number of other options,\nIBM's fair-360, Aequitas, EthicalML/XAI, Fairness-Comparison and others.\nThey're all great at what they do, they're just not right for us.\nWe will however be influenced by them.\nWhere appropriate, we even subsume some of these libraries.\n\n## Installation\n\nEthicML requires Python \u003e= 3.8.\nTo install EthicML, just do\n```\npip3 install ethicml\n```\n\nIf you want to use the method by Agarwal et al., you have to explicitly install _all_ dependencies:\n```\npip3 install 'ethicml[all]'\n```\n(The quotes are needed in `zsh` and will also work in `bash`.)\n\n**Attention**: In order to use all features of EthicML, PyTorch needs to be installed separately.\nWe are not including PyTorch as a requirement of EthicML,\nbecause there are many different versions for different systems.\n\n## Documentation\n\nThe documentation can be found here: https://wearepal.ai/EthicML/\n\n## Design Principles\n\n```mermaid\nflowchart LR\n    A(Datasets) -- load --\u003e B(Data tuples);\n    B --\u003e C[evaluate_models];\n    G(Algorithms) --\u003e C;\n    C --\u003e D(Metrics);\n```\n\nKeep things simple.\n\n### The Triplet\n\nGiven that we're considering fairness, the base of the toolbox is the triplet {x, s, y}\n\n- X - Features\n- S - Sensitive Label\n- Y - Class Label\n\n__Developer note__: All methods must assume S and Y are multi-class.\n\nWe use a DataTuple class to contain the triplet\n\n```python\ntriplet = DataTuple(x: pandas.DataFrame, s: pandas.DataFrame, y: pandas.DataFrame)\n```\n\nIn addition, we have a variation: the TestTuple which contains the pair\n```python\npair = TestTuple(x: pandas.DataFrame, s: pandas.DataFrame)\n```\nThis is to reduce the risk of a user accidentally evaluating performance on their training set.\n\nUsing dataframes may be a little inefficient,\nbut given the amount of splicing on conditions that we're doing, it feels worth it.\n\n### Separation of Methods\n\nWe purposefully keep pre, during and post algorithm methods separate. This is because they have different return types.\n\n```python\npre_algorithm.run(train: DataTuple, test: TestTuple)  # -\u003e Tuple[DataTuple, TestTuple]\nin_algorithm.run(train: DataTuple, test: TestTuple)  # -\u003e Prediction\npost_algorithm.run(train_prediction: Prediction, train: DataTuple, test_prediction: Prediction, test: TestTuple)  # -\u003e Prediction\n```\nwhere `Prediction` holds a pandas.Series of the class label.\nIn the case of a \"soft\" output, `SoftPrediction` extends `Prediction` and provides a mapping from\n\"soft\" to \"hard\" labels.\nSee the documentation for more details.\n\n### General Rules of Thumb\n\n- Mutable data structures are bad.\n- At the very least, functions should be Typed.\n- Readability \u003e Efficiency.\n- Warnings must be addressed.\n- Always write tests first.\n\n## Future Plans\n\nThe aim is to make EthicML operate on 2 levels.\n\n1. We want a high-level API so that a user can define a new model or metric, then get publication-ready\nresults in just a couple of lines of code.\n2. We understand that truly ground-breaking work sometimes involves tearing up the rulebook.\nTherefore, we want to also expose a lower-level API so that a user can make use of as much, or little of the library\nas is suitable for them.\n\nWe've built everything with this philosophy in mind, but acknowledge that we still have a way to go.\n\n# Contributing\n\nIf you're interest in this research area, we'd love to have you aboard.\nFor more details check out [CONTRIBUTING.md](./CONTRIBUTING.md).\nWhether your skills are in coding-up papers you've read, writing tutorials, or designing a logo, please reach out.\n\n## Development\nInstall development dependencies with `pip install -e .[dev]`\n\nTo use the pre-commit hooks run `pre-commit install`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwearepal%2Fethicml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwearepal%2Fethicml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwearepal%2Fethicml/lists"}