{"id":26508504,"url":"https://github.com/corels/corels","last_synced_at":"2025-03-21T00:32:57.010Z","repository":{"id":40537997,"uuid":"88920171","full_name":"corels/corels","owner":"corels","description":"Learning Certifiably Optimal Rule Lists","archived":false,"fork":false,"pushed_at":"2021-10-16T16:53:49.000Z","size":17175,"stargazers_count":170,"open_issues_count":7,"forks_count":20,"subscribers_count":16,"default_branch":"master","last_synced_at":"2024-05-18T17:54:46.529Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://corels.eecs.harvard.edu/","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/corels.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-20T23:43:51.000Z","updated_at":"2024-03-06T07:03:15.000Z","dependencies_parsed_at":"2022-07-30T23:17:55.163Z","dependency_job_id":null,"html_url":"https://github.com/corels/corels","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/corels%2Fcorels","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/corels%2Fcorels/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/corels%2Fcorels/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/corels%2Fcorels/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/corels","download_url":"https://codeload.github.com/corels/corels/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244717391,"owners_count":20498283,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-21T00:32:56.275Z","updated_at":"2025-03-21T00:32:56.996Z","avatar_url":"https://github.com/corels.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CORELS\nCertifiably Optimal RulE ListS\n\n**CORELS is a custom discrete optimization\ntechnique for building rule lists over a categorical feature space.**\nOur algorithm provides the optimal solution, with a certificate of optimality.\nBy leveraging algorithmic bounds, efficient data structures,\nand computational reuse, we achieve several orders of magnitude speedup in time\nand a massive reduction of memory consumption.\nOur approach produces optimal rule lists on practical\nproblems in seconds.\nThis framework is a novel alternative to CART and other decision tree methods.\n\n* Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Seltzer, and Cynthia Rudin.\n**Learning Certifiably Optimal Rule Lists for Categorical Data**.\n[JMLR](http://www.jmlr.org/papers/volume18/17-716/17-716.pdf), 2018.\n\n* Nicholas Larus-Stone, Elaine Angelino, Daniel Alabi, Margo Seltzer, Vassilios Kaxiras, Aditya Saligrama, Cynthia Rudin\n**Systems Optimizations for Learning Certifiably Optimal Rule Lists**\n[SysML](https://saligrama.io/files/sysml.pdf), 2018\n\n* Nicholas Larus-Stone. **Learning Certifiably Optimal Rule Lists: A Case For Discrete Optimization in the 21st Century**.\n[Senior thesis](https://dash.harvard.edu/handle/1/38811502), 2017.\n\n* Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Seltzer, Cynthia Rudin\n**Learning certifiably optimal rule lists for categorical data**\n[KDD](https://www.kdd.org/kdd2017/papers/view/learning-certifiably-optimal-rule-lists-for-categorical-data), 2017\n\nCORELS is a custom branch-and-bound algorithm for optimizing rule lists.\n\n**Web UI can be found at:** https://corels.eecs.harvard.edu/\n\n**R Package can be found at:** https://cran.r-project.org/package=corels\n\n**Python package can be found at:** https://pypi.org/project/corels/\n\n## Overview\n\n* [C/C++ dependencies](#cc-dependencies)\n* [Sample command](#sample-command)\n* [Usage](#usage)\n* [Data format](#data-format)\n* [Arguments](#arguments)\n* [Example dataset](#example-dataset)\n* [Example rule list](#example-rule-list)\n* [Optimization algorithm and objective](#optimization-algorithm-and-objective)\n* [Data structures](#data-structures)\n* [Related work](#related-work)\n\n### C/C++ dependencies\n\n* [gmp](https://gmplib.org/) (GNU Multiple Precision Arithmetic Library)\n* [mpfr](http://www.mpfr.org/) (GNU MPFR Library for multiple-precision floating-point computations; depends on gmp)\n* [libmpc](http://www.multiprecision.org/) (GNU MPC for arbitrarily high precision and correct rounding; depends on gmp and mpfr)\n\ne.g., [install libmpc](http://brewformulas.org/Libmpc) via [homewbrew](https://brew.sh/) (this will also install gmp and mpfr):\n\n    brew install libmpc\n\n### Sample command\n\nRun the following from the `src/` directory.\n\n    make\n    ./corels -r 0.015 -c 2 -p 1 ../data/compas_train.out ../data/compas_train.label ../data/compas_train.minor\n\n### Usage\n\n    ./corels [-b] [-n max_num_nodes] [-r regularization] [-v verbosity] -c (1|2|3|4) -p (0|1|2)\n             [-f logging_frequency] -a (1|2|3) [-s] [-L latex_out] data.out data.label [data.minor]\n\n### Data format\n\nFor examples, see `compas.out` and `compas.label` in `data/`.  Also see `compas.minor` (optional). Note that our data parsing is fragile and errors will occur if the format is not followed exactly.\n\n* The input data files must be space-delimited text.\n* Each line contains `N + 1` fields, where `N` is the number of observations, and ends with `\\n` (including the last line).\n* In each line, the last `N` fields are `0`'s and `1`'s, and encode a bit vector;\nthe first field has the format `{text-description}`, where the text between the brackets provides a description of the bit vector.\n* There can be no spaces in the text descriptions--words should be separated by dashes or underscores.\n\n### Arguments\n\n**[-b]** Breadth-first search (BFS). You must specify a search policy;\nuse exactly one of `(-b | -c)`.\n\n**[-n max_num_nodes]** Maximum trie (cache) size.\nStop execution if the number of nodes in the trie exceeds this number.\nDefault value corresponds to `-n 100000`.\n\n**[-r regularization]** Regularization parameter (optional).\nThe default value corresponds to `-r 0.01` and can be thought of as adding a\npenalty equivalent to misclassifying 1% of data when increasing a rule list's\nlength by one association rule.\n\n**[-v verbosity]** Verbosity.\nDefault value corresponds to `-v 0`.\nIf verbosity is at least `10`, then print machine info.\nIf verbosity is at least `1000`, then also print input antecedents and labels.\n\n**-c (1|2|3|4)** Best-first search policy.\nYou must specify a search policy; use exactly one of `(-b | -c)`.\nWe include four different prioritization schemes:\n\n* Use `-c 1` to prioritize by curiosity (see *Section 5.1 Custom scheduling policies* of our paper).\n* Use `-c 2` to prioritize by the lower bound.\n* Use `-c 3` to prioritize by the objective.\n* Use `-c 4` for depth-first search.\n\n**-p (0|1|2)** Symmetry-aware map (optional).\n\n* Use `-p 0` for no symmetry-aware map (default).\n* Use `-p 1` for the permutation map.\n* Use `-p 2` for the captured vector map.\n\n**[-f logging_frequency]** Logging frequency.  Default value corresponds to `-f 1000`.\n\n**-a (0|1|2)** Exclude a specific algorithm optimization.\nDefault value corresponds to `-a 0`.\n\n* Use `-a 0` to include the following optimizations.\n* Use `-a 1` to exclude the minimum support bounds (see *Section 3.7 Lower bounds on antecedent support* of our paper).\n* Use `-a 2` to exclude the lookahead bound (see *Lemma 2 in Section 3.4 Hierarchical objective lower bound*).\n\n**[-s]** Calculate upper bound on remaining search space size (optional).\nThis adds a small overhead; the default behavior does not perform the calculation.\nWith `-s`, we dynamically and incrementally calculate `floor(log10(Gamma(Rc, Q)))`,\nwhere `Gamma(Rc, Q)` is the upper bound (see *Theorem 7 in Section 3.6* of our paper).\n\n**[-L latex_out]** Latex output.  Include this flag to generate a latex representation of the output rule list.\n\n**data.out** File path to training data.  See **Data format**, above.\n\n* Encode `M` antecedents as `M` space-delimited lines of text. Each line contains `N + 1` fields.\n* The first field has the format `{antecedent-description}`,\nwhere the text between the brackets describes the antecedent, e.g., `{hair-color:brown}`, or `{age:20-25}`.\n* The remaining `N` fields indicate whether the antecedent is true or false for the `N` observations.\n\n**data.label** File path to training labels.  See **Data format**, above.\n\n* Encode labels as two space-delimited lines of text.\n*The first line starts with a description of the negative class label, e.g., `{label=0}`;\nthe remaining `N` fields indicate which of the `N` observations have this label.\n* The second line contains analogous (redundant) information for the positive class label.\n\n**data.minor** File path to a bit vector to support use of the equivalent points bound (optional, see *Theorem 20 in Section 3.14* of our paper).\n\n### Example dataset\n\nThe files in `data/` were generated from [ProPublica's COMPAS recidivism dataset](https://github.com/propublica/compas-analysis),\nspecifically, the file `compas-scores-two-years.csv`.\n\nWe include one training set (`N = 6489`) and one test set (`N = 721`) from a 10-fold cross-validation experiment.\nThere are four training data files:\n\n* `compas_train.csv` : `7` categorical and integer-valued features and binary class labels extracted from `compas-scores-two-years.csv`\n\n    sex (Male, Female), age, juvenile-felonies, juvenile-misdemeanors, juvenile-crimes,\n    priors, current-charge-degree (Misdemeanor, Felony)\n\n* `compas_train-binary.csv` : `14` equivalent binary features and class labels (included for convenience / use with other algorithms)\n\n    sex:Male, age:18-20, age:21-22, age:23-25, age:26-45, age:\u003e45\n    juvenile-felonies:\u003e0, juvenile-misdemeanors:\u003e0, juvenile-crimes:\u003e0\n    priors:2-3, priors:=0, priors:=1, priors:\u003e3, current-charge-degree:Misdemeanor\n\n* `compas_train.out` : `155` mined antecedents (binary features and length-2 conjunctions of binary features with support in `[0.005, 0.995]`), e.g.,\n\n    {sex:Female}, {age:18-20}, {sex:Male,current-charge-degree:Misdemeanor}, {age:26-45,juvenile-felonies:\u003e0}\n\n* `compas_train.labels` : class labels\n\n    {recidivate-within-two-years:No}, {recidivate-within-two-years:Yes}\n\n* `compas_train.minor` : bit vector used to support the equivalent points bound\n\nThe corresponding test data files are: `compas_test.csv`, `compas_test-binary.csv`, `compas_test.out`, and `compas_test.labels`.\n\n### Example rule list\n\n    if (age=23−25) and (priors=2−3) then predict yes\n    else if (age = 18 − 20) then predict yes\n    else if (sex = male) and (age = 21 − 22) then predict yes\n    else if (priors \u003e 3) then predict yes\n    else predict no\n\nThis **rule list** predicts two-year recidivism for the [ProPublica two-year recidivism dataset](https://github.com/propublica/compas-analysis),\nand was found by CORELS.\nIts **prefix** corresponds to the first four rules and its **default rule** corresponds to the last rule.\n\n### Optimization algorithm and objective\n\nCORELS is a custom branch-and-bound algorithm that minimizes the following objective\ndefined for a rule list `d`, training data `x` and corresponding labels `y`:\n\n    R(d, x, y) = misc(d, x, y) + c * length(d).\n\nThis objective is a regularized empirical risk that consists of a loss\nand a regularization term that penalizes longer rule lists.\n\n* The loss `misc(d, x, y)` measures `d`'s misclassification error.\n* The regularization parameter `c \u003e= 0` is a small constant.\n* `length(d)` is the number of rules in `d`'s prefix.\n\nLet `p` be `d`'s prefix.  The following objective lower bound drives\nour branch-and-bound procedure:\n\n    b(p, x, y) = misc(p, x, y) + c * length(d) \u003c= R(d, x, y)\n\nwhere `misc(p, x, y)` is the prefix misclassification error\n(due to mistakes made by the prefix, but not the default rule).\n\n### Data structures\n\n* A **trie** (prefix tree) functions as a cache and supports incremental computation.\n* A **priority queue** supports multiple best-first search policies, as well as both breadth-first and depth-first search.\n* A **map** supports symmetry-aware pruning.\n\n### Related work\n\nCORELS builds directly on:\n\n* Hongyu Yang, Cynthia Rudin, and Margo Seltzer.\n**Scalable Bayesian Rule Lists**. [arXiv:1602.08610](https://arxiv.org/abs/1602.08610), 2016. [code](https://github.com/Hongyuy/sbrlmod)\n\n* Benjamin Letham, Cynthia Rudin, Tyler McCormick and David Madigan.\n**Interpretable Classifiers Using Rules and Bayesian Analysis: Building a Better Stroke Prediction Model**.\n*The Annals of Applied Statistics*, 2015, Vol. 9, No. 3, 1350–1371. [pdf](https://users.cs.duke.edu/~cynthia/docs/LethamRuMcMa15.pdf) [code](https://users.cs.duke.edu/~cynthia/code/BRL_supplement_code.zip)\n\nIn particular, CORELS uses a library by Yang et al. for efficiently representing and operating on bit vectors.\nSee the files [src/rule.h](src/rule.h) and [src/rule.c](src/rule.c).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcorels%2Fcorels","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcorels%2Fcorels","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcorels%2Fcorels/lists"}