{"id":19750178,"url":"https://github.com/wepe/efficient-decision-tree-notes","last_synced_at":"2025-07-08T23:06:39.750Z","repository":{"id":87469948,"uuid":"115422941","full_name":"wepe/efficient-decision-tree-notes","owner":"wepe","description":"高效决策树算法系列笔记","archived":false,"fork":false,"pushed_at":"2019-05-23T02:11:57.000Z","size":980,"stargazers_count":230,"open_issues_count":1,"forks_count":47,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-07-08T23:06:36.706Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wepe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-12-26T13:07:03.000Z","updated_at":"2025-06-02T03:38:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"a036f15d-99ac-49db-8627-340480672cd8","html_url":"https://github.com/wepe/efficient-decision-tree-notes","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/wepe/efficient-decision-tree-notes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wepe%2Fefficient-decision-tree-notes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wepe%2Fefficient-decision-tree-notes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wepe%2Fefficient-decision-tree-notes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wepe%2Fefficient-decision-tree-notes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wepe","download_url":"https://codeload.github.com/wepe/efficient-decision-tree-notes/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wepe%2Fefficient-decision-tree-notes/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264363794,"owners_count":23596507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T02:31:12.572Z","updated_at":"2025-07-08T23:06:39.730Z","avatar_url":"https://github.com/wepe.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# efficient-decision-tree-notes\n\n这个笔记记录高效的决策树系列算法，主要阅读论文，结合一些开源框架，希望在弄清算法的基础上，尝试着改进算法，尝试着工程实现．\n\n我们知道，目前比较流行的两个GBDT开源框架是XGBoost和LightGBM，无论内存占用还是计算速度，它们都做到了淋漓尽致．在LightGBM的[Feature](http://lightgbm.readthedocs.io/en/latest/Features.html)上提到了，XGBoost的decision tree用的是pre-sorted based的算法，也就是在tree building之前对各维特征先排序，代表性的算法是SLIQ[1]和SPRINT[2]．而LightGBM的decision tree是histogram based的算法，也就是先将特征离散化，代表性的算法是CLOUDS[3],Mcrank[4]和Machado[5]．\n\nSLIQ和SPRINT算法的特点决定了树生长的方式是level-wise(breadth-first)的，与之对应的是leaf-wise(depth-wise，best-wise[6])的方式，LightGBM正是采用leaf-wise的方式．\n\n内容大致按以下几部分展开:\n\n- 基于特征预排序的算法[SLIQ](https://github.com/wepe/efficient-decision-tree-notes/blob/master/SLIQ.md)\n\n- 基于特征预排序的算法[SPRINT](https://github.com/wepe/efficient-decision-tree-notes/blob/master/SPRINT.md)\n\n- 基于特征离散化的算法[CLOUDS](https://github.com/wepe/efficient-decision-tree-notes/blob/master/ClOUDS.md)\n\n- 研究开源框架: [LightGBM，XGBoost](https://github.com/wepe/efficient-decision-tree-notes/blob/master/LightGBM%20vs%20XGBoost.md)\n\n- 自己动手实现一个高效决策树 [tgboost](https://github.com/wepe/tgboost)\n\n\n### 参考文献\n\n\n[1] Mehta, Manish, Rakesh Agrawal, and Jorma Rissanen. “SLIQ: A fast scalable classifier for data mining.” International Conference on Extending Database Technology. Springer Berlin Heidelberg, 1996.\n\n[2] Shafer, John, Rakesh Agrawal, and Manish Mehta. “SPRINT: A scalable parallel classifier for data mining.” Proc. 1996 Int. Conf. Very Large Data Bases. 1996.\n\n[3] Ranka, Sanjay, and V. Singh. “CLOUDS: A decision tree classifier for large datasets.” Proceedings of the 4th Knowledge Discovery and Data Mining Conference. 1998.\n\n[4] Machado, F. P. “Communication and memory efficient parallel decision tree construction.” (2003).\n\n[5] Li, Ping, Qiang Wu, and Christopher J. Burges. “Mcrank: Learning to rank using multiple classification and gradient boosting.” Advances in neural information processing systems. 2007.\n\n[6] Shi, Haijian. “Best-first decision tree learning.” Diss. The University of Waikato, 2007.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwepe%2Fefficient-decision-tree-notes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwepe%2Fefficient-decision-tree-notes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwepe%2Fefficient-decision-tree-notes/lists"}