{"id":28690906,"url":"https://github.com/weltxing/liblinear-sc-reading","last_synced_at":"2025-06-14T06:08:00.756Z","repository":{"id":112059888,"uuid":"452325786","full_name":"WeltXing/liblinear-sc-reading","owner":"WeltXing","description":"LIBLINEAR理论与源码解读（已完结）","archived":false,"fork":false,"pushed_at":"2022-02-05T13:31:53.000Z","size":2137,"stargazers_count":22,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-23T12:43:49.322Z","etag":null,"topics":["cpp","liblinear","linear-models","machine-learning","optimization","source-code","svm"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WeltXing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-01-26T15:18:37.000Z","updated_at":"2024-10-28T14:41:00.000Z","dependencies_parsed_at":null,"dependency_job_id":"f3cdcca9-3add-4f99-997a-a3291ae62433","html_url":"https://github.com/WeltXing/liblinear-sc-reading","commit_stats":null,"previous_names":["weltxing/liblinear-sc-reading"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/WeltXing/liblinear-sc-reading","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeltXing%2Fliblinear-sc-reading","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeltXing%2Fliblinear-sc-reading/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeltXing%2Fliblinear-sc-reading/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeltXing%2Fliblinear-sc-reading/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WeltXing","download_url":"https://codeload.github.com/WeltXing/liblinear-sc-reading/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WeltXing%2Fliblinear-sc-reading/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259768621,"owners_count":22908232,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","liblinear","linear-models","machine-learning","optimization","source-code","svm"],"created_at":"2025-06-14T06:07:59.878Z","updated_at":"2025-06-14T06:08:00.738Z","avatar_url":"https://github.com/WeltXing.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# LIBLINEAR理论解读\n\n[LIBLINEAR](https://www.csie.ntu.edu.tw/~cjlin/liblinear/)是用于大型线性分类与回归的流行软件包，在支持多种算法的同时也支持多平台多功能的机器学习。相比于LIBSVM，LIBLINEAR丢弃了SVM中的核方法，只支持线性分类，但大大增加了模型对大数据的适应能力。笔者在阅读完LIBSVM源码后（\u003chttps://github.com/Kaslanarian/libsvm-sc-reading\u003e）便着手于LIBLINEAR源码的阅读，随后发现LIBLINEAR涉及到的优化知识要比LIBSVM多（LIBSVM只有拉格朗日对偶法和SMO算法）。因此将工作重心先放在了理论基础上，然后再研究代码(主要是C++)。\n\n## 理论部分\n\n我们在`theory.pdf`中记录LIBLINEAR中设计到的知识基础和优化方法:\n\n\u003cimg src=\"src/1.png\" alt=\"1\" style=\"zoom: 80%;\" /\u003e\n\n\u003cimg src=\"src/2.png\" alt=\"1\" style=\"zoom: 80%;\" /\u003e\n\nLIBLINEAR总共可以求解12种问题，除了Crammer-Singer的多分类模型和One-class SVM对偶问题之外，剩余的10种优化问题可总结为“带L1/L2正则化的某种损失函数的分类/回归模型的原/对偶问题”，具体如表1所示\n\n| 基础模型 | L1正则化的原问题 | L2正则化的原问题 | L2正则化的对偶问题 |\n| :------: | :--------------: | :--------------: | :----------------: |\n|  L1-SVC  |                  |                  |         √          |\n|  L2-SVC  |        √         |        √         |         √          |\n|    LR    |        √         |        √         |         √          |\n|  L1-SVR  |                  |                  |         √          |\n|  L2-SVR  |                  |        √         |         √          |\n\n\u003ccenter\u003e表1: LIBLINEAR支持的问题类型\n\n基础模型前三种为分类模型，其中第三个模型为对率回归(逻辑回归, Logistic Regression, LR)，后两个模型为回归模型。模型中的\"L1\"和\"L2\"表示损失函数类型而不是正则化，具体可见[`theory.pdf`](./theory.pdf)。\n\n以下为目前笔者的文献阅读和研究进度，表格中为对应问题的优化方法(==已完成==):\n\n| 基础模型 | L1正则化的原问题 | L2正则化的原问题 | L2正则化的对偶问题 |\n| :------: | :--------------: | :--------------: | :----------------: |\n|  L1-SVC  |                  |                  |         CD         |\n|  L2-SVC  |      CD+LS       |      CD+LS       |         CD         |\n|    LR    |      CD+LS       |     TRON+CG      |         CD         |\n|  L1-SVR  |                  |                  |         CD         |\n|  L2-SVR  |                  |     TRON+CG      |         CD         |\n\n\u003e - CD: Coordinate descent, 即坐标下降算法；\n\u003e - TRON: Trust region newton method, 带置信域的牛顿法；\n\u003e - CG: Conjugate gradient, 共轭梯度法；\n\u003e - LS: Line search, 线搜索算法.\n\n下面是剩余两种算法的研究进度(==已完成==):\n\n- [x] Crammer-Singer多分类：坐标下降法；\n- [x] One-class SVM对偶问题：外贪心内循环的二层坐标下降框架.\n\n## 获取\n\n通过\n\n```bash\ngit clone https://github.com/Kaslanarian/liblinear-sc-reading\n```\n\n获取文件。\n\n## 文献清单\n\n下面列出笔者阅读过的主要文献，LIBLINEAR中的各种优化算法**全部**出于下面的论文。如果觉得本文有表述不清的地方可以查阅原文。\n\n1. [LIBLINEAR: A library for large linear classification](https://www.jmlr.org/papers/volume9/fan08a/fan08a.pdf)：LIBLINEAR的综述论文;\n2. [A dual coordinate descent method for large-scale linear SVM](https://dl.acm.org/doi/abs/10.1145/1390156.1390208)：坐标下降法求解SVM分类对偶问题；\n3. [Trust region Newton method for large-scale logistic regression](https://www.jmlr.org/papers/volume9/lin08b/lin08b.pdf)：置信域牛顿法求解对率回归原问题；\n4. [Large-scale linear support vector regression](https://www.jmlr.org/papers/volume13/ho12a/ho12a.pdf)：求解支持向量回归原问题和对偶问题的算法；\n5. [Coordinate descent method for large-scale l2-loss linear support vector machines](https://www.jmlr.org/papers/volume9/chang08a/chang08a.pdf)：坐标下降求解L2-loss SVM原问题；\n6. [Dual coordinate descent methods for logistic regression and maximum entropy models](https://link.springer.com/article/10.1007/s10994-010-5221-8)：坐标下降法求解对率回归(逻辑回归)的对偶问题；\n7. [A comparison of optimization methods and software for large-scale l1-regularized linear classification](https://www.jmlr.org/papers/volume11/yuan10c/yuan10c.pdf)：L1正则化下的对率回归与L2-loss SVM的求解（这篇文章也是对L1正则的优化算法综述）；\n8. [Dual coordinate-descent methods for linear one-class SVM and SVDD](https://epubs.siam.org/doi/abs/10.1137/1.9781611976236.21)：二层坐标下降法框架求解单类SVM的对偶问题（同时也提出了支持向量数据描述(SVDD)的训练方法）；\n9. [A sequential dual method for large scale multi-class li](https://dl.acm.org/doi/abs/10.1145/1401890.1401942)：坐标下降法求解多分类SVM.\n\n你可以在\u003chttps://welts.xyz\u003e中找到笔者对这些文献的解读。我们这里省略了一些辅助文献，详情可查看pdf中的参考文献。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweltxing%2Fliblinear-sc-reading","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fweltxing%2Fliblinear-sc-reading","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweltxing%2Fliblinear-sc-reading/lists"}