{"id":13603367,"url":"https://github.com/ZhiningLiu1998/mesa","last_synced_at":"2025-04-11T19:31:54.967Z","repository":{"id":37654368,"uuid":"269029186","full_name":"ZhiningLiu1998/mesa","owner":"ZhiningLiu1998","description":"[NeurIPS’20] ⚖️ Build powerful ensemble class-imbalanced learning models via meta-knowledge-powered resampler. | 设计元知识驱动的采样器解决类别不平衡问题","archived":false,"fork":false,"pushed_at":"2024-06-17T23:00:16.000Z","size":1703,"stargazers_count":105,"open_issues_count":4,"forks_count":25,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-10-29T11:58:22.848Z","etag":null,"topics":["class-imbalance","ensemble","ensemble-machine-learning","ensemble-model","imbalance-classification","imbalanced-data","imbalanced-learn","imbalanced-learning","mesa","meta-learning-algorithms","meta-sampler","meta-training"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2010.08830","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ZhiningLiu1998.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-06-03T08:13:44.000Z","updated_at":"2024-10-17T08:14:48.000Z","dependencies_parsed_at":"2024-11-07T07:44:28.979Z","dependency_job_id":null,"html_url":"https://github.com/ZhiningLiu1998/mesa","commit_stats":{"total_commits":15,"total_committers":3,"mean_commits":5.0,"dds":"0.33333333333333337","last_synced_commit":"5447ce92a21a6a9f0f21cb559f3196a195affbcc"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZhiningLiu1998%2Fmesa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZhiningLiu1998%2Fmesa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZhiningLiu1998%2Fmesa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZhiningLiu1998%2Fmesa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ZhiningLiu1998","download_url":"https://codeload.github.com/ZhiningLiu1998/mesa/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248466967,"owners_count":21108571,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["class-imbalance","ensemble","ensemble-machine-learning","ensemble-model","imbalance-classification","imbalanced-data","imbalanced-learn","imbalanced-learning","mesa","meta-learning-algorithms","meta-sampler","meta-training"],"created_at":"2024-08-01T19:00:20.144Z","updated_at":"2025-04-11T19:31:54.959Z","avatar_url":"https://github.com/ZhiningLiu1998.png","language":"Jupyter Notebook","funding_links":[],"categories":["Uncategorized","Jupyter Notebook","其他_机器学习与深度学习"],"sub_categories":["Uncategorized"],"readme":"\u003ch1 align=\"center\"\u003e MESA: Meta-sampler for imbalanced learning \u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003c!-- \u003cimg src=\"https://img.shields.io/badge/ZhiningLiu1998-MESA-orange\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/stars/ZhiningLiu1998/mesa\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/forks/ZhiningLiu1998/mesa\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/issues/ZhiningLiu1998/mesa\"\u003e  \n  \u003cimg src=\"https://img.shields.io/github/license/ZhiningLiu1998/mesa\"\u003e --\u003e\n  \u003ca href=\"https://github.com/ZhiningLiu1998/mesa\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/ZhiningLiu1998-MESA-orange\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/ZhiningLiu1998/mesa/stargazers\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/stars/ZhiningLiu1998/mesa\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/ZhiningLiu1998/mesa/network/members\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/forks/ZhiningLiu1998/mesa\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/ZhiningLiu1998/mesa/issues\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/issues/ZhiningLiu1998/mesa\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/ZhiningLiu1998/mesa/graphs/traffic\"\u003e\n    \u003cimg src=\"https://visitor-badge.glitch.me/badge?page_id=ZhiningLiu1998.mesa\"\u003e\n  \u003c/a\u003e\n  \u003c!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section --\u003e\n\u003ca href=\"https://github.com/ZhiningLiu1998/mesa#contributors-\"\u003e\u003cimg src=\"https://img.shields.io/badge/all_contributors-1-orange.svg\"\u003e\u003c/a\u003e\n\u003c!-- ALL-CONTRIBUTORS-BADGE:END --\u003e\n  \u003ca href=\"https://github.com/ZhiningLiu1998/mesa/blob/master/LICENSE\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/license/ZhiningLiu1998/mesa\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003ch3 align=\"center\"\u003e MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler (NeurIPS 2020)\n\u003c/h3\u003e\n\n\u003ch3 align=\"center\"\u003e\nLinks: \n\u003ca href=\"https://papers.nips.cc/paper/2020/file/a64bd53139f71961c5c31a9af03d775e-Paper.pdf\"\u003ePaper\u003c/a\u003e | \n\u003ca href=\"https://arxiv.org/pdf/2010.08830.pdf\"\u003ePDF with Appendix\u003c/a\u003e | \n\u003ca href=\"https://studio.slideslive.com/web_recorder/share/20201020T134559Z__NeurIPS_posters__17343__mesa-effective-ensemble-imbal?s=d3745afc-cfcf-4d60-9f34-63d3d811b55f\"\u003eVideo\u003c/a\u003e | \n\u003ca href=\"https://arxiv.org/abs/2010.08830\"\u003earXiv\u003c/a\u003e | \n\u003ca href=\"https://zhuanlan.zhihu.com/p/268539195\"\u003eZhihu/知乎\u003c/a\u003e\n\u003c/h3\u003e\n\n**MESA is a ***meta-learning-based ensemble learning framework*** for solving class-imbalanced learning problems. It is a task-agnostic general-purpose solution that is able to boost most of the existing machine learning models' performance on imbalanced data.**\n\n\u003c!-- \u003e **NOTE:** The paper will be available through [arXiv](https://arxiv.org/) in a few days. We will provide a link to the .pdf file ASAP. --\u003e\n\n# Cite Us\n\n**If you find this repository helpful in your work or research, we would greatly appreciate citations to the following paper:**\n\n```\n@inproceedings{liu2020mesa,\n    title={MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler},\n    author={Liu, Zhining and Wei, Pengfei and Jiang, Jing and Cao, Wei and Bian, Jiang and Chang, Yi},\n    booktitle={Conference on Neural Information Processing Systems},\n    year={2020},\n}\n```\n\n# Table of Contents\n\n- [Cite Us](#cite-us)\n- [Table of Contents](#table-of-contents)\n- [Background](#background)\n  - [About MESA](#about-mesa)\n  - [Pros and Cons of MESA](#pros-and-cons-of-mesa)\n- [Requirements](#requirements)\n- [Usage](#usage)\n  - [Running main.py](#running-mainpy)\n  - [Running mesa-example.ipynb](#running-mesa-exampleipynb)\n- [Visualization and Results](#visualization-and-results)\n  - [From mesa-example.ipynb](#from-mesa-exampleipynb)\n    - [Class distribution of Mammography dataset](#class-distribution-of-mammography-dataset)\n    - [Visualize the meta-training process](#visualize-the-meta-training-process)\n    - [Comparison with baseline methods](#comparison-with-baseline-methods)\n  - [Other results](#other-results)\n    - [Dataset description](#dataset-description)\n    - [Comparisons of MESA with under-sampling-based EIL methods](#comparisons-of-mesa-with-under-sampling-based-eil-methods)\n    - [Comparisons of MESA with over-sampling-based EIL methods](#comparisons-of-mesa-with-over-sampling-based-eil-methods)\n    - [Comparisons of MESA with resampling-based EIL methods](#comparisons-of-mesa-with-resampling-based-eil-methods)\n- [Miscellaneous](#miscellaneous)\n- [References](#references)\n  - [Contributors ✨](#contributors-)\n\n\n# Background\n\n## About MESA\n\nWe introduce a novel ensemble imbalanced learning (EIL) framework named MESA. It adaptively resamples the training set in iterations to get multiple classifiers and forms a cascade ensemble model. MESA directly learns a parameterized sampling strategy (i.e., meta-sampler) from data to optimize the final metric beyond following random heuristics. It consists of three parts: ***meta sampling*** as well as ***ensemble training*** to build ensemble classifiers, and ***meta-training*** to optimize the meta-sampler. \n\nThe figure below gives an overview of the MESA framework. \n\n![image](https://github.com/ZhiningLiu1998/figures/blob/master/mesa/framework.png)\n\n## Pros and Cons of MESA\n\nHere are some personal thoughts on the advantages and disadvantages of MESA. More discussions are welcome!\n\n**Pros:**\n- \u0026#x1F34E; *Wide compatiblilty.*   \nWe decoupled the model-training and meta-training process in MESA, making it compatible with most of the existing machine learning models.\n- \u0026#x1F34E; *High data efficiency.*  \nMESA performs strictly balanced under-sampling to train each base-learner in the ensemble. This makes it more data-efficient than other methods, especially on highly skewed data sets.\n- \u0026#x1F34E; *Good performance.*  \nThe sampling strategy is optimized for better final generalization performance, we expect this can provide us with a better ensemble model.\n- \u0026#x1F34E; *Transferability.*  \nWe use only task-agnostic meta-information during meta-training, which means that a meta-sampler can be directly used in unseen new tasks, thereby greatly reducing the computational cost brought about by meta-training.\n\n**Cons:**\n- \u0026#x1F34F; *Meta-training cost.*  \nMeta-training repeats the ensemble training process multiple times, which can be costly in practice (By shrinking the dataset used in meta-training, the computational cost can be reduced at the cost of minor performance loss).\n- \u0026#x1F34F; *Need to set aside a separate validation set for training.*  \nThe meta-state is formed by computing the error distribution on both the training and validation sets.\n- \u0026#x1F34F; *Possible unstable performance on small datasets.*  \nSmall datasets may cause the obtained error distribution statistics to be inaccurate/unstable, which will interfere with the meta-training process.\n\n# Requirements\n**Main dependencies:**\n- [Python](https://www.python.org/) (\u003e=3.5)\n- [PyTorch](https://pytorch.org/) (=1.0.0)\n- [Gym](https://gym.openai.com/) (\u003e=0.17.3)\n- [pandas](https://pandas.pydata.org/) (\u003e=0.23.4)\n- [numpy](https://numpy.org/) (\u003e=1.11)\n- [scikit-learn](https://scikit-learn.org/stable/) (\u003e=0.20.1)\n- [imbalanced-learn](https://imbalanced-learn.readthedocs.io/en/stable/index.html) (=0.5.0, optional, for baseline methods)\n\nTo install requirements, run:\n\n```Shell\npip install -r requirements.txt\n```\n\n\u003e **NOTE**: this implementation requires an old version of PyTorch (v1.0.0).\n\u003e You may want to start a new conda environment to run our code. The step-by-step guide is as follows (using torch-cpu for an example):\n\u003e - `conda create --name mesa python=3.7.11`\n\u003e - `conda activate mesa`\n\u003e - `conda install pytorch-cpu==1.0.0 torchvision-cpu==0.2.1 cpuonly -c pytorch`\n\u003e - `pip install -r requirements.txt`\n\u003e \n\u003e These commands should help you to get ready for running mesa. If you have any further questions, please feel free to open an issue or drop me an email.\n\n# Usage\n\nA typical usage example:\n\n```python\n# load dataset \u0026 prepare environment\nargs = parser.parse_args()\nrater = Rater(args.metric)\nX_train, y_train, X_valid, y_valid, X_test, y_test = load_dataset(args.dataset)\nbase_estimator = DecisionTreeClassifier()\n\n# meta-training\nmesa = Mesa(\n    args=args, \n    base_estimator=base_estimator, \n    n_estimators=10)\nmesa.meta_fit(X_train, y_train, X_valid, y_valid, X_test, y_test)\n\n# ensemble training\nmesa.fit(X_train, y_train, X_valid, y_valid)\n\n# evaluate\ny_pred_test = mesa.predict_proba(X_test)[:, 1]\nscore = rater.score(y_test, y_pred_test)\n```\n\n## Running [main.py](https://github.com/ZhiningLiu1998/mesa/blob/master/main.py)\n\nHere is an example:\n\n```powershell\npython main.py --dataset Mammo --meta_verbose 10 --update_steps 1000\n```\n\nYou can get help with arguments by running:\n\n```powershell\npython main.py --help\n```\n\n```\noptional arguments:\n  # Soft Actor-critic Arguments\n  -h, --help            show this help message and exit\n  --env-name ENV_NAME\n  --policy POLICY       Policy Type: Gaussian | Deterministic (default:\n                        Gaussian)\n  --eval EVAL           Evaluates a policy every 10 episode (default:\n                        True)\n  --gamma G             discount factor for reward (default: 0.99)\n  --tau G               target smoothing coefficient(τ) (default: 0.01)\n  --lr G                learning rate (default: 0.001)\n  --lr_decay_steps N    step_size of StepLR learning rate decay scheduler\n                        (default: 10)\n  --lr_decay_gamma N    gamma of StepLR learning rate decay scheduler\n                        (default: 0.99)\n  --alpha G             Temperature parameter α determines the relative\n                        importance of the entropy term against the reward\n                        (default: 0.1)\n  --automatic_entropy_tuning G\n                        Automaically adjust α (default: False)\n  --seed N              random seed (default: None)\n  --batch_size N        batch size (default: 64)\n  --hidden_size N       hidden size (default: 50)\n  --updates_per_step N  model updates per simulator step (default: 1)\n  --update_steps N      maximum number of steps (default: 1000)\n  --start_steps N       Steps sampling random actions (default: 500)\n  --target_update_interval N\n                        Value target update per no. of updates per step\n                        (default: 1)\n  --replay_size N       size of replay buffer (default: 1000)\n\n  # Mesa Arguments\n  --cuda                run on CUDA (default: False)\n  --dataset N           the dataset used for meta-training (default: Mammo)\n  --metric N            the metric used for evaluate (default: aucprc)\n  --reward_coefficient N\n  --num_bins N          number of bins (default: 5). state-size = 2 *\n                        num_bins.\n  --sigma N             sigma of the Gaussian function used in meta-sampling\n                        (default: 0.2)\n  --max_estimators N    maximum number of base estimators in each meta-\n                        training episode (default: 10)\n  --meta_verbose N      number of episodes between verbose outputs. If 'full'\n                        print log for each base estimator (default: 10)\n  --meta_verbose_mean_episodes N\n                        number of episodes used for compute latest mean score\n                        in verbose outputs.\n  --verbose N           enable verbose when ensemble fit (default: False)\n  --random_state N      random_state (default: None)\n  --train_ir N          imbalance ratio of the training set after meta-\n                        sampling (default: 1)\n  --train_ratio N       the ratio of the data used in meta-training. set\n                        train_ratio\u003c1 to use a random subset for meta-training\n                        (default: 1)\n```\n\n## Running [mesa-example.ipynb](https://github.com/ZhiningLiu1998/mesa/blob/master/mesa-example.ipynb)\n\nWe include a highly imbalanced dataset [Mammography](https://imbalanced-learn.readthedocs.io/en/stable/generated/imblearn.datasets.fetch_datasets.html#imblearn.datasets.fetch_datasets) (#majority class instances = 10,923, #minority class instances = 260, imbalance ratio = 42.012) and its variants with flip label noise for quick testing and visualization of MESA and other baselines. \nYou can use [mesa-example.ipynb](https://github.com/ZhiningLiu1998/mesa/blob/master/mesa-example.ipynb) to quickly:\n- conduct a comparative experiment\n- visualize the meta-training process of MESA\n- visualize the experimental results of MESA and other baselines\n\n**Please check [mesa-example.ipynb](https://github.com/ZhiningLiu1998/mesa/blob/master/mesa-example.ipynb) for more details.**\n\n# Visualization and Results\n\n## From [mesa-example.ipynb](https://github.com/ZhiningLiu1998/mesa/blob/master/mesa-example.ipynb)\n\n### Class distribution of Mammography dataset\n![image](https://github.com/ZhiningLiu1998/figures/blob/master/mesa/class-distribution.png)\n\n### Visualize the meta-training process\n\u003c!-- ![image](https://github.com/ZhiningLiu1998/figures/blob/master/mesa/meta-training-process.png) --\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/ZhiningLiu1998/figures/blob/master/mesa/meta-training-process.png\" /\u003e\n\u003c/p\u003e\n\n### Comparison with baseline methods\n![image](https://github.com/ZhiningLiu1998/figures/blob/master/mesa/result.png)\n\n## Other results\n\n### Dataset description\n\n![image](https://github.com/ZhiningLiu1998/figures/blob/master/mesa/datasets.png)\n\n### Comparisons of MESA with under-sampling-based EIL methods\n\n![image](https://github.com/ZhiningLiu1998/figures/blob/master/mesa/comp-USEIL.png)\n\n### Comparisons of MESA with over-sampling-based EIL methods\n\n![image](https://github.com/ZhiningLiu1998/figures/blob/master/mesa/comp-OSEIL.png)\n\n### Comparisons of MESA with resampling-based EIL methods\n\n![image](https://github.com/ZhiningLiu1998/figures/blob/master/mesa/comp-resample.png)\n\n\n\n# Miscellaneous\n\n**Check out our previous work [Self-paced Ensemble](https://github.com/ZhiningLiu1998/self-paced-ensemble) (ICDE 2020).  \nIt is a simple heuristic-based method, but being very fast and works reasonably well.**\n\n**This repository contains:**\n- Implementation of MESA\n- Implementation of 7 ensemble imbalanced learning baselines\n  - `SMOTEBoost` [1]\n  - `SMOTEBagging` [2]\n  - `RAMOBoost` [3]\n  - `RUSBoost` [4]\n  - `UnderBagging` [5]\n  - `BalanceCascade` [6]\n  - `SelfPacedEnsemble` [7]\n- Implementation of 11 resampling imbalanced learning baselines [8]\n\n\u003e **NOTE:** The implementations of the above baseline methods are based on [imbalanced-algorithms](https://github.com/dialnd/imbalanced-algorithms) and [imbalanced-learn](https://github.com/scikit-learn-contrib/imbalanced-learn). \n\n# References\n\n| #   | Reference |\n|-----|-------|\n| [1] | N. V. Chawla, A. Lazarevic, L. O. Hall, and K. W. Bowyer, Smoteboost: Improving prediction of the minority class in boosting. in European conference on principles of data mining and knowledge discovery. Springer, 2003, pp. 107–119|\n| [2] | S. Wang and X. Yao, Diversity analysis on imbalanced data sets by using ensemble models. in 2009 IEEE Symposium on Computational Intelligence and Data Mining. IEEE, 2009, pp. 324–331.|\n| [3] | Sheng Chen, Haibo He, and Edwardo A Garcia. 2010. RAMOBoost: ranked minority oversampling in boosting. IEEE Transactions on Neural Networks 21, 10 (2010), 1624–1642.|\n| [4] | C. Seiffert, T. M. Khoshgoftaar, J. Van Hulse, and A. Napolitano, Rusboost: A hybrid approach to alleviating class imbalance. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 40, no. 1, pp. 185–197, 2010.|\n| [5] | R. Barandela, R. M. Valdovinos, and J. S. Sanchez, New applications´ of ensembles of classifiers. Pattern Analysis \u0026 Applications, vol. 6, no. 3, pp. 245–256, 2003.|\n| [6] | X.-Y. Liu, J. Wu, and Z.-H. Zhou, Exploratory undersampling for class-imbalance learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 39, no. 2, pp. 539–550, 2009. |\n| [7] | Zhining Liu, Wei Cao, Zhifeng Gao, Jiang Bian, Hechang Chen, Yi Chang, and Tie-Yan Liu. 2019. Self-paced Ensemble for Highly Imbalanced Massive Data Classification. 2020 IEEE 36th International Conference on Data Engineering (ICDE). IEEE, 2020, pp. 841-852.\n| [8] | Guillaume Lemaître, Fernando Nogueira, and Christos K. Aridas. Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning. Journal of Machine Learning Research, 18(17):1–5, 2017. |\n## Contributors ✨\n\nThanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --\u003e\n\u003c!-- prettier-ignore-start --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e\u003ca href=\"http://zhiningliu.com\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/26108487?v=4?s=100\" width=\"100px;\" alt=\"\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eZhining Liu\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"#ideas-ZhiningLiu1998\" title=\"Ideas, Planning, \u0026 Feedback\"\u003e🤔\u003c/a\u003e \u003ca href=\"https://github.com/ZhiningLiu1998/mesa/commits?author=ZhiningLiu1998\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003c!-- markdownlint-restore --\u003e\n\u003c!-- prettier-ignore-end --\u003e\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:END --\u003e\n\nThis project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FZhiningLiu1998%2Fmesa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FZhiningLiu1998%2Fmesa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FZhiningLiu1998%2Fmesa/lists"}