{"id":13655250,"url":"https://github.com/anitan0925/ResFGB","last_synced_at":"2025-04-23T10:32:21.251Z","repository":{"id":202377755,"uuid":"121972276","full_name":"anitan0925/ResFGB","owner":"anitan0925","description":"Functional gradient boosting based on residual network perception","archived":false,"fork":false,"pushed_at":"2020-09-15T07:51:13.000Z","size":58,"stargazers_count":28,"open_issues_count":2,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-11-10T06:34:33.972Z","etag":null,"topics":["boosting","classification","deep-learning","functional-gradient-boosting"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anitan0925.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-02-18T17:13:32.000Z","updated_at":"2024-10-22T09:49:54.000Z","dependencies_parsed_at":null,"dependency_job_id":"4a228742-764f-4483-894f-4abcc73cbcf4","html_url":"https://github.com/anitan0925/ResFGB","commit_stats":null,"previous_names":["anitan0925/resfgb"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anitan0925%2FResFGB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anitan0925%2FResFGB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anitan0925%2FResFGB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anitan0925%2FResFGB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anitan0925","download_url":"https://codeload.github.com/anitan0925/ResFGB/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250416590,"owners_count":21427018,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["boosting","classification","deep-learning","functional-gradient-boosting"],"created_at":"2024-08-02T03:01:00.402Z","updated_at":"2025-04-23T10:32:16.234Z","avatar_url":"https://github.com/anitan0925.png","language":"Python","funding_links":[],"categories":["2018"],"sub_categories":[],"readme":"Remark: The code is updated from the ICML version. The ICML version corresponds to a commit on May 25, 2018.\n  \n\n# ResFGB\nThis is a Theano(\u003e=1.0.0) implementation of \"[Functional gradient boosting based on residual network perception](https://arxiv.org/abs/1802.09031)\".\n\nResFGB is a functional gradient boosting method for learning a resnet-like deep neural network for non-linear classification problems. The model is composed of a linear classifier such as logistic regression and support vector machine, and a feature extraction.\nIn each iteration, these components are trained by alternate optimization, that is, a linear classifier is trained to classify obtained samples through a feature extraction and this extraction map is updated by stacking a resnet-type layer to move samples along the direction of increasing the linear separability. We finally obtain a highly non-linear classifier forming a residual network.\n\n## Usage\nA simple pseudocode is provided below.\n\n__Note__: `(X,Y)`: training data, `(Xv,Yv)`: validation data, `(Xt,Yt)`: test data.\nThese are numpy arrays.\n`n_data`: the number of training data, `input_dim`: dimension of the input space, `n_class`: the number of classes.\n__A label set should be an integer sequence starting with zero.__\n\n```python\nfrom resfgb.models import ResFGB, get_hyperparams\n\nhparams = get_hyperparams( n_data, input_dim, n_class )\nmodel = ResFGB( **hparams )\nbest_iters,_ ,_ = model.fit( X, Y, Xv, Yv, use_best_iter=True )\n\ntrain_loss, train_acc = model.evaluate( X, Y )\nprint( 'train_loss: {0}, train_acc: {1}'.format(train_loss, train_acc) )\n\ntest_loss, test_acc  = model.evaluate( Xt,  Yt )\nprint( 'test_loss : {0}, test_acc : {1}'.format(test_loss, test_acc) )\n```\n\nSee `examples/sample_resfgb.py` for more detail.\n\n## Hyperparameters\nHyperparameters of ResFGB are mainly divided three types: the first is for learning a linear classifier, the second is for learning a multi-layer network as a resblock, and the other is for the functional gradient method.\n\nThe hyperparameters are listed below.\n'Default' is a value set by the function `resfgb.models.get_hyperparams`.\n`input_dim` and `n_class` stand for the dimension of the input space and the number of classes, respectively.\n\n### For the linear model\n- `shape`[default=(input\\_dim, n\\_class)]\n\t- Shape of the linear model, which __should not be changed__.\n- `wr`[default=1/n_data]\n\t- L2-regularization parameter.\n- `bias`[default=True]\n\t- Flag for whether to include bias term or not.\n- `eta`[default=1e-2]\n\t- Learning rate for Nesterov's momentum method.\n- `momentum`[default=0.9]\n\t- Momentum parameter for Nesterov's momentum method.\n- `minibatch_size`[default=100]\n\t- Minibatch size to compute stochastic gradients.\n- `max_epoch`[default=100]\n\t- The number of epochs for learning a linear model.\n- `tune_eta`[default=True]\n\t- Flag for whether to tune learning rate or not.\n- `scale`[default=1.0] \n\t- Positive number by which a tuned learning rate is multiplied.\n- `eval_iters`[default=1000]\n\t- The number of iterations in a trial for tuning learning rate.\n- `early_stop`[default=10]\n\t- When the training loss does not improve while this number of epochs, the training is stopped. \n\n### For the resblock\n- `shape`[default=(input_dim,100,100,100,100,input_dim)]\n\t- Shape of the multi-layer perceptron. __Dimensions of the input and last layer should set to input_dim__.\n- `wr`[default=1/n_data]\n\t- L2-regularization parameter.\n- `eta`[default=1e-2]\n\t- Learning rate for Nesterov's momentum method.\n- `momentum`[default=0.9] \n\t- Momentum parameter for Nesterov's momentum method.\n- `minibatch_size`[default=100]\n\t- Minibatch size to compute stochastic gradients.\n- `max_epoch`[default=50]\n\t- The number of epochs for learning a linear model.\n- `tune_eta`[default=True]\n\t- Flag for whether to tune learning rate or not.\n- `scale`[default=1.0]\n\t- Positive number by which a tuned learning rate is multiplied.\n- `eval_iters`[default=1000]\n\t- The number of iterations in a trial for tuning learning rate.\n- `early_stop`[default=10]\n\t- When the training loss does not improve while this number of epochs, the training is stopped. \n\n### For the functional gradient method\n- `model_type`[default='logistic']\n\t- Type of the linear model: 'logistic' or 'smooth_hinge'.\n- `model_hparams`[default=model_hparams] \n\t- Dictionary of the hyperparameter for the linear model.\n- `resblock_hparams`[default=resblock_hparams] \n\t- Dictionary of the hyperparameter for the resblock.\n- `fg_eta`[default=1e-1]\n\t- Learning rate used in the functional gradient method.\n- `max_iters`[default=30\n\t- The number of iterations of the functional gradient method, which corresponds to the depth of an obtained network. \n- `seed`[default=1]\n   - Random seed used in the method.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanitan0925%2FResFGB","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanitan0925%2FResFGB","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanitan0925%2FResFGB/lists"}