{"id":13627365,"url":"https://github.com/lyttonhao/Factorized-Bilinear-Network","last_synced_at":"2025-04-16T19:31:35.025Z","repository":{"id":86343067,"uuid":"90536713","full_name":"lyttonhao/Factorized-Bilinear-Network","owner":"lyttonhao","description":"FBN: Factorized Bilinear Models for Image Recognition (ICCV 2017)","archived":false,"fork":false,"pushed_at":"2018-01-30T08:06:20.000Z","size":19,"stargazers_count":68,"open_issues_count":1,"forks_count":13,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-11-08T17:48:23.978Z","etag":null,"topics":["factorized-bilinear","image-recognition"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lyttonhao.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-05-07T14:02:59.000Z","updated_at":"2024-04-24T07:05:44.000Z","dependencies_parsed_at":"2023-03-05T23:15:31.642Z","dependency_job_id":null,"html_url":"https://github.com/lyttonhao/Factorized-Bilinear-Network","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyttonhao%2FFactorized-Bilinear-Network","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyttonhao%2FFactorized-Bilinear-Network/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyttonhao%2FFactorized-Bilinear-Network/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lyttonhao%2FFactorized-Bilinear-Network/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lyttonhao","download_url":"https://codeload.github.com/lyttonhao/Factorized-Bilinear-Network/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249268555,"owners_count":21240943,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["factorized-bilinear","image-recognition"],"created_at":"2024-08-01T22:00:33.322Z","updated_at":"2025-04-16T19:31:30.006Z","avatar_url":"https://github.com/lyttonhao.png","language":"Python","funding_links":[],"categories":["\u003ca name=\"Vision\"\u003e\u003c/a\u003e2. Vision"],"sub_categories":["2.1 Image Classification"],"readme":"# Factorized-Bilinear-Network\nThis repository holds the MXNet code for the paper:\n\n\u003e\n**Factorized Bilinear Models for Image Recognition**,\nYanghao Li, Naiyan Wang, Jiaying Liu, and Xiaodi Hou,\narXiv preprint arXiv:1611.05709, 2016\n\u003e\n[[Arxiv Preprint](https://arxiv.org/abs/1611.05709)]\n\n\n## Introduction\n\nFactorized-Bilinear-Network proposes a novel Factorized Bilinear (FB) layer to model the pairwise feature interactions by considering the quadratic terms in the transformations. The factorized parameterization makes our FB layer only require a linear increase of parameters and affordable computational cost. A specific remedy called DropFactor is also devised during the training process to further reduce the risk of overfitting of the FB layer.\n\n#### CIFAR Results:\n\n| Method | params | CIFAR-10 | CIFAR-100 |\n| :------------ | :---------: | :---------: | :-------------: |\n| Inception-BN | 1.7M  | 5.82 | 24.70 |\n| ResNet-164 (ours) | 1.7M | 5.30 | 23.64 |\n| ResNet-1001 (ours) | 10.2M | 4.04 | 20.50 |\n| Inception-BN-FBN | 2.4M | 5.58 | 21.98 |\n| ResNet-164-FBN | 2.2M  | 5.00 | 22.50 |\n| ResNet-1001-FBN | 10.7M | 4.09  | 19.67 |\n\n#### ImageNet Results:\n\n| Method  | Top-1(%) | Top-5 (%) |\n| :------------ | :---------: | :---------: |\n| Inception-BN | 27.5 | 9.2 |\n| Inception-BN-FBN | 26.4 | 8.4 |\n| ResNet-34 | 27.7     | 9.1 |\n| ResNet-34-FBN | 26.3 | 8.4 |\n| ResNet-50     | 24.7 | 7.4 |\n| ResNet-50-FBN | 24.0 | 7.1 |\n\n\n## Prepare Datasets\n\nPrepare the corresponding datasets (CIFAR-10, CIFAR-100 or ImageNet) before training FBNs. In our experiments, we use RecordIO data format to generate `.rec` files for different datasets. Please refer to [MXNet Example](https://github.com/dmlc/mxnet/tree/master/example/image-classification#prepare-datasets). \n\nFor example, you should first prepare the `train.lst` and `test.lst`, then generate the `.rec` file using `im2rec` tool. An example for ImageNet:\n```shell\n$im2rec_path train.lst train/ data/imagenet/train_480_q90.rec resize=480 quality=90\n```\n\n\n## Usage\n\nDownload this repo recursively:\n```shell\ngit clone --recursive https://github.com/lyttonhao/Factorized-Bilinear-Network.git\n```\n\nWe implement two versions of factorized bilinear layer:\n1. `mx.symbol.FMConvolution1()`: similar to Convolution layer with additional FB parameters:\n    * **num_factor**: the factor number for the FB layer. We use 20 in CIFAR dataset. For ImageNet, larger value (50) may improve the performance.\n    * **p**: drop rate for DropFactor\n\n2. `mx.symbol.FMConvolution3()`: speed-up version of `FMConvolution1` using batch_dot, but cost more memory. (The GPU memory may not be enough for large model.)\n\n### CIFAR\nThe original fully connected layer can be replaced by a `FMConvolution3` layer:\n```python\nbilinear = mx.symbol.FMConvolution3(data=in5b, num_filter=num_classes, num_factor=args.fmconv_factor,\n                                    kernel=(1, 1), stride=(1, 1),\n                                    pad=(0, 0), p=args.fmconv_drop, name='bilinear1')\n```\nThe training command:\n```shell\npython train_cifar.py --gpus 2,3 --data-dir /mnt/hdd/lytton/mx_data/cifar-100 --lr 0.1 --wd 0.0001 --batch-size 128 --data-shape 32 --num-epoches 100 --network resnet-small-fmconv --num-classes 100 --lr-step 50,75 --model-prefix cifar100_res18_fmconv22-3  --res-module-num 18  --fmconv-slowstart 3 --fmconv-drop 0.5 --fmconv-factor 20\n```\nMore examples can be seen in the `cifar/run.sh` and you can run `python cifar/train_cifar.py` with `-h` to see more options.\n\n### ImageNet\nOur training policy and augmentation methods follow the [MXNet ResNet](https://github.com/tornadomeet/ResNet#imagenet) which reproduced ResNet in MXNet. Please using specific settings (like lr schedulers and training epochs) in [MXNet ResNet](https://github.com/tornadomeet/ResNet#imagenet) if compares with its original MXNet ResNet results.\n\nAn example of training command:\n```shell\npython train_imagenet.py --gpus 0,1,2,3,4,5,6,7  --model-prefix resnet34-fmconv --network resnet-fmconv --batch-size 256 --aug-level 2  --num-epoches 120 --frequent 50  --lr 0.1 --wd 0.0001   --lr-step 60,75,90  --fmconv-drop 0.5  --fmconv-slowstart 1 --fmconv-factor 50 --depth 34\n```\nUsually, we will cancel the scale/color/aspect augmentation during training (set `--aug-level=1` for around additional 10 epochs) for better result.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flyttonhao%2FFactorized-Bilinear-Network","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flyttonhao%2FFactorized-Bilinear-Network","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flyttonhao%2FFactorized-Bilinear-Network/lists"}