{"id":41976744,"url":"https://github.com/lancopku/label-embedding-network","last_synced_at":"2026-01-25T23:31:54.017Z","repository":{"id":63682217,"uuid":"108612046","full_name":"lancopku/label-embedding-network","owner":"lancopku","description":"Label Embedding Network","archived":false,"fork":false,"pushed_at":"2018-01-29T07:15:51.000Z","size":1352,"stargazers_count":87,"open_issues_count":2,"forks_count":26,"subscribers_count":12,"default_branch":"master","last_synced_at":"2023-02-27T19:39:31.474Z","etag":null,"topics":["cifar10","cifar100","computer-vision","deep-learning","label-embedding","label-representation","mnist","natural-language-processing"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lancopku.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-10-28T01:58:55.000Z","updated_at":"2023-02-27T19:39:31.474Z","dependencies_parsed_at":"2022-11-23T21:00:59.416Z","dependency_job_id":null,"html_url":"https://github.com/lancopku/label-embedding-network","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"purl":"pkg:github/lancopku/label-embedding-network","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lancopku%2Flabel-embedding-network","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lancopku%2Flabel-embedding-network/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lancopku%2Flabel-embedding-network/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lancopku%2Flabel-embedding-network/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lancopku","download_url":"https://codeload.github.com/lancopku/label-embedding-network/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lancopku%2Flabel-embedding-network/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28761814,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-25T23:06:19.311Z","status":"ssl_error","status_checked_at":"2026-01-25T23:03:50.555Z","response_time":113,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cifar10","cifar100","computer-vision","deep-learning","label-embedding","label-representation","mnist","natural-language-processing"],"created_at":"2026-01-25T23:31:53.908Z","updated_at":"2026-01-25T23:31:54.002Z","avatar_url":"https://github.com/lancopku.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Label Embedding Network\nThis is an implementation of the paper [Label Embedding Network: Learning Label Representation for Soft Training of Deep Networks https://arxiv.org/abs/1710.10393](https://arxiv.org/abs/1710.10393).   \n\nLabel Embedding Network can learn label representation (label embedding) during the training process of deep networks. With the proposed method, the label embedding is adaptively and automatically learned through back propagation. The original one-hot represented loss function is converted into a new loss function with soft distributions, such that the originally unrelated labels have continuous interactions with each other during the training process. As a result, the trained model can achieve substantially higher accuracy and with faster convergence speed. Experimental results based on competitive tasks demonstrate the effectiveness of the proposed method, and the learned label embedding is reasonable and interpretable. The proposed method achieves comparable or even better results than the state-of-the-art systems.  \n  \nThe contributions of this work are as follows:  \n**Learning label embedding and compressed embedding**: We propose the Label Embedding Network that can learn label representation for soft training of deep networks. Furthermore, some large-scale tasks have a massive number of labels, and a naive version of label embedding network will suffer from intractable memory cost problem. We propose a solution to automatically learn compressed label embedding, such that the memory cost is substantially reduced.  \n\n**Interpretable and reusable**: The learned label embeddings are reasonable and interpretable, such that we can find meaningful similarities among the labels. The proposed method can learn interpretable label embeddings on both image processing tasks and natural language processing tasks. In addition, the learned label embeddings can be directly adapted for training a new model with improved accuracy and convergence speed.  \n\n**General-purpose solution and competitive results**: The proposed method can be widely applied to various models, including CNN, ResNet, and Seq-to-Seq models. We conducted experiments on computer vision tasks including CIFAR-100, CIFAR-10, and MNIST, and on natural language processing tasks including LCSTS text summarization task and IWSLT2015 machine translation task.\nResults suggest that the proposed method achieves significantly better accuracy than the existing methods (CNN, ResNet, and Seq-to-Seq). We achieve results comparable or even better than the state-of-the-art systems on those tasks.   \n\u003cbr /\u003e  \nbibtex:\n```\n@article{LabelEmb,\nauthor = {Xu Sun and Bingzhen Wei and Xuancheng Ren and Shuming Ma},\ntitle = {Label Embedding Network: Learning Label Representation for Soft Training of Deep Networks},\njournal = {CoRR},\nvolume = {abs/1710.10393},\nyear = {2017}\n}\n```\n\u003cbr /\u003e  \n\n## DataSet\nCIFAR100: [Download](https://www.cs.toronto.edu/~kriz/cifar.html)  \nCIFAR10: [Download](https://www.cs.toronto.edu/~kriz/cifar.html)  \nMNIST: [Download](http://yann.lecun.com/exdb/mnist/)  \nLCSTS: [Download](http://icrc.hitsz.edu.cn/Article/show/139.html)  \nIWSLT2015: [Download](https://nlp.stanford.edu/projects/nmt/data/iwslt15.en-vi/)  \n\n\u003cbr /\u003e  \n\n## Environment and Dependency\n- Ubuntu 16.04\n- Python 3.5\n- Tensorflow 1.3 (for MNIST)\n- Pytorch 0.2.0  (for other tasks)\n\n\u003cbr /\u003e  \n\n## Training for Computer Vision Tasks\nYou can change the training mode by setting the parameter \"mode\", as the `mode=baseline`\nmeans the baseline models(CNN, ResNet-8 or ResNet-18) and the `mode=emb` means our proposed\nlabel embedding network. There are also some other super parameters, see the codes for more\ndetails.  \n### CIFAR-100\n`python3 resnet18.py --mode=baseline`  \n`python3 resnet18.py --mode=emb`  \nThe outputs will be in `./100_results`  \n### CIFAR-10\n`python3 resnet8.py --mode=baseline`  \n`python3 resnet8.py --mode=emb`  \nThe outputs will be in `./10_results`  \n### MNIST\n`python3 cnn.py --mode=baseline`  \n`python3 cnn.py --mode=emb`  \nThe outputs will be in `./cnn_results`  \n`python3 mlp.py --mode=baseline`  \n`python3 mlp.py --mode=emb`  \nThe outputs will be in `./mlp_results`  \n\n\u003cbr /\u003e  \n\u003cbr /\u003e  \n\n## Training for Natural Language Processing Tasks\n### LCSTS (Text Summarization)\n```bash\npython3 preprocess.py -train_src TRAIN_SRC_DATA -train_tgt TRAIN_TGT_DATA\n\t\t      -test_src TEST_SRC_DATA -test_tgt TEST_TGT_DATA\n\t\t      -valid_src VALID_SRC_DATA -valid_tgt VALID_TGT_DATA\n\t\t      -save_data data/lcsts/lcsts.low.share.train.pt\n\t\t      -lower -share\n```\n```bash\npython3 train.py -gpus 0 -config lcsts.yaml -unk -score emb -loss emb -log label_embedding\n```\n```bash\npython3 predict.py -gpus 0 -config lcsts.yaml -unk -score emb -restore data/lcsts/label_embedding/best_rouge_checkpoint.pt\n```\n### IWSLT2015 (English-Vietnamese Machine Translation)\n```bash\npython3 preprocess.py -train_src TRAIN_SRC_DATA -train_tgt TRAIN_TGT_DATA\n\t\t      -test_src TEST_SRC_DATA -test_tgt TEST_TGT_DATA\n\t\t      -valid_src VALID_SRC_DATA -valid_tgt VALID_TGT_DATA\n\t\t      -save_data data/iwslt15/iwslt.low.train.pt\n\t\t      -lower\n```\n```bash\npython3 train.py -gpus 0 -config iwslt.yaml -unk -score emb -loss emb -log label_embedding\n```\n```bash\npython3 predict.py -gpus 0 -config iwslt.yaml -unk -score emb -restore data/lcsts/label_embedding/best_bleu_checkpoint.pt\n```  \n\u003cbr /\u003e  \n\n## Results for Computer Vision Tasks\n### Results of Label Embedding on computer vision:  \n![cv_tab.png](https://github.com/lancopku/LabelEmb/blob/master/Figures/cv_tab.PNG)  \n\n### Error rate curve for CIFAR-100, CIFAR-10, and MNSIT. 20 times experiments (the light color curves) are conducted for credible results both on the baseline and our proposed model. The average results are shown as deep color curves:  \n![cv_fig.png](https://github.com/lancopku/LabelEmb/blob/master/Figures/cv_fig.PNG)  \n\n### Heatmaps generated by the label embeddings:  \n![cv_heatmap.png](https://github.com/lancopku/LabelEmb/blob/master/Figures/cv_heatmap.PNG)  \n\n\u003cbr /\u003e  \n\n## Results for Natural Language Processing Tasks\nResults of Label Embedding for LCSTS:\n![lcsts_tab.png](https://github.com/lancopku/LabelEmb/blob/master/Figures/lcsts_tab.png)  \n\nResults of Label Embedding for IWSLT2015:\n![iwslt_tab.png](https://github.com/lancopku/LabelEmb/blob/master/Figures/iwslt_tab.png)  \n\nExamples of the similarity results on IWSLT2015:\n![iwslt_heatmap.png](https://github.com/lancopku/LabelEmb/blob/master/Figures/iwslt_heatmap.png)  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flancopku%2Flabel-embedding-network","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flancopku%2Flabel-embedding-network","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flancopku%2Flabel-embedding-network/lists"}