{"id":13689228,"url":"https://github.com/asyml/texar-pytorch","last_synced_at":"2025-10-08T17:15:54.094Z","repository":{"id":39174805,"uuid":"174444931","full_name":"asyml/texar-pytorch","owner":"asyml","description":"Integrating the Best of TF into PyTorch, for Machine Learning, Natural Language Processing, and Text Generation.  This is part of the CASL project: http://casl-project.ai/","archived":false,"fork":false,"pushed_at":"2022-04-14T01:26:58.000Z","size":3226,"stargazers_count":745,"open_issues_count":35,"forks_count":115,"subscribers_count":24,"default_branch":"master","last_synced_at":"2025-04-08T13:06:49.425Z","etag":null,"topics":["bert","casl-project","data-processing","deep-learning","dialog-systems","gpt-2","machine-learning","machine-translation","natural-language-processing","python","pytorch","roberta","texar","texar-pytorch","text-data","text-generation","xlnet"],"latest_commit_sha":null,"homepage":"https://asyml.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/asyml.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-08T01:04:09.000Z","updated_at":"2025-01-17T14:37:20.000Z","dependencies_parsed_at":"2022-07-16T10:01:15.262Z","dependency_job_id":null,"html_url":"https://github.com/asyml/texar-pytorch","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asyml%2Ftexar-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asyml%2Ftexar-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asyml%2Ftexar-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/asyml%2Ftexar-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/asyml","download_url":"https://codeload.github.com/asyml/texar-pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253411566,"owners_count":21904151,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","casl-project","data-processing","deep-learning","dialog-systems","gpt-2","machine-learning","machine-translation","natural-language-processing","python","pytorch","roberta","texar","texar-pytorch","text-data","text-generation","xlnet"],"created_at":"2024-08-02T15:01:39.020Z","updated_at":"2025-10-08T17:15:49.032Z","avatar_url":"https://github.com/asyml.png","language":"Python","readme":"\n\u003cdiv align=\"center\"\u003e\n   \u003cimg src=\"./docs/_static/img/logo_h_035.png\"\u003e\u003cbr\u003e\u003cbr\u003e\n\u003c/div\u003e\n \n-----------------\n\n[![pypi](https://img.shields.io/pypi/v/texar_pytorch.svg)](https://pypi.python.org/pypi/texar-pytorch)\n[![Python Build](https://github.com/asyml/texar-pytorch/actions/workflows/main.yml/badge.svg)](https://github.com/asyml/texar-pytorch/actions/workflows/main.yml)\n[![codecov](https://codecov.io/gh/asyml/texar-pytorch/branch/master/graph/badge.svg)](https://codecov.io/gh/asyml/texar-pytorch)\n[![Documentation Status](https://readthedocs.org/projects/texar-pytorch/badge/?version=latest)](https://texar-pytorch.readthedocs.io/en/latest/?badge=latest)\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/asyml/texar-pytorch/blob/master/LICENSE)\n \n\n**Texar-PyTorch** is a toolkit aiming to support a broad set of machine learning, especially natural language processing and text generation tasks. Texar provides a library of easy-to-use ML modules and functionalities for composing whatever models and algorithms. The tool is designed for both researchers and practitioners for fast prototyping and experimentation. Texar-PyTorch was originally developed and is \nactively contributed by [Petuum](https://petuum.com/) and [CMU](https://www.cmu.edu/) in collaboration with other institutes.\nA mirror of this repository is maintained by [Petuum Open Source](https://github.com/petuum).\n\nTexar-PyTorch integrates many of the best features of TensorFlow into PyTorch, delivering highly usable and customizable modules superior to PyTorch native ones. \n\n### Key Features\n* **Two Versions, (Mostly) Same Interfaces**. Texar-PyTorch (this repo) and **[Texar-TF](https://github.com/asyml/texar)** have mostly the same interfaces. Both further combine the best design of TF and PyTorch:\n  - Interfaces and variable sharing in *PyTorch convention*\n  - Excellent factorization and rich functionalities in *TF convention*.\n* **Versatile** to support broad needs:\n   - data processing, model architectures, loss functions, training and inference algorithms, evaluation, ...\n   - encoder(s) to decoder(s), sequential- and self-attentions, memory, hierarchical models, classifiers, ... \n   - maximum likelihood learning, reinforcement learning, adversarial learning, probabilistic modeling, ... \n* **Fully Customizable** at multiple abstraction level -- both novice-friendly and expert-friendly. \n  - Free to plug in whatever external modules, since Texar is fully compatible with the native PyTorch APIs. \n* **Modularized** for maximal re-use and clean APIs, based on principled decomposition of *Learning-Inference-Model Architecture*. \n* **Rich Pre-trained Models, Rich Usage with Uniform Interfaces**. BERT, GPT2, XLNet, etc, for encoding, classification, generation, and composing complex models with other Texar components!\n* Clean, detailed [documentation](https://texar-pytorch.readthedocs.io) and rich [examples](./examples).\n\n\u003cdiv align=\"center\"\u003e\n   \u003cimg src=\"./docs/_static/img/texar_stack.png\"\u003e\u003cbr\u003e\u003cbr\u003e\n\u003c/div\u003e \n\n\u003cdiv align=\"center\"\u003e\n   \u003cimg src=\"./docs/_static/img/texar_modules_big.png\"\u003e\u003cbr\u003e\u003cbr\u003e\n\u003c/div\u003e \n\n\n### Library API Example\nA code example that builds and trains a **Conditional GPT2** model (e.g., for machine translation and text summarization):\n\n```python\nimport texar.torch as tx\nfrom texar.torch.run import *\n\n# (1) Modeling\nclass ConditionalGPT2Model(nn.Module):\n  \"\"\"An encoder-decoder model with GPT-2 as the decoder.\"\"\"\n  def __init__(self, vocab_size):\n    super().__init__()\n    # Use hyperparameter dict for model configuration\n    self.embedder = tx.modules.WordEmbedder(vocab_size, hparams=emb_hparams)\n    self.encoder = tx.modules.TransformerEncoder(hparams=enc_hparams)\n    self.decoder = tx.modules.GPT2Decoder(\"gpt2-small\")  # With pre-trained weights\n\n  def _get_decoder_output(self, batch, train=True):\n    \"\"\"Perform model inference, i.e., decoding.\"\"\"\n    enc_states = self.encoder(inputs=self.embedder(batch['source_text_ids']),\n                              sequence_length=batch['source_length'])\n    if train:  # Teacher-forcing decoding at training time\n      return self.decoder(\n          inputs=batch['target_text_ids'], sequence_length=batch['target_length'] - 1,\n          memory=enc_states, memory_sequence_length=batch['source_length'])\n    else:      # Beam search decoding at prediction time\n      start_tokens = torch.full_like(batch['source_text_ids'][:, 0], BOS)\n      return self.decoder(\n          beam_width=5, start_tokens=start_tokens,\n          memory=enc_states, memory_sequence_length=batch['source_length'])\n\n  def forward(self, batch):\n    \"\"\"Compute training loss.\"\"\"\n    outputs = self._get_decoder_output(batch)\n    loss = tx.losses.sequence_sparse_softmax_cross_entropy(  # Sequence loss\n        labels=batch['target_text_ids'][:, 1:], logits=outputs.logits,\n        sequence_length=batch['target_length'] - 1)  # Automatic masking\n    return {\"loss\": loss}\n\n  def predict(self, batch):\n    \"\"\"Compute model predictions.\"\"\"\n    sequence, _ = self._get_decoder_output(batch, train=False)\n    return {\"gen_text_ids\": sequence}\n\n  \n# (2) Data\n# Create dataset splits using built-in data loaders\ndatasets = {split: tx.data.PairedTextData(hparams=data_hparams[split])\n            for split in [\"train\", \"valid\", \"test\"]}\n\nmodel = ConditionalGPT2Model(datasets[\"train\"].target_vocab.size)\n\n# (3) Training\n# Manage the train-eval loop with the Executor API\nexecutor = Executor(\n  model=model, datasets=datasets,\n  optimizer={\"type\": torch.optim.Adam, \"kwargs\": {\"lr\": 5e-4}},\n  stop_training_on=cond.epoch(20),\n  log_every=cond.iteration(100),\n  validate_every=cond.epoch(1),\n  train_metric=(\"loss\", metric.RunningAverage(10, pred_name=\"loss\")),\n  valid_metric=metric.BLEU(pred_name=\"gen_text_ids\", label_name=\"target_text_ids\"),\n  save_every=cond.validation(better=True),\n  checkpoint_dir=\"outputs/saved_models/\")\nexecutor.train()\nexecutor.test(datasets[\"test\"])\n```\nMany more examples are available [here](./examples).\n\n\n### Installation\nTexar-PyTorch requires:\n\n* `python == 3.6` or `3.7`\n* `torch \u003e= 1.0.0`. Please follow the [official instructions](https://pytorch.org/get-started/locally/#start-locally) to install the appropriate version.\n\nAfter `torch` is installed, install Texar from PyPI: \n```bash\npip install texar-pytorch\n```\n\nTo use cutting-edge features or develop locally, install from source: \n```\ngit clone https://github.com/asyml/texar-pytorch.git\ncd texar-pytorch\npip install .\n```\n\nTo use *tensorboard* support with `Executor`, please install `tensorboardX` with the following command\n\n```commandline\npip install tensorboardX\n```\n\n\n### Getting Started\n* [Examples](./examples)\n* [Documentation](https://texar-pytorch.readthedocs.io)\n\n\n### Reference\nIf you use Texar, please cite the [tech report](https://arxiv.org/abs/1809.00794) with the following BibTex entry:\n\n```\nTexar: A Modularized, Versatile, and Extensible Toolkit for Text Generation\nZhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Sachan and Eric Xing\nACL 2019\n\n@inproceedings{hu2019texar,\n  title={Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation},\n  author={Hu, Zhiting and Shi, Haoran and Tan, Bowen and Wang, Wentao and Yang, Zichao and Zhao, Tiancheng and He, Junxian and Qin, Lianhui and Wang, Di and others},\n  booktitle={ACL 2019, System Demonstrations},\n  year={2019}\n}\n```\n\n### License\n[Apache License 2.0](./LICENSE)\n\n### Companies and Universities Supporting Texar\n\u003cp float=\"left\"\u003e\n   \u003cimg src=\"./docs/_static/img/Petuum.png\" width=\"200\" align=\"top\"\u003e\n   \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n   \u003cimg src=\"https://asyml.io/assets/institutions/cmu.png\", width=\"200\" align=\"top\"\u003e\n\u003c/p\u003e\n","funding_links":[],"categories":["Pytorch \u0026 related libraries｜Pytorch \u0026 相关库","Python","Pytorch \u0026 related libraries"],"sub_categories":["NLP \u0026 Speech Processing｜自然语言处理 \u0026 语音处理:","NLP \u0026 Speech Processing:"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasyml%2Ftexar-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fasyml%2Ftexar-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fasyml%2Ftexar-pytorch/lists"}