{"id":13418397,"url":"https://github.com/baidu-research/warp-ctc","last_synced_at":"2025-05-14T13:08:46.064Z","repository":{"id":39620094,"uuid":"49623605","full_name":"baidu-research/warp-ctc","owner":"baidu-research","description":"Fast parallel CTC.","archived":false,"fork":false,"pushed_at":"2024-03-04T07:10:15.000Z","size":353,"stargazers_count":4076,"open_issues_count":89,"forks_count":1038,"subscribers_count":353,"default_branch":"master","last_synced_at":"2025-04-12T04:44:10.932Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Cuda","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/baidu-research.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-01-14T04:53:11.000Z","updated_at":"2025-04-09T02:52:42.000Z","dependencies_parsed_at":"2025-02-28T03:07:26.441Z","dependency_job_id":"8f70ee99-db9a-4d70-b4fe-d0bb12ff13b8","html_url":"https://github.com/baidu-research/warp-ctc","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/baidu-research%2Fwarp-ctc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/baidu-research%2Fwarp-ctc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/baidu-research%2Fwarp-ctc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/baidu-research%2Fwarp-ctc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/baidu-research","download_url":"https://codeload.github.com/baidu-research/warp-ctc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254149975,"owners_count":22022852,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T22:01:01.819Z","updated_at":"2025-05-14T13:08:41.052Z","avatar_url":"https://github.com/baidu-research.png","language":"Cuda","funding_links":[],"categories":["Speech Processing","Librarys","Cuda","C++"],"sub_categories":["Tools","[Tools](#tools-1)","Speech Recognition"],"readme":"![Baidu Logo](/doc/baidu-research-logo-small.png)\n\n[In Chinese 中文版](README.zh_cn.md)\n\n# warp-ctc\n\nA fast parallel implementation of CTC, on both CPU and GPU.\n\n## Introduction\n\n[Connectionist Temporal Classification](http://www.cs.toronto.edu/~graves/icml_2006.pdf)\nis a loss function useful for performing supervised learning on sequence data,\nwithout needing an alignment between input data and labels.  For example, CTC\ncan be used to train\n[end-to-end](http://www.jmlr.org/proceedings/papers/v32/graves14.pdf)\n[systems](http://arxiv.org/pdf/1408.2873v2.pdf) for\n[speech recognition](http://arxiv.org/abs/1512.02595),\nwhich is how we have been using it at Baidu's Silicon Valley AI Lab.\n\n![DSCTC](/doc/deep-speech-ctc-small.png)\n\nThe illustration above shows CTC computing the probability of an output\nsequence \"THE CAT \", as a sum over all possible alignments of input sequences\nthat could map to \"THE CAT \", taking into account that labels may be duplicated\nbecause they may stretch over several time steps of the input data (represented by\nthe spectrogram at the bottom of the image).\nComputing the sum of all such probabilities explicitly would be prohibitively costly due to the\ncombinatorics involved, but CTC uses dynamic programming to dramatically\nreduce the complexity of the computation. Because CTC is a differentiable function,\nit can be used during standard SGD training of deep neural networks.\n\nIn our lab, we focus on scaling up recurrent neural networks, and CTC loss is an\nimportant component. To make our system efficient, we parallelized the CTC\nalgorithm, as described in [this paper](http://arxiv.org/abs/1512.02595).\nThis project contains our high performance CPU and CUDA versions of the CTC loss,\nalong with bindings for [Torch](http://torch.ch/).\nThe library provides a simple C interface, so that it is easy to\nintegrate into deep learning frameworks.\n\nThis implementation has improved training scalability beyond the\nperformance improvement from a faster parallel CTC implementation. For\nGPU-focused training pipelines, the ability to keep all data local to\nGPU memory allows us to spend interconnect bandwidth on increased data\nparallelism.\n\n## Performance\n\nOur CTC implementation is efficient compared with many of the other publicly available implementations.  It is\nalso written to be as numerically stable as possible.  The algorithm is numerically sensitive and we have observed\ncatastrophic underflow even in double precision with the standard calculation - the result of division of \ntwo numbers on the order of 1e-324 which should have been approximately one, instead become infinity \nwhen the denominator underflowed to 0.  Instead, by performing the calculation in log space, it is numerically\nstable even in single precision floating point at the cost of significantly more expensive operations.  Instead of\none machine instruction, addition requires the evaluation of multiple transcendental functions.  Because of this,\nthe speed of CTC implementations can only be fairly compared if they are both performing the calculation the same\nway.\n\nWe compare our performance with [Eesen](https://github.com/srvk/eesen/commit/68f2bc2d46a5513cce3c232a645292632a1b08f9), \na CTC implementation built on \n[Theano](https://github.com/mohammadpz/CTC-Connectionist-Temporal-Classification/commit/904e8c72e15334887609d399254cf05a591d570f),\nand a Cython CPU only implementation [Stanford-CTC](https://github.com/amaas/stanford-ctc/commit/c8859897336a349b6c561d2bf2d179fae90b4d67).\nWe benchmark the Theano implementation operating on 32-bit floating-point numbers and doing the calculation in log-space,\nin order to match the other implementations we compare against.  Stanford-CTC was modified to perform the calculation\nin log-space as it did not support it natively.  It also does not support minibatches larger than 1, so would require\nan awkward memory layout to use in a real training pipeline, we assume linear increase in cost with minibatch size.\n\nWe show results on two problem sizes relevant to our English and Mandarin end-to-end models, respectively, where *T* represents the number of timesteps in the input to CTC, *L* represents the length of the labels for each example, and *A* represents the alphabet size.\n\nOn the GPU, our performance at a minibatch of 64 examples ranges from 7x faster to 155x faster than Eesen, and 46x to 68x faster than the Theano implementation.\n\n### GPU Performance\nBenchmarked on a single NVIDIA Titan X GPU.\n\n| *T*=150, *L*=40, *A*=28           | warp-ctc  | Eesen   | Theano  |\n|-----------------------------------|-------|---------|---------|\n| *N*=1                             | 3.1 ms| .5 ms   | 67 ms |\n| *N*=16                            | 3.2 ms| 6  ms   | 94 ms |\n| *N*=32                            | 3.2 ms| 12 ms   | 119 ms |\n| *N*=64                            | 3.3 ms| 24 ms   | 153 ms |\n| *N*=128                           | 3.5 ms| 49 ms   | 231 ms |\n\n\n| *T*=150, *L*=20, *A*=5000         | warp-ctc  | Eesen   | Theano  |\n|-----------------------------------|-------|---------|---------|\n| *N*=1                             | 7 ms  | 40   ms | 120 ms |\n| *N*=16                            | 9 ms  | 619  ms | 385 ms |\n| *N*=32                            | 11 ms | 1238 ms | 665 ms |\n| *N*=64                            | 16 ms | 2475 ms | 1100 ms |\n| *N*=128                           | 23 ms | 4950 ms | 2100 ms |\n\n### CPU Performance\n\nBenchmarked on a dual-socket machine with two Intel E5-2660 v3\nprocessors - warp-ctc used 40 threads to maximally take advantage of the CPU resources.\nEesen doesn't provide a CPU implementation. We noticed that the Theano implementation was not\nparallelizing computation across multiple threads.  Stanford-CTC provides no mechanism\nfor parallelization across threads.\n\n\n| *T*=150, *L*=40, *A*=28           | warp-ctc  | Stanford-CTC   | Theano  |\n|-----------------------------------|-------|---------|---------|\n| *N*=1                             | 2.6 ms|  13 ms  | 15 ms |\n| *N*=16                            | 3.4 ms|  208 ms | 180 ms |\n| *N*=32                            | 3.9 ms|  416 ms | 375 ms |\n| *N*=64                            | 6.6 ms|  832 ms | 700 ms |\n| *N*=128                           |12.2 ms| 1684 ms | 1340 ms |\n\n\n| *T*=150, *L*=20, *A*=5000         | warp-ctc  | Stanford-CTC   | Theano  |\n|-----------------------------------|-------|---------|---------|\n| *N*=1                             | 21 ms |  31 ms  | 850 ms  |\n| *N*=16                            | 37 ms |  496 ms | 10800 ms|\n| *N*=32                            | 54 ms |  992 ms | 22000 ms|\n| *N*=64                            | 101 ms| 1984 ms | 42000 ms|\n| *N*=128                           | 184 ms| 3968 ms | 86000 ms|\n\n\n\n\n\n## Interface\n\nThe interface is in [`include/ctc.h`](include/ctc.h).\nIt supports CPU or GPU execution, and you can specify OpenMP parallelism\nif running on the CPU, or the CUDA stream if running on the GPU. We\ntook care to ensure that the library does not perform memory\nallocation internally, in order to avoid synchronizations and\noverheads caused by memory allocation.\n\n## Compilation\n\nwarp-ctc has been tested on Ubuntu 14.04 and OSX 10.10.  Windows is not supported\nat this time.\n\nFirst get the code:\n\n```\ngit clone https://github.com/baidu-research/warp-ctc.git\ncd warp-ctc\n```\n\ncreate a build directory:\n\n```\nmkdir build\ncd build\n```\n\nif you have a non standard CUDA install `export CUDA_BIN_PATH=/path_to_cuda` so that CMake detects CUDA and\nto ensure Torch is detected, make sure `th` is in `$PATH`\n\nrun cmake and build:\n\n```\ncmake ../\nmake\n```\n\nThe C library and torch shared libraries should now be built along with test\nexecutables.  If CUDA was detected, then `test_gpu` will be built; `test_cpu`\nwill always be built.\n\n## Tests\n\nTo run the tests, make sure the CUDA libraries are in `LD_LIBRARY_PATH` (`DYLD_LIBRARY_PATH` for OSX).\n\nThe Torch tests must be run from the `torch_binding/tests/` directory.\n\n## Torch Installation\n\n```luarocks make torch_binding/rocks/warp-ctc-scm-1.rockspec```\n\nYou can also install without cloning the repository using\n\n```luarocks install http://raw.githubusercontent.com/baidu-research/warp-ctc/master/torch_binding/rocks/warp-ctc-scm-1.rockspec```\n\nThere is a Torch CTC [tutorial](torch_binding/TUTORIAL.md).\n\n## Contributing\n\nWe welcome improvements from the community, please feel free to submit pull\nrequests.\n\n## Known Issues  / Limitations\n\nThe CUDA implementation requires a device of at least compute capability 3.0.\n\nThe CUDA implementation supports a maximum label length of 639 (timesteps are\nunlimited).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbaidu-research%2Fwarp-ctc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbaidu-research%2Fwarp-ctc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbaidu-research%2Fwarp-ctc/lists"}