https://github.com/torchpipe/torchpipe

Serving Inside Pytorch With Multi-threads
https://github.com/torchpipe/torchpipe

deployment inference llm-serving pipeline-parallelism pytorch ray serve serving tensorrt torch2trt triton-inference-server

Last synced: about 2 months ago
JSON representation

Serving Inside Pytorch With Multi-threads

• Host: GitHub
• URL: https://github.com/torchpipe/torchpipe
• Owner: torchpipe
• License: apache-2.0
• Created: 2023-10-24T10:42:00.000Z (11 months ago)
• Default Branch: main
• Last Pushed: 2024-07-29T08:34:27.000Z (about 2 months ago)
• Last Synced: 2024-07-29T11:42:55.460Z (about 2 months ago)
• Topics: deployment, inference, llm-serving, pipeline-parallelism, pytorch, ray, serve, serving, tensorrt, torch2trt, triton-inference-server
• Language: C++
• Homepage: https://torchpipe.github.io/
• Size: 40 MB
• Stars: 137
• Watchers: 6
• Forks: 12
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project