https://github.com/bentoml/llm-inference-handbook

Everything you need to know about LLM inference
https://github.com/bentoml/llm-inference-handbook

inference-handbook inference-infrastructure inference-optimization llm llm-inference

Last synced: 5 months ago
JSON representation

Everything you need to know about LLM inference

• Host: GitHub
• URL: https://github.com/bentoml/llm-inference-handbook
• Owner: bentoml
• License: apache-2.0
• Created: 2025-07-07T07:44:54.000Z (8 months ago)
• Default Branch: main
• Last Pushed: 2025-09-29T00:33:30.000Z (5 months ago)
• Last Synced: 2025-09-29T02:32:30.000Z (5 months ago)
• Topics: inference-handbook, inference-infrastructure, inference-optimization, llm, llm-inference
• Language: TypeScript
• Homepage: http://www.bentoml.com/llm
• Size: 11.7 MB
• Stars: 235
• Watchers: 2
• Forks: 22
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# 📖 LLM Inference Handbook

This repository contains the source content for [LLM Inference Handbook](https://bentoml.com/llm/), a practical guide for understanding, optimizing, scaling, and operating LLM inference.

[![Twitter](https://badgen.net/badge/icon/@bentomlai/1DA1F2?icon=twitter&label=Follow)](https://twitter.com/bentomlai)

[![Community](https://badgen.net/badge/Join/Community/cyan?icon=slack)](https://l.bentoml.com/join-slack)

## 🔧 Local preview

To preview the site locally:

```bash

pnpm install

pnpm start

```

It will be running at [http://localhost:3000/llm/](http://localhost:3000/llm/).

## 🤝 Contributing

Contributions are welcome! Feel free to open issues, suggest improvements, or submit pull requests.

## 📄 Licenses

This repository is dual-licensed:

- All files in the `docs/` folder are licensed under the [Creative Commons Attribution 4.0 International (CC BY 4.0) License](https://creativecommons.org/licenses/by/4.0/).

- All other files are licensed under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).