https://github.com/1lmao/queuing-theory
Modeling and simulating various queuing models such as: M/M/1, M/M/k, G/G/1, G/M/1, and M/G/1 for Large Language Model (LLM) inference systems. These simlulations bridge the gap between theoretical frameworks and practical simulations, this provides valuable insights for developing scalable, energy-efficient AI infrastructures.
https://github.com/1lmao/queuing-theory
gg1 gm1 jupyter-notebook mm1 mmk python queuing-models queuing-simulator queuing-system queuing-theory simpy simpy-library
Last synced: about 2 months ago
JSON representation
Modeling and simulating various queuing models such as: M/M/1, M/M/k, G/G/1, G/M/1, and M/G/1 for Large Language Model (LLM) inference systems. These simlulations bridge the gap between theoretical frameworks and practical simulations, this provides valuable insights for developing scalable, energy-efficient AI infrastructures.
- Host: GitHub
- URL: https://github.com/1lmao/queuing-theory
- Owner: 1lmao
- Created: 2025-01-05T19:06:25.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-01-05T19:55:27.000Z (over 1 year ago)
- Last Synced: 2025-01-21T02:12:52.819Z (over 1 year ago)
- Topics: gg1, gm1, jupyter-notebook, mm1, mmk, python, queuing-models, queuing-simulator, queuing-system, queuing-theory, simpy, simpy-library
- Language: Jupyter Notebook
- Homepage:
- Size: 6.56 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## System Configuration
- **OS:** Linux Mint
- **GPU:** NVIDIA RTX 3090 (24GB)
- **RAM:** 16GB
# Queuing-Theory
Modeling and simulating various queuing models for Large Language Model (LLM) inference systems such as-
M/M/1, M/M/k, G/G/1, G/M/1, and M/G/1:
### Queueing Theory Models (Kendall's Notation)
This project utilizes standard queueing models described by the notation **A/B/C**, where:
* **M (Markovian):** Poisson arrivals or Exponential service times (random/memoryless).
* **G (General):** Arbitrary probability distribution (could be anything).
* **1 or k:** The number of servers available.
**Supported Models:**
* **M/M/1**
* **Arrivals:** Poisson (Random).
* **Service:** Exponential (Random).
* **Servers:** 1.
* *Description:* The classic "Hello World" of queueing theory. Simple random arrivals and service times with a single processor.
* **M/M/k**
* **Arrivals:** Poisson (Random).
* **Service:** Exponential (Random).
* **Servers:** $k$ (Multiple).
* *Description:* A multi-server version of M/M/1. Think of a bank with a single line feeding into $k$ open teller windows.
* **M/G/1**
* **Arrivals:** Poisson (Random).
* **Service:** General (Any distribution).
* **Servers:** 1.
* *Description:* Arrivals are random, but the service time follows a specific, non-random distribution (e.g., fixed time or heavy-tailed). Often analyzed using the Pollaczek–Khinchine formula.
* **G/M/1**
* **Arrivals:** General (Any distribution).
* **Service:** Exponential (Random).
* **Servers:** 1.
* *Description:* The inverse of M/G/1. The service rate is random, but the incoming traffic follows a complex or specific pattern (e.g., bursty traffic).
* **G/G/1**
* **Arrivals:** General.
* **Service:** General.
* **Servers:** 1.
* *Description:* The most complex single-server model. Both arrival and service times can follow any distribution. No simple formulas exist; typically solved via approximation or simulation.