{"id":30695994,"url":"https://github.com/ruivieira/white-rabbit","last_synced_at":"2026-05-06T17:33:38.714Z","repository":{"id":308974920,"uuid":"1034737282","full_name":"ruivieira/white-rabbit","owner":"ruivieira","description":"vLLM Emulator","archived":false,"fork":false,"pushed_at":"2025-08-16T23:44:03.000Z","size":350,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-17T01:12:33.554Z","etag":null,"topics":["deno","emulator","openai","typescript","vllm"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ruivieira.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-08T22:35:15.000Z","updated_at":"2025-08-16T23:44:03.000Z","dependencies_parsed_at":"2025-08-09T02:34:59.024Z","dependency_job_id":"69643757-5569-451c-b954-d5f034d1cf34","html_url":"https://github.com/ruivieira/white-rabbit","commit_stats":null,"previous_names":["ruivieira/white-rabbit"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/ruivieira/white-rabbit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruivieira%2Fwhite-rabbit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruivieira%2Fwhite-rabbit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruivieira%2Fwhite-rabbit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruivieira%2Fwhite-rabbit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ruivieira","download_url":"https://codeload.github.com/ruivieira/white-rabbit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruivieira%2Fwhite-rabbit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273250252,"owners_count":25072167,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-02T02:00:09.530Z","response_time":77,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deno","emulator","openai","typescript","vllm"],"created_at":"2025-09-02T07:32:57.455Z","updated_at":"2026-05-06T17:33:33.677Z","avatar_url":"https://github.com/ruivieira.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# white-rabbit\n\n[![Tests](https://github.com/ruivieira/white-rabbit/actions/workflows/ci.yml/badge.svg)](https://github.com/ruivieira/white-rabbit/actions/workflows/ci.yml)\n[![JSR](https://jsr.io/badges/@rui/white-rabbit)](https://jsr.io/@rui/white-rabbit)\n[![JSR Score](https://jsr.io/badges/@rui/white-rabbit/score)](https://jsr.io/@rui/white-rabbit)\n[![quay.io](https://quay.io/repository/ruimvieira/white-rabbit/status \"Docker Repository on Quay\")](https://quay.io/repository/ruimvieira/white-rabbit)\n[![pre-commit.ci status](https://results.pre-commit.ci/badge/github/ruivieira/white-rabbit/main.svg)](https://results.pre-commit.ci/latest/github/ruivieira/white-rabbit/main)\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/white-rabbit.png\" alt=\"White Rabbit\" width=\"300\" /\u003e\n\u003c/div\u003e\n\nDeno vLLM emulator providing mock OpenAI-compatible API endpoints for testing and development.\n\n## Purpose\n\nWhite Rabbit is designed to **test integration with vLLM APIs** without requiring a real LLM\ndeployment. The responses are typically gibberish since no actual language model is served - this is\nintentional for testing API compatibility, request/response formats, and integration workflows.\n\nPerfect for:\n\n- Testing vLLM API integration code\n- Development environments where you need vLLM-compatible endpoints\n- CI/CD pipelines that need mock LLM services\n- Load testing API clients without GPU resources\n\n## Installation\n\n### From JSR\n\n```typescript\nimport { genParagraph } from \"jsr:@rui/white-rabbit\";\nimport type { ChatCompletionsRequest, EmbeddingRequest } from \"jsr:@rui/white-rabbit/api\";\n\n// Generate mock text\nconst mockText = genParagraph(5);\n```\n\n### Using specific modules\n\n```typescript\n// Import API types\nimport type {\n  ChatCompletionsRequest,\n  CompletionsRequest,\n  EmbeddingRequest,\n} from \"jsr:@rui/white-rabbit/api\";\n\n// Import text generation utilities\nimport { genParagraph } from \"jsr:@rui/white-rabbit/text-generation\";\n```\n\n## Run locally\n\n```bash\ncd /home/rui/Sync/code/typescript/white-rabbit\n# Deno 1.41+ recommended\ndeno task dev\n# or\ndeno task start\n\n# Run with custom model name\nWR_MODEL=\"my-custom-model\" deno task start\n```\n\n## Configuration\n\n### Environment Variables\n\n**Model Configuration:**\n\n- `WR_MODEL` - Override the model name returned in API responses. If not set, defaults to\n  `Qwen/Qwen2.5-1.5B-Instruct`.\n- `WR_HOST` - Set the host address to bind the server to. If not set, defaults to `localhost`.\n- `WR_PORT` - Set the port number for the server to listen on. If not set, defaults to `8000`.\n\n**Logging Configuration:**\n\n- `WR_LOG_LEVEL` - Set logging level: `DEBUG`, `INFO`, `WARNING`, or `ERROR` (default: `INFO`)\n- `WR_LOG_PREFIX` - Customise log message prefix (default: `🐰`)\n- `WR_LOG_COLORS` - Enable/disable coloured log output: `true` or `false` (default: `true`)\n\n**Log Levels:**\n\n- `DEBUG` - Includes detailed HTTP request logging with headers and body payloads\n- `INFO` - Standard logging without detailed request information\n- `WARNING` - Only warnings and errors\n- `ERROR` - Only error messages\n\n**Examples:**\n\n```bash\n# Set model name to \"granite-3.1-8b\"\nexport WR_MODEL=\"granite-3.1-8b\"\ndeno task start\n\n# Or inline\nWR_MODEL=\"granite-3.1-8b\" deno task start\n\n# Configure host and port\nexport WR_HOST=\"0.0.0.0\"\nexport WR_PORT=\"8080\"\ndeno task start\n\n# Or inline\nWR_HOST=\"0.0.0.0\" WR_PORT=\"8080\" deno task start\n\n# Configure logging\nWR_LOG_LEVEL=DEBUG WR_LOG_PREFIX=\"MY_SERVER\" deno task start\n\n# Disable coloured logs (useful for log files)\nWR_LOG_COLORS=false deno task start\n```\n\n## Direct File Dataset Configuration\n\nYou can configure White Rabbit to use a custom dataset by setting the following environment\nvariables:\n\n- `WR_HF_DATASET`: A direct URL to a CSV or text file (e.g.,\n  `https://huggingface.co/datasets/toxigen/toxigen-data/resolve/main/toxigen.csv`)\n- `WR_HF_COLUMN`: The column name within the dataset to use for text generation\n\n**Important**: For Hugging Face datasets, use the `/resolve/` endpoint instead of `/blob/` to get\nthe raw file content:\n\n- ❌ `https://huggingface.co/datasets/name/repo/blob/main/file.csv` (HTML page)\n- ✅ `https://huggingface.co/datasets/name/repo/resolve/main/file.csv` (raw file)\n\n### Examples\n\n**Using direct file URL (Recommended):**\n\n```bash\nexport WR_HF_DATASET=\"https://huggingface.co/datasets/toxigen/toxigen-data/resolve/main/toxigen.csv\"\nexport WR_HF_COLUMN=\"text\"\ndeno task start\n```\n\n**Using Docker with direct file URL:**\n\n```bash\ndocker run -p 8000:8000 \\\n  -e WR_HF_DATASET=\"https://huggingface.co/datasets/toxigen/toxigen-data/resolve/main/toxigen.csv\" \\\n  -e WR_HF_COLUMN=\"text\" \\\n  white-rabbit:latest\n```\n\n**Using Docker with custom host and port:**\n\n```bash\ndocker run -p 8080:8080 \\\n  -e WR_HOST=\"0.0.0.0\" \\\n  -e WR_PORT=\"8080\" \\\n  white-rabbit:latest\n```\n\n### Example - Toxigen Dataset\n\nThe toxigen dataset contains the following columns:\n\n- `text`: The input text prompt (use this for text generation)\n- `generation`: Generated text response\n- `generation_method`: Method used for generation\n- `group`: Group classification\n- `prompt_label`: Label for the prompt\n- `roberta_prediction`: RoBERTa model prediction\n\nFor text generation, use `WR_HF_COLUMN=\"text\"`.\n\n## Supported Endpoints\n\n### Text Generation\n\n- `POST /generate` - Generate text using Markov chains\n- `GET /health` - Health check endpoint\n\n## Using Custom Hugging Face Datasets\n\nWhite Rabbit supports loading custom datasets directly from Hugging Face using the `/resolve/`\nendpoint. This allows you to train the Markov chain on any CSV dataset hosted on Hugging Face.\n\n### How It Works\n\n1. **Dataset Source**: The system fetches CSV files directly from Hugging Face using the `/resolve/`\n   endpoint\n2. **Column Selection**: You specify which column contains the text data for training\n3. **Automatic Parsing**: The system automatically parses the CSV and extracts the specified column\n4. **Markov Training**: The extracted text is used to train the Markov chain for text generation\n5. **Lazy Loading**: The dataset is loaded only when first needed, then cached in memory for\n   subsequent requests\n\n**Performance Note**: The first text generation request may experience a delay while the dataset\ndownloads and processes. However, once loaded, the dataset is cached in memory, so all subsequent\ninference requests will be fast with no additional delays.\n\n### Example: Toxigen Dataset for Toxic Model Detection\n\nThe [Toxigen dataset](https://huggingface.co/datasets/toxigen/toxigen-data) is particularly useful\nfor testing and evaluating toxic content detection models. This dataset contains:\n\n- **Purpose**: Designed to test how well language models can detect and avoid generating toxic\n  content\n- **Content**: Contains prompts that are designed to elicit toxic responses from language models\n- **Use Case**: Perfect for testing whether your text generation system can avoid producing harmful\n  content\n\n#### Setting Up Toxigen Dataset\n\n```bash\n# Set the dataset URL (use /resolve/ for raw file access)\nexport WR_HF_DATASET=\"https://huggingface.co/datasets/toxigen/toxigen-data/resolve/main/toxigen.csv\"\n\n# Specify the column containing the text prompts\nexport WR_HF_COLUMN=\"text\"\n\n# Start the server\ndeno task start\n```\n\n#### Dataset Structure\n\nThe toxigen dataset contains these columns:\n\n- `text`: The input text prompt (use this for text generation)\n- `generation`: Generated text response\n- `generation_method`: Method used for generation\n- `group`: Group classification\n- `prompt_label`: Label for the prompt\n- `roberta_prediction`: RoBERTa model prediction\n\n#### Testing Toxic Content Detection\n\nWith the toxigen dataset loaded, you can:\n\n1. **Generate Text**: Use the `/generate` endpoint to create text based on the dataset\n2. **Evaluate Safety**: Check if the generated text maintains appropriate content standards\n3. **Model Testing**: Test how well your system handles potentially problematic prompts\n4. **Content Filtering**: Implement additional safety measures based on the generated content\n\n#### Example API Call\n\n```bash\ncurl -X POST http://localhost:8000/generate \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"prompt\": \"Write a story about\",\n    \"max_tokens\": 100\n  }'\n```\n\n### Other Dataset Examples\n\nYou can use any CSV dataset hosted on Hugging Face. Here are some other popular options:\n\n- **Creative Writing**:\n  `https://huggingface.co/datasets/writing-prompts/resolve/main/writing-prompts.csv`\n- **Conversation Data**:\n  `https://huggingface.co/datasets/conversation-ai/resolve/main/conversation.csv`\n- **Custom Datasets**: Upload your own CSV files to Hugging Face and use the `/resolve/` endpoint\n\n### Best Practices\n\n1. **Use `/resolve/` endpoint**: Always use `/resolve/` instead of `/blob/` for raw file access\n2. **Column validation**: Ensure the specified column exists and contains appropriate text data\n3. **Content review**: Review generated content, especially when using datasets with sensitive\n   content\n4. **Testing**: Test your system thoroughly before deploying with custom datasets\n\n### Chat Completions\n\n- `POST /v1/chat/completions` - Generate chat completions\n\n### Completions (Legacy)\n\n- `POST /v1/completions` - Generate text completions\n\n### Embeddings\n\n- `POST /v1/embeddings` - Generate text embeddings\n\n### Models\n\n- `GET /v1/models` - List available models\n\n### Tokenization\n\n- `POST /tokenize` - Tokenise text into token IDs\n- `POST /detokenize` - Convert token IDs back to text\n\n### Server Information\n\n- `GET /version` - Return vLLM version information\n- `GET /stats` - Return server statistics and metrics\n\n## Usage Examples\n\n### Chat Completions\n\n```bash\ncurl --request POST \\\n  --url http://localhost:8000/v1/chat/completions \\\n  --header 'Content-Type: application/json' \\\n  --data '{\n  \"model\": \"test-model\",\n  \"messages\": [\n    {\n      \"role\": \"user\",\n      \"content\": \"What is the opposite of down?\"\n    }\n  ],\n  \"temperature\": 0,\n  \"logprobs\": true,\n  \"max_tokens\": 500\n}'\n```\n\n### Text Completions\n\n```bash\ncurl --request POST \\\n  --url http://localhost:8000/v1/completions \\\n  --header 'Content-Type: application/json' \\\n  --data '{\n  \"model\": \"test-model\",\n  \"prompt\": \"Once upon a time\",\n  \"max_tokens\": 100,\n  \"n\": 1\n}'\n```\n\n### Embeddings\n\n```bash\ncurl --request POST \\\n  --url http://localhost:8000/v1/embeddings \\\n  --header 'Content-Type: application/json' \\\n  --data '{\n  \"model\": \"test-embedding-model\",\n  \"input\": \"The quick brown fox jumps over the lazy dog\",\n  \"encoding_format\": \"float\"\n}'\n```\n\n### Multiple Text Embeddings\n\n```bash\ncurl --request POST \\\n  --url http://localhost:8000/v1/embeddings \\\n  --header 'Content-Type: application/json' \\\n  --data '{\n  \"model\": \"test-embedding-model\",\n  \"input\": [\n    \"First text to embed\",\n    \"Second text to embed\",\n    \"Third text to embed\"\n  ],\n  \"encoding_format\": \"float\",\n  \"dimensions\": 768\n}'\n```\n\n### Models\n\n```bash\ncurl --request GET \\\n  --url http://localhost:8000/v1/models\n```\n\n### Tokenization\n\n```bash\ncurl --request POST \\\n  --url http://localhost:8000/tokenize \\\n  --header 'Content-Type: application/json' \\\n  --data '{\n  \"model\": \"test-model\",\n  \"text\": \"Hello, world!\",\n  \"add_special_tokens\": true\n}'\n```\n\n### Detokenization\n\n```bash\ncurl --request POST \\\n  --url http://localhost:8000/detokenize \\\n  --header 'Content-Type: application/json' \\\n  --data '{\n  \"model\": \"test-model\",\n  \"tokens\": [1, 15496, 11, 1917, 0, 2]\n}'\n```\n\n### Version Information\n\n```bash\n# Basic version info\ncurl --request GET \\\n  --url http://localhost:8000/version\n\n# Detailed version info with build details\ncurl --request GET \\\n  --url http://localhost:8000/version?details=true\n```\n\n### Server Statistics\n\n```bash\ncurl --request GET \\\n  --url http://localhost:8000/stats\n```\n\n## Features\n\n### Core Functionality\n\n- **Mock Data Generation**: Generates realistic-looking mock responses with random text and\n  embeddings\n- **Markov completions**: Uses a small QA dataset and a weighted Markov chain to produce more\n  topic-relevant answers for `/v1/completions` and `/v1/chat/completions`. Supports custom Hugging\n  Face datasets with configurable column extraction and format handling.\n- **OpenAI API Compatibility**: Follows OpenAI API specifications for request/response formats\n- **Multiple Input Support**: Supports single strings, arrays of strings, and token ID arrays\n- **Configurable Parameters**: Supports parameters like `max_tokens`, `n`, `logprobs`, `dimensions`,\n  etc.\n- **Normalised Embeddings**: Generated embeddings are unit vectors (normalised to length 1)\n- **Token Usage Tracking**: Returns realistic token usage statistics\n- **Model Management**: Lists available models with metadata\n- **Tokenization Support**: Mock tokenization and detokenization with consistent token IDs\n- **Server Monitoring**: Provides version information and real-time server statistics\n\n### Logging and Monitoring\n\n- **vLLM-Compatible Logging**: Professional logging system that matches vLLM's output format\n- **Periodic Statistics**: Automatic throughput reporting every 10 seconds (like vLLM)\n  - Prompt tokens per second\n  - Generation tokens per second\n  - Running and total request counts\n  - Server uptime tracking\n- **Request Tracing**: Debug-level logging of all incoming requests and processing steps\n- **HTTP Request Logging**: When `LOG_LEVEL=DEBUG`, logs all HTTP requests with method, path,\n  headers, and body payloads\n- **Configurable Log Levels**: DEBUG, INFO, WARNING, ERROR with environment variable control\n- **Coloured Output**: Colour-coded log messages for easy reading (configurable)\n- **Graceful Shutdown**: Proper signal handling and resource cleanup\n\n### HTTP Request Logging\n\nWhen `WR_LOG_LEVEL=DEBUG` is set, White Rabbit provides comprehensive HTTP request logging that\nincludes:\n\n- **Request Method**: HTTP method (GET, POST, etc.)\n- **Request Path**: Full URL path\n- **Request Headers**: All HTTP headers with values\n- **Request Body**: Complete request body payload for POST requests\n- **Structured Format**: Clear markers to identify request log boundaries\n\nThis is particularly useful for:\n\n- **Debugging**: Troubleshooting API integration issues\n- **Development**: Understanding exactly what clients are sending\n- **Testing**: Verifying request payloads during development\n- **Monitoring**: Tracking API usage patterns\n\n**Example DEBUG level output:**\n\n```\n🐰:server DEBUG 08-13 17:45:49 [server.ts:386] === HTTP Request Log ===\n🐰:server DEBUG 08-13 17:45:49 [server.ts:387] Method: POST\n🐰:server DEBUG 08-13 17:45:49 [server.ts:388] Path: /v1/chat/completions\n🐰:server DEBUG 08-13 17:45:49 [server.ts:389] Headers: {\n  \"content-type\": \"application/json\",\n  \"user-agent\": \"curl/7.68.0\"\n}\n🐰:server DEBUG 08-13 17:45:49 [server.ts:397] Body: {\"model\":\"test\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}]}\n🐰:server DEBUG 08-13 17:45:49 [server.ts:404] === End Request Log ===\n```\n\n**Note**: Request body logging is only performed for POST requests. GET requests will log method,\npath, and headers but not body content.\n\nAny string is accepted for the `model` argument across all endpoints. However, the actual model name\nreturned in responses is determined by the `WR_MODEL` environment variable (or the default\n`Qwen/Qwen2.5-1.5B-Instruct` if not set), regardless of what the client requests.\n\n## Docker\n\n### Build and Run\n\n```bash\n# Build the Docker image\ndocker build -t white-rabbit .\n\n# Run the container\ndocker run -p 8000:8000 white-rabbit\n\n# Run with custom port\ndocker run -p 9000:8000 white-rabbit\n\n# Run with custom model name\ndocker run -p 8000:8000 -e WR_MODEL=\"granite-3.1-8b\" white-rabbit\n\n# Run with custom host and port\ndocker run -p 8080:8080 \\\n  -e WR_HOST=\"0.0.0.0\" \\\n  -e WR_PORT=\"8080\" \\\n  white-rabbit\n\n# Run with direct file dataset\ndocker run -p 8000:8000 \\\n  -e WR_HF_DATASET=\"https://huggingface.co/datasets/toxigen/toxigen-data/resolve/main/toxigen.csv\" \\\n  -e WR_HF_COLUMN=\"prompt\" \\\n  white-rabbit\n```\n\n### Docker Features\n\n- **Multi-stage build**: Uses UBI9 as builder base for security and compliance\n- **Compiled binary**: Compiles Deno application to a single executable binary\n- **Minimal runtime**: Final image uses UBI9 minimal for reduced attack surface\n- **Non-root user**: Runs as dedicated `whiterabbit` user for security\n- **Health check**: Built-in health check endpoint monitoring\n- **Optimised layers**: Efficient Docker layer caching for faster rebuilds\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruivieira%2Fwhite-rabbit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fruivieira%2Fwhite-rabbit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruivieira%2Fwhite-rabbit/lists"}