https://github.com/mermelstein/duckdb_connection_benchmark

https://github.com/mermelstein/duckdb_connection_benchmark

Last synced: 4 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/mermelstein/duckdb_connection_benchmark
• Owner: mermelstein
• Created: 2024-07-11T02:15:03.000Z (4 months ago)
• Default Branch: main
• Last Pushed: 2024-07-16T19:33:21.000Z (4 months ago)
• Last Synced: 2024-07-16T23:44:03.603Z (4 months ago)
• Language: Python
• Size: 4.88 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# DuckDB Concurrent Connections Benchmark

This repository contains a benchmark script to test the maximum number of concurrent connections to a DuckDB database. The script measures the performance and success rate of running concurrent queries as the number of connections increases. The original script was mostly written by Claude Sonnet 3.5 with guidance and requirements described by a human over several iterations.

## Contents

- `benchmark.py`: The main Python script that runs the benchmark.

- `Dockerfile`: Used to create a Docker image for running the benchmark.

## Requirements

- Docker

## How It Works

The benchmark script does the following:

1. Creates a sample DuckDB database with a simple table.

2. Attempts to run concurrent queries, increasing the number of connections in steps.

3. For each step, it measures:

   - Number of successful queries

   - Time taken to complete all queries

   - Any errors encountered

4. The benchmark stops when it fails to complete all concurrent queries successfully.

## Running the Benchmark

You can run the benchmark using Docker. Follow these steps:

1. Clone this repository

2. run this from the cli: `MAX_CONNECTIONS=10000 STEP=1000 docker-compose up --build`

You can replace the number of max connection and step increases to whatever you want