{"id":23332515,"url":"https://github.com/clauspruefer/python-dbpool","last_synced_at":"2025-10-05T22:45:58.117Z","repository":{"id":209513198,"uuid":"119281694","full_name":"clauspruefer/python-dbpool","owner":"clauspruefer","description":"Tiny Python PostgreSQL Database Connection Pool","archived":false,"fork":false,"pushed_at":"2025-08-24T09:31:59.000Z","size":25928,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-24T16:54:51.870Z","etag":null,"topics":["database-pooling","dbpool","postgres","psycopg2","python"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/pgdbpool","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clauspruefer.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["webcodex1","clauspruefer"]}},"created_at":"2018-01-28T17:35:50.000Z","updated_at":"2025-08-24T08:56:46.000Z","dependencies_parsed_at":"2024-11-19T10:31:44.251Z","dependency_job_id":"cba245fc-b461-4688-a534-850c37833365","html_url":"https://github.com/clauspruefer/python-dbpool","commit_stats":null,"previous_names":["clauspruefer/python-dbpool"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/clauspruefer/python-dbpool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clauspruefer%2Fpython-dbpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clauspruefer%2Fpython-dbpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clauspruefer%2Fpython-dbpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clauspruefer%2Fpython-dbpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clauspruefer","download_url":"https://codeload.github.com/clauspruefer/python-dbpool/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clauspruefer%2Fpython-dbpool/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278532355,"owners_count":26002345,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-05T02:00:06.059Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database-pooling","dbpool","postgres","psycopg2","python"],"created_at":"2024-12-20T23:14:59.905Z","updated_at":"2025-10-05T22:45:58.111Z","avatar_url":"https://github.com/clauspruefer.png","language":"Python","funding_links":["https://github.com/sponsors/webcodex1","https://github.com/sponsors/clauspruefer"],"categories":[],"sub_categories":[],"readme":"# :elephant: Python PgDatabase-Pool Module\n\n![push main](https://github.com/clauspruefer/python-dbpool/actions/workflows/pylint.yaml/badge.svg)\n[![PyPI version](https://badge.fury.io/py/pgdbpool.svg)](https://badge.fury.io/py/pgdbpool)\n\n## 1. Primary Scope\n\nThe **pgdbpool** Python module is a tiny **PostgreSQL Database Connection De-Multiplexer**, primarily designed for *Web- / Application Servers*.\n\n**Key Features:**\n- **Multi-endpoint support**: Load balance across multiple PostgreSQL servers\n- **Flexible threading models**: Choose between threaded and non-threaded modes\n- **Transaction control**: Manual commit support for complex transactions\n- **High availability**: Built-in failover and connection management\n\n## 2. Current Implementation\n\n```text\n+----------------------+                         +---------------------\n| WebServer Service.py | -- Handler Con #1 ----\u003e | PostgreSQL\n| Request / Thread #1  |                         | Backend #1\n+----------------------+                         |\n                                                 |\n+----------------------+                         |\n| WebServer Service.py | -- Handler Con #2 ----\u003e | PostgreSQL\n| Request / Thread #2  |                         | Backend #2\n+----------------------+                         +---------------------\n```\n\n### 2.1. Multiple Database Endpoints\n\nThe connection pool now supports **multiple PostgreSQL database endpoints** for load balancing and high availability:\n\n- ✅ Configure multiple database hosts in the configuration\n- ✅ Connections are automatically distributed across available endpoints\n- ✅ Provides built-in load balancing for read operations\n- ✅ Enhances fault tolerance and scalability\n\n### 2.2. Concept / Simplicity\n\nIf configured in a Web Server's WSGI Python script, the pooling logic is straightforward:\n\n1. Check if a free connection in the pool exists.\n2. Verify if the connection is usable (SQL ping).\n3. Use the connection and protect it from being accessed until the query/queries are completed.\n4. Release the connection for reuse.\n5. Reconnect to the endpoint if the connection is lost.\n\n## 3. Thread Safety / Global Interpreter Lock\n\n### 3.1. Threading Model Configuration\n\nThe pool now supports **two threading models** that can be configured based on your application's architecture:\n\n- **`threaded`** (default): Uses `threading.Lock()` for thread safety, suitable for traditional multi-threaded web servers\n- **`non-threaded`**: Disables locking for single-threaded applications, eliminating GIL overhead\n\n### 3.2. Threaded Mode\n\nThread safety is ensured via `lock = threading.Lock()`, which relies on a kernel mutex `syscall()`.\n\nWhile this concept works, the GIL (Global Interpreter Lock) in Python thwarts scalability under heavy loads in a threaded Web Server setup.\n\n### 3.3. Non-Threaded Mode\n\nFor applications using a single-threaded, process-per-request model (like the FalconAS Python Application Server), the non-threaded mode provides:\n\n- **No locking overhead** - eliminates mutex syscalls\n- **Better performance** - avoids GIL contention  \n- **Simpler architecture** - designed for 1 Process == 1 Python Interpreter\n\n\u003e[!IMPORTANT]\n\u003e Refer to Section **6: Future** for more details on threading-less architectures.\n\n## 4. Dependencies / Installation\n\n**Python 3** and the **psycopg2** module are required.\n\n```bash\n# install (debian)\napt-get install python3-psycopg2\npip install pgdbpool\n```\n\n## 5. Documentation / Examples\n\nSee documentation either at `./doc` or [https://pythondocs.webcodex.de/pgdbpool/v1.0](https://pythondocs.webcodex.de/pgdbpool/v1.0)\nfor detailed explanation / illustrative examples.\n\n### 5.1. Multiple Database Configuration\n\n```python\nconfig = {\n    'db': [\n        {\n            'host': 'postgres-server-1.example.com',\n            'name': 'mydb',\n            'user': 'dbuser',\n            'pass': 'dbpass'\n        },\n        {\n            'host': 'postgres-server-2.example.com', \n            'name': 'mydb',\n            'user': 'dbuser',\n            'pass': 'dbpass'\n        }\n    ],\n    'groups': {\n        'default': {\n            'connection_count': 20,\n            'autocommit': True\n        }\n    }\n}\n```\n\n### 5.2. Threading Model Configuration\n\n```python\n# for non-threaded applications (e.g., FalconAS)\nconfig = {\n    'type': 'non-threaded',\n    'db': { ... },\n    'groups': { ... }\n}\n\n# for traditional threaded applications (default)\nconfig = {\n    'type': 'threaded',  # or omit for default\n    'db': { ... },\n    'groups': { ... }\n}\n```\n\n### 5.3. Manual Transaction Control\n\n```python\nimport pgdbpool as dbpool\n\ndbpool.Connection.init(config)\n\n# for autocommit=False connections\nwith dbpool.Handler('group1') as db:\n    db.query('INSERT INTO table1 VALUES (%s)', ('value1',))\n    db.query('INSERT INTO table2 VALUES (%s)', ('value2',))\n    db.commit()  # Manual commit\n```\n\n## 6. Future\n\n### 6.1. FalconAS Compatibility\n\nThe DB-pooling functionality is now compatible with the FalconAS\nPython Application Server (https://github.com/WEBcodeX1/http-1.2).\n\nThe implemented model: **1 Process == 1 Python Interpreter (threading-less)**,\neffectively solving the GIL issue through the `non-threaded` configuration mode.\n\n### 6.2. Load Balancing\n\nThe pool now supports multiple (read-load-balanced) PostgreSQL endpoints:\n\n- ✅ **Implemented**: Multiple database endpoint configuration\n- ✅ **Implemented**: Automatic connection distribution across endpoints\n- ✅ **Implemented**: Built-in load balancing for database connections\n- ✅ **Implemented**: Read / write / endpoint group separation\n\n[![linting: pylint](https://img.shields.io/badge/linting-pylint-yellowgreen)](https://github.com/PyCQA/pylint)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclauspruefer%2Fpython-dbpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclauspruefer%2Fpython-dbpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclauspruefer%2Fpython-dbpool/lists"}