{"id":17971077,"url":"https://github.com/idatsy/py-questdb","last_synced_at":"2025-05-05T23:15:18.485Z","repository":{"id":253157436,"uuid":"842642474","full_name":"idatsy/py-questdb","owner":"idatsy","description":"Python wrapper for interacting with QuestDB","archived":false,"fork":false,"pushed_at":"2024-10-02T06:14:45.000Z","size":85,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-05T23:15:13.629Z","etag":null,"topics":["questdb"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/idatsy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-14T19:07:27.000Z","updated_at":"2025-03-14T10:32:48.000Z","dependencies_parsed_at":"2024-08-29T18:28:18.494Z","dependency_job_id":"cf0104b1-0294-4301-becf-16bb7cf137f4","html_url":"https://github.com/idatsy/py-questdb","commit_stats":null,"previous_names":["idatsy/py-questdb"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idatsy%2Fpy-questdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idatsy%2Fpy-questdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idatsy%2Fpy-questdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idatsy%2Fpy-questdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/idatsy","download_url":"https://codeload.github.com/idatsy/py-questdb/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252590633,"owners_count":21772940,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["questdb"],"created_at":"2024-10-29T15:36:37.096Z","updated_at":"2025-05-05T23:15:18.468Z","avatar_url":"https://github.com/idatsy.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# QuestDB Python Library\n\nA robust, strongly-typed Python library for interacting with QuestDB, providing an easy-to-use interface for querying and writing data, with additional features such as easy integration with the built in python logging library automatically saving all logs, including tracebacks and stack traces to DB. Easy set-up with docker-compose and a grafana dashboard is included.\n\n## Key Features\n\n- Asynchronous and synchronous query support\n- Strongly-typed data structures using `msgspec`\n- Easy data insertion with automatic flushing\n- Pandas DataFrame integration\n- Seamless logging integration with Python's built-in logging system\n- Docker Compose setup for QuestDB with Prometheus and Grafana monitoring\n- Comprehensive test coverage\n- Optional error handling callback\n\n## Installation\n\nInstall the QuestDB Python library using pip:\n\n```bash\npip install py-questdb\n```\n\n## Quick Start\n\n```python\nfrom py_questdb import QuestDB\nfrom datetime import datetime\n\n# Initialize QuestDB client\ndb = QuestDB(host=\"localhost\", port=9000)\n\n# Write data\ndb.write({\n    \"table_name\": \"my_table\",\n    \"symbols\": {\"symbol_column\": \"AAPL\"},\n    \"columns\": {\"number_column\": 42, \"string_column\": \"Hello, QuestDB!\"},\n    \"at\": datetime.now()\n})\n\n# Query data (synchronous)\nfor row in db.query_sync(\"SELECT * FROM my_table\"):\n    print(row)\n\n# Query data (asynchronous)\nasync for row in db.query(\"SELECT * FROM my_table\"):\n    print(row)\n\n# Query data as Pandas DataFrame\ndf = db.query_df_sync(\"SELECT * FROM my_table\")\nprint(df)\n```\n\n## Detailed Usage\n\n### Initialization\n\n```python\nfrom py_questdb import QuestDB\n\n# Default connection\ndb = QuestDB()\n\n# Custom connection\ndb = QuestDB(host=\"questdb.example.com\", port=9000, username=\"user\", password=\"pass\")\n```\n\n### Querying Data\n\n```python\n# Synchronous query\nfor row in db.query_sync(\"SELECT * FROM sensors WHERE location = 'NYC'\"):\n    print(f\"Temperature: {row['temperature']}, Humidity: {row['humidity']}\")\n\n# Asynchronous query\nasync for row in db.query(\"SELECT * FROM sensors WHERE location = 'LA'\"):\n    print(f\"Temperature: {row['temperature']}, Humidity: {row['humidity']}\")\n\n# Query into Pandas DataFrame\ndf = db.query_df_sync(\"SELECT * FROM sensors\")\nprint(df.describe())\n\n# Query with type hinting\nclass SensorReading(msgspec.Struct):\n    location: str\n    temperature: float\n    humidity: int\n\nasync for reading in db.query(\"SELECT * FROM sensors\", into_type=SensorReading):\n    print(f\"Location: {reading.location}, Temp: {reading.temperature}°C, Humidity: {reading.humidity}%\")\n\n# Query with error handling\ndef error_handler(response: bytes):\n    print(f\"Error in query: {response.decode()}\")\n\nresults = db.query_sync(\"SELECT * FROM non_existent_table\", error_handler=error_handler)\n```\n\n### Writing Data\n\n```python\nfrom datetime import datetime\nfrom py_questdb.db_types import QuestDBFields\nimport msgspec\n\n# Single write with auto-flush\ndb.write({\n    \"table_name\": \"sensors\",\n    \"symbols\": {\"location\": \"NYC\"},\n    \"columns\": {\"temperature\": 25.5, \"humidity\": 60},\n    \"at\": datetime.now()\n})\n\n# Buffer multiple writes\ndb.buffer_write({\n    \"table_name\": \"sensors\",\n    \"symbols\": {\"location\": \"LA\"},\n    \"columns\": {\"temperature\": 28.0, \"humidity\": 55},\n    \"at\": datetime.now()\n})\n\n# Manually flush buffered writes\ndb.flush()\n\n# Write using custom message class\nclass SensorData(msgspec.Struct):\n    location: str\n    temperature: float\n    humidity: int\n    timestamp: datetime\n\n    def to_quest_db_format(self) -\u003e QuestDBFields:\n        return {\n            \"table_name\": \"sensors\",\n            \"symbols\": {\"location\": self.location},\n            \"columns\": {\"temperature\": self.temperature, \"humidity\": self.humidity},\n            \"at\": self.timestamp\n        }\n\nsensor_data = SensorData(location=\"SF\", temperature=22.5, humidity=65, timestamp=datetime.now())\ndb.write(sensor_data)\n```\n\n### Logging Integration\n\nThe library provides seamless integration with Python's built-in logging system, allowing you to easily write logs directly to QuestDB:\n\n```python\nimport logging\nfrom py_questdb import QuestDB\nfrom py_questdb.log_handler import QuestDBLogHandler\n\n# Initialize QuestDB client\nquestdb_client = QuestDB()\n\n# Create QuestDB log handler\nhandler = QuestDBLogHandler(questdb_client=questdb_client)\n\n# Get logger and add handler\nlogger = logging.getLogger()\nlogger.addHandler(handler)\nlogger.setLevel(logging.INFO)\n\n# Now all your logs will be written to QuestDB\nlogger.info(\"System started\")\nlogger.warning(\"Low disk space\")\nlogger.error(\"Connection failed\", exc_info=True)\n\n# You can log exceptions with full stack traces\ntry:\n    1 / 0\nexcept Exception as e:\n    logger.exception(\"An error occurred\")\n\n# The logs are written to QuestDB in real-time, allowing for immediate analysis and monitoring\n```\n\nThis integration allows you to leverage QuestDB's powerful querying capabilities for log analysis, making it easy to search, filter, and analyze your application logs.\n\n## Docker Compose Setup\n\nThe project includes a Docker Compose configuration to run QuestDB along with Prometheus for metrics collection and Grafana for visualization. To use this setup:\n\n1. Navigate to the `/etc` directory in the project.\n2. Run the following command:\n\n```bash\ndocker-compose up -d\n```\n\nThis will start QuestDB, Prometheus, and Grafana containers. You can access:\n- QuestDB Web Console: http://localhost:9000\n- Prometheus: http://localhost:9090\n- Grafana: http://localhost:3000 (default credentials: admin/admin)\n\n\u003e A pre-configured Grafana dashboard for QuestDB metrics is included in the `questdb_dashboard.json` file, allowing for immediate visualization of your QuestDB instance's performance.\n\n## Testing\n\nRun the test suite using pytest:\n\n```bash\npytest\n```\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nThis project is licensed under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fidatsy%2Fpy-questdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fidatsy%2Fpy-questdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fidatsy%2Fpy-questdb/lists"}