{"id":24087909,"url":"https://github.com/tinybirdco/tinybird-python-sdk","last_synced_at":"2025-07-15T12:37:26.657Z","repository":{"id":113973341,"uuid":"451470712","full_name":"tinybirdco/tinybird-python-sdk","owner":"tinybirdco","description":null,"archived":false,"fork":false,"pushed_at":"2025-06-06T10:44:48.000Z","size":72,"stargazers_count":2,"open_issues_count":2,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-25T13:04:12.196Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinybirdco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-01-24T13:15:36.000Z","updated_at":"2025-06-06T10:44:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"1df5b261-d54a-4f97-86f3-b2c4cb88a639","html_url":"https://github.com/tinybirdco/tinybird-python-sdk","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tinybirdco/tinybird-python-sdk","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Ftinybird-python-sdk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Ftinybird-python-sdk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Ftinybird-python-sdk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Ftinybird-python-sdk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinybirdco","download_url":"https://codeload.github.com/tinybirdco/tinybird-python-sdk/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinybirdco%2Ftinybird-python-sdk/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265437054,"owners_count":23765094,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-10T03:55:54.606Z","updated_at":"2025-07-15T12:37:26.612Z","avatar_url":"https://github.com/tinybirdco.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tinybird Python SDK\n\nSDK around [Tinybird](https://www.tinybird.co/) APIs.\n\nIf you want to manage Workspaces, Data Sources and Pipes you might be looking for the [tinybird-cli](https://pypi.org/project/tinybird-cli/).\n\nThe SDK is meant to programatically ingest `NDJSON` data or send any request to an `API` instance.\n\nIt contains handlers for:\n- logging events to a Tinybird Data Source from your Python module.\n- logging events from [litellm](https://www.litellm.ai/) to a Tinybird Data Source.\n\n## Ingest to a Tinybird DataSource\n\n```python\nfrom tb.datasource import Datasource\n\nwith Datasource(datasource_name, tinybird_token) as ds:\n    ds \u003c\u003c {'key': 'value', 'key1': 'value1'}\n```\n\nYou can also use the async version:\n\n```python\nfrom tb.a.datasource import AsyncDatasource\n\nasync with AsyncDatasource(datasource_name, tinybird_token, api_url='https://api.us-east.tinybird.co') as ds:\n    await ds \u003c\u003c {'key': 'value', 'key1': 'value1'}\n```\n\nNotes:\n- The `Datasource` object does some in-memory buffering and uses the [events API](https://www.tinybird.co/docs/v2/get-data-in/events-api). \n- It only supports `ndjson` data\n- It automatically handles [Rate Limits](https://www.tinybird.co/docs/get-started/plans/limits#ingestion-limits-api)\n\n## Ingest using an API instance\n\n```python\n\nfrom tb.a.api import AsyncAPI\n\nasync with AsyncAPI(tinybird_token, api_url) as api:\n    await api.post('datasources',\n        params={\n            'name': 'datasource_name',\n            'mode': 'append',\n            'format': 'ndjson',\n            'url': 'https://storage.googleapis.com/davidm-wadus/events.ndjson',\n        }\n    )\n```\n\n- It automatically handles [Rate Limits](https://docs.tinybird.co/api-reference/api-reference.html#limits)\n- Works with any Tinybird API\n- The `post`, `get`, `send` methods signatures are equivalent to the [requests](https://docs.python-requests.org/en/latest/) library.\n\n## Logging from your Python module to a Tinybird Data Source\n\n```python\nimport logging\nfrom tb.logger import TinybirdLoggingHandler\nfrom dotenv import load_dotenv\n\nload_dotenv()\nTB_API_URL = os.getenv(\"TINYBIRD_API_URL\")\nTB_WRITE_TOKEN = os.getenv(\"TINYBIRD_WRITE_TOKEN\")\n\nlogger = logging.getLogger('your-logger-name')\nhandler = TinybirdLoggingHandler(TB_API_URL, TB_WRITE_TOKEN, 'your-app-name')\nformatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')\nhandler.setFormatter(formatter)\nlogger.addHandler(handler)\n```\n\nEach time you call the logger an event to the `tb_logs` DataSource in your Workspace is sent.\n\nTo configure the DataSource name initialize the `TinybirdLogginHandler` like this:\n\n```python\nhandler = TinybirdLoggingHandler(TB_API_URL, TB_WRITE_TOKEN, 'your-app-name', ds_name=\"your_tb_ds_name\")\n```\n\n### Non-blocking logging\n\nIf you want to avoid blocking the main thread you can use a queue to send the logs to a different thread.\n\n```python\nimport logging\nfrom multiprocessing import Queue\nfrom tb.logger import TinybirdLoggingQueueHandler\nfrom dotenv import load_dotenv\n\nload_dotenv()\nTB_API_URL = os.getenv(\"TINYBIRD_API_URL\")\nTB_WRITE_TOKEN = os.getenv(\"TINYBIRD_WRITE_TOKEN\")\n\nlogger = logging.getLogger('your-logger-name')\nhandler = TinybirdLoggingQueueHandler(Queue(-1), TB_API_URL, TB_WRITE_TOKEN, 'your-app-name', ds_name=\"your_tb_ds_name\")\nformatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')\nhandler.setFormatter(formatter)\nlogger.addHandler(handler)\n```\n\n## Logging from Litellm to a Tinybird Data Source\n\nInstall the `ai` extra:\n\n```\npip install tinybird-python-sdk[ai]\n```\n\nThen use the following handler:\n\n```python\nimport litellm\nfrom litellm import acompletion\nfrom tb.litellm.handler import TinybirdLitellmAsyncHandler\n\ncustomHandler = TinybirdLitellmAsyncHandler(\n    api_url=\"https://api.us-east.aws.tinybird.co\", \n    tinybird_token=os.getenv(\"TINYBIRD_TOKEN\"), \n    datasource_name=\"litellm\"\n)\n\nlitellm.callbacks = [customHandler]\n\nresponse = await acompletion(\n    model=\"gpt-3.5-turbo\", \n    messages=[{\"role\": \"user\", \"content\": \"Hi 👋 - i'm openai\"}],\n    stream=True,\n    metadata={\n        \"organization\": \"tinybird\",\n        \"environment\": \"dev\",\n        \"project\": \"litellm_test\",\n        \"chat_id\": \"1234567890\",\n    },\n)\n```\n\nTrack custom metadata using the `metadata` dictionary.\n\nThis is the schema for the `litellm` data source:\n\n```sql\nSCHEMA \u003e\n    `model` LowCardinality(String) `json:$.model` DEFAULT 'unknown',\n    `messages` Array(Map(String, String)) `json:$.messages[:]` DEFAULT [],\n    `user` String `json:$.user` DEFAULT 'unknown',\n    `start_time` DateTime `json:$.start_time` DEFAULT now(),\n    `end_time` DateTime `json:$.end_time` DEFAULT now(),\n    `id` String `json:$.id` DEFAULT '',\n    `stream` Boolean `json:$.stream` DEFAULT false,\n    `call_type` LowCardinality(String) `json:$.call_type` DEFAULT 'unknown',\n    `provider` LowCardinality(String) `json:$.provider` DEFAULT 'unknown',\n    `api_key` String `json:$.api_key` DEFAULT '',\n    `log_event_type` LowCardinality(String) `json:$.log_event_type` DEFAULT 'unknown',\n    `llm_api_duration_ms` Float32 `json:$.llm_api_duration_ms` DEFAULT 0,\n    `cache_hit` Boolean `json:$.cache_hit` DEFAULT false,\n    `response_status` LowCardinality(String) `json:$.standard_logging_object_status` DEFAULT 'unknown',\n    `response_time` Float32 `json:$.standard_logging_object_response_time` DEFAULT 0,\n    `proxy_metadata` String `json:$.proxy_metadata` DEFAULT '',\n    `organization` String `json:$.proxy_metadata.organization` DEFAULT '',\n    `environment` String `json:$.proxy_metadata.environment` DEFAULT '',\n    `project` String `json:$.proxy_metadata.project` DEFAULT '',\n    `chat_id` String `json:$.proxy_metadata.chat_id` DEFAULT '',\n    `response` String `json:$.response` DEFAULT '',\n    `response_id` String `json:$.response.id`,\n    `response_object` String `json:$.response.object` DEFAULT 'unknown',\n    `response_choices` Array(String) `json:$.response.choices[:]` DEFAULT [],\n    `completion_tokens` UInt16 `json:$.response.usage.completion_tokens` DEFAULT 0,\n    `prompt_tokens` UInt16 `json:$.response.usage.prompt_tokens` DEFAULT 0,\n    `total_tokens` UInt16 `json:$.response.usage.total_tokens` DEFAULT 0,\n    `cost` Float32 `json:$.cost` DEFAULT 0,\n    `exception` String `json:$.exception` DEFAULT '',\n    `traceback` String `json:$.traceback` DEFAULT '',\n    `duration` Float32 `json:$.duration` DEFAULT 0\n\n\nENGINE MergeTree\nENGINE_SORTING_KEY start_time, organization, project, model\nENGINE_PARTITION_KEY toYYYYMM(start_time)\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybirdco%2Ftinybird-python-sdk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinybirdco%2Ftinybird-python-sdk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinybirdco%2Ftinybird-python-sdk/lists"}