{"id":28639470,"url":"https://github.com/taosdata/perspective-connect-demo","last_synced_at":"2026-01-30T06:36:04.644Z","repository":{"id":282986686,"uuid":"948279637","full_name":"taosdata/perspective-connect-demo","owner":"taosdata","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-18T03:02:47.000Z","size":19854,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-06-12T19:41:33.003Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/taosdata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-14T03:38:51.000Z","updated_at":"2025-03-25T17:33:11.000Z","dependencies_parsed_at":"2025-03-18T02:25:48.235Z","dependency_job_id":"a2d04c54-2d55-45c0-b54f-2504a582dc7f","html_url":"https://github.com/taosdata/perspective-connect-demo","commit_stats":null,"previous_names":["taosdata/perspective-connect-demo"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/taosdata/perspective-connect-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taosdata%2Fperspective-connect-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taosdata%2Fperspective-connect-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taosdata%2Fperspective-connect-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taosdata%2Fperspective-connect-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/taosdata","download_url":"https://codeload.github.com/taosdata/perspective-connect-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taosdata%2Fperspective-connect-demo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28906668,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-30T04:02:34.702Z","status":"ssl_error","status_checked_at":"2026-01-30T04:02:33.562Z","response_time":66,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-12T19:40:27.133Z","updated_at":"2026-01-30T06:36:04.597Z","avatar_url":"https://github.com/taosdata.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- omit in toc --\u003e\n# Integration With Perspective\n\nPerspective is an open-source and powerful data visualization library that enables interactive, real-time data analysis in web applications. Developed by Prospective.co, Perspective leverages WebAssembly and Web Workers to provide high-performance data visualization capabilities directly in the browser. With Perspective, you can create dynamic dashboards, charts, and tables that update in real-time, allowing users to explore and interact with data seamlessly. Perspective's flexibility, speed, and ease of use make it an excellent choice for building data-driven applications that require real-time data visualization and analysis.\n\nThrough TDengine's Python connector` Perspective can support TDengine data sources and provide real-time functions such as various data charts and analysis.\n\n\u003c!-- omit in toc --\u003e\n## Table of Contents\n\n- [1. Introduction](#1-introduction)\n- [2. Prerequisites on Linux](#2-prerequisites-on-linux)\n- [3. Display Visualized Data](#3-display-visualized-data)\n- [4. Develop Perspective Server](#4-develop-perspective-server)\n  - [4.1 Insert Data into TDengine](#41-insert-data-into-tdengine)\n  - [4.2 Read TDengine Time-Series Data in Real-Time](#42-read-tdengine-time-series-data-in-real-time)\n  - [4.3 Embed \\\u0026 Configure Perspective Viewer](#43-embed--configure-perspective-viewer)\n\n## 1. Introduction\n\nPerspective Server uses `TDengine Python Connector` to obtain real-time time-series data from the TDengine database, and provides the data to `Perspective Viewer` through Websocket to display various charts.\n\n![TDengine - Perspective Architecture](imgs/tdengine_prsp_architecture.jpg)\n\n## 2. Prerequisites on Linux\n\n- TDengine 3.3.5.8 and above version is installed and running normally (both Enterprise and Community versions are available).\n- taosAdapter is running normally, refer to [taosAdapter Reference](../../../tdengine-reference/components/taosadapter/).\n- Install python 3.10+ version, refer to [install python](https://docs.python.org/).\n- Run the 'install.sh' script to download and install the TDengine client library and related dependencies locally.\n\n## 3. Display Visualized Data\n\n   1. **start the perspective service:**\n\n       Run the 'run.sh' script, start the perspective service, pull data from TDengine every 300 milliseconds, and stream the data to the web-based perspective viewer.\n       \n       ```shell\n       sh run.sh\n       ```\n\n   2. **Start a static web service:**\n\n      Start a static web service and access prsp-viewer.html resources to display visual data in the browser.\n      \n      ```pycon\n      python -m http.server 8081\n      ```\n\n      Display effect\n\n      [![TDengine - Perspective Integration](imgs/prsp-tdengine_short.gif)](imgs/prsp-tdengine.gif)\n\n## 4. Develop Perspective Server\n\n### 4.1 Insert Data into TDengine\n\n   The producer.py script to periodically insert data into TDengine. This script simulates real-time data ingestion by generating random data points and inserting them into the TDengine database.\n   \n   1. **Connecting to TDengine:**\n\n      ```python\n      import taosws\n      \n      TAOS_HOST = \"localhost\"\n      \n      # =============================================================================\n      # TDengine connection parameters\n      # =============================================================================\n      TAOS_HOST = \"localhost\"          # TDengine server host\n      TAOS_PORT = 6041                 # TDengine server port\n      TAOS_USER = \"root\"               # TDengine username\n      TAOS_PASSWORD = \"taosdata\"       # TDengine password\n      \n      TAOS_DATABASE = \"power\"          # TDengine database name\n      TAOS_TABLENAME = \"meters\"        # TDengine table name\n      \n      \n      TAOS_PORT = 6041\n      TAOS_USER = \"root\"\n      TAOS_PASSWORD = \"taosdata\"\n      \n      conn = taosws.connect(host=TAOS_HOST, port=TAOS_PORT, user=TAOS_USER, password=TAOS_PASSWORD)\n      ```\n\n   2. **Creating a table:**\n\n      ```python\n      sql = f\"\"\"\n          CREATE TABLE IF NOT EXISTS meters (\n              `ts` TIMESTAMP, \n              `current` FLOAT, \n              `voltage` INT, \n              `phase` FLOAT\n              ) TAGS (\n                  `groupid` INT, \n                  `location` BINARY(16)\n              )\n          \"\"\"\n   \n      conn.execute(sql)\n      ```\n\n   3. **Inserting data:**\n\n      The `gen_data()` method generates a series of random stock trades on every call:\n   \n      ```python\n      import random\n      from datetime import datetime, date, timezone as tz\n   \n      def gen_data():\n          modifier = random.random() * random.randint(1, 50)\n          return [{\n              \"ts\": datetime.now(),\n              \"current\": random.uniform(0, 75) + random.randint(0, 9) * modifier,\n              \"voltage\": random.randint(200, 225),\n              \"phase\": random.uniform(0, 105) + random.randint(1, 3) * modifier,\n          } for _ in range(NUM_ROWS_PER_INTERVAL)]\n      ```\n   \n      The `insert_data()` method uses prepared statements and batch inserts to enhance performance. By generating a batch of records at a time and using a prepared SQL statement, the method minimizes the overhead associated with multiple individual insert operations. This approach ensures efficient data insertion into the TDengine database.\n   \n      ```python\n      def insert_data(\n              conn, \n              progress_counter,\n              table_name: str = TAOS_TABLENAME\n              ) -\u003e None:\n          \"\"\"\n          Insert data into the TDengine table\n          \"\"\"\n          records = gen_data()\n       \n          # prepare a parameterized SQL statement\n          sql = f\"INSERT INTO ? USING `{table_name}` (groupid, location) TAGS(?,?) VALUES (?,?,?,?)\"\n          stmt = conn.statement()\n          stmt.prepare(sql)\n          tableNo = progress_counter % 10\n          tbname = f\"d_bind_{tableNo}\"\n          tags = [\n              taosws.int_to_tag(tableNo),\n              taosws.varchar_to_tag(SECURITIES[tableNo]),\n          ]\n   \n          stmt.set_tbname_tags(tbname, tags)\n       \n          # prepare the columns into their respective lists\n          timestamps = [int(record['ts'].timestamp() * 1000) for record in records]\n          currents = [record['current'] for record in records]\n          voltages = [record['voltage'] for record in records]\n          phases = [record['phase'] for record in records]\n   \n          # bind the parameters and execute the statement\n          stmt.bind_param([\n              taosws.millis_timestamps_to_column(timestamps),\n              taosws.floats_to_column(currents),\n              taosws.ints_to_column(voltages),\n              taosws.floats_to_column(phases),\n              ]\n          )\n          # send the batch for insert\n          stmt.add_batch()\n          stmt.execute()\n          logger.debug(f\"TDengine - Wrote {len(records)} rows to table {table_name}\")\n      ```\n\n### 4.2 Read TDengine Time-Series Data in Real-Time\n\nThe `perspective_server.py` script starts a Perspective server that reads data from TDengine and streams it to a Perspective Table via a Tornado WebSocket.\n\n   1. **Connecting to TDengine:**\n\n      ```python\n      import taosws\n      \n      TAOS_HOST = \"localhost\"\n      TAOS_PORT = 6041\n      TAOS_USER = \"root\"\n      TAOS_PASSWORD = \"taosdata\"\n      \n      conn = taosws.connect(host=TAOS_HOST, port=TAOS_PORT, user=TAOS_USER, password=TAOS_PASSWORD)\n      ```\n\n   2. **Reading data from TDengine:**\n\n      The `read_tdengine()` function queries the TDengine database and retrieves the latest stock data:\n   \n      ```python\n      def read_tdengine(conn):\n              conn: taosws.Connection, \n              ) -\u003e list[dict]:\n          try:\n              # query the database\n              sql = f\"\"\"\n                  SELECT `ts`, location, groupid, current, voltage, phase\n                  FROM {TAOS_TABLENAME}\n                  WHERE `ts` \u003e= NOW() - 12h\n                  ORDER BY `ts` DESC\n                  LIMIT 1000\n                  \"\"\"\n              logger.debug(f\"Executing query: {sql}\")\n              res = conn.query(sql)\n              data = [\n                  {\n                      \"timestamp\": convert_ts(row[0]),\n                      \"location\": row[1],\n                      \"groupid\": row[2],\n                      \"current\": row[3],\n                      \"voltage\": row[4],\n                      \"phase\": row[5],\n                  }\n                  for row in res\n              ]\n              logger.info(f\"select result: {data}\")\n              return data\n          except Exception as err:\n              logger.error(f\"Failed to query tdengine: {err}\")\n              raise err\n   \n      ```\n\n   3. **Updating Perspective Table:**\n\n      The `perspective_thread()` function creates a Perspective table and updates it with new data from TDengine every 300 milliseconds:\n   \n      ```python\n      def perspective_thread(perspective_server, tdengine_conn):\n          \"\"\"\n          Create a new Perspective table and update it with new data every 50ms\n          \"\"\"\n          # create a new Perspective table\n          client = perspective_server.new_local_client()\n          schema = {\n              \"timestamp\": datetime,\n              \"location\": str,\n              \"groupid\": int,\n              \"current\": float,\n              \"voltage\": int,\n              \"phase\": float,\n          }\n          # define the table schema\n          table = client.table(\n              schema,\n              limit=1000,                     # maximum number of rows in the table\n              name=PERSPECTIVE_TABLE_NAME,    # table name. Use this with perspective-viewer on the client side\n          )\n          logger.info(\"Created new Perspective table\")\n   \n          # update with new data every 50ms\n          def updater():\n              data = read_tdengine(tdengine_conn)\n              table.update(data)\n              logger.debug(f\"Updated Perspective table: {len(data)} rows\")\n   \n          logger.info(f\"Starting tornado ioloop update loop every {PERSPECTIVE_REFRESH_RATE} milliseconds\")\n          # start the periodic callback to update the table data\n          callback = tornado.ioloop.PeriodicCallback(callback=updater, callback_time=PERSPECTIVE_REFRESH_RATE)\n          callback.start()\n      ```\n\n   4. **Starting Tornado WebSocket Server:**\n\n      The `make_app()` function sets up a Tornado application with a WebSocket handler to serve the Perspective table:\n   \n      ```python\n      def make_app(perspective_server):\n          return tornado.web.Application([\n              (\n                  r\"/websocket\",\n                  perspective.handlers.tornado.PerspectiveTornadoHandler,\n                  {\"perspective_server\": perspective_server},\n              ),\n          ])\n      ```\n\n   5. **Running the server:**\n\n      The `perspective-viewer` element is configured with the \"Pro Dark\" theme to match the dark background and provide a consistent visual appearance.\n   \n      ```python\n      if __name__ == \"__main__\":\n          perspective_server = perspective.Server()\n          tdengine_conn = create_tdengine_connection()\n          app = make_app(perspective_server)\n          app.listen(8080, address='0.0.0.0')\n       \n          loop = tornado.ioloop.IOLoop.current()\n          loop.call_later(0, perspective_thread, perspective_server, tdengine_conn)\n          loop.start()\n      ```\n\n### 4.3 Embed \u0026 Configure Perspective Viewer\n\nThe `prsp-viewer.html` file embeds a Perspective Table in an HTML page. It connects to the Perspective server via a WebSocket and displays the real-time data streamed from TDengine.\n\n   1. **HTML Component:**\n\n      The HTML file includes the necessary Perspective libraries and sets up a `\u003cperspective-viewer\u003e` element within a container. This custom HTML component, written in WebAssembly, provides easily embeddable and highly interactive real-time data visualization on top of TDengine data. The viewer is configured to connect to the Perspective server via WebSocket and load the `meters_values` table, allowing for dynamic data visualization.\n\n   2. **Styling:**\n\n      CSS styles are applied to ensure the viewer occupies the full viewport and has a dark background.\n\n   3. **JavaScript Initialization:**\n\n      A script is included to load the Perspective viewer and connect it to the Perspective server via WebSocket. The viewer is bound to the `meters_values` table on the server, allowing real-time data updates to be displayed.\n\n      ```html\n      \u003cscript type=\"module\"\u003e\n          import perspective from \"https://cdn.jsdelivr.net/npm/@finos/perspective@3.1.3/dist/cdn/perspective.js\";\n   \n          document.addEventListener(\"DOMContentLoaded\", function() {\n              async function load_viewer() {\n                  const table_name = \"meters_values\";\n                  const viewer = document.getElementById(\"prsp-viewer\");\n                  const websocket = await perspective.websocket(\"ws://localhost:8080/websocket\");\n                  const server_table = await websocket.open_table(table_name);\n                  await viewer.load(server_table);\n              }\n              load_viewer();\n          });\n      \u003c/script\u003e\n      ```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaosdata%2Fperspective-connect-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftaosdata%2Fperspective-connect-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaosdata%2Fperspective-connect-demo/lists"}