{"id":26897302,"url":"https://github.com/atomiechen/matsense","last_synced_at":"2025-05-12T23:44:02.492Z","repository":{"id":57440133,"uuid":"442115769","full_name":"atomiechen/MatSense","owner":"atomiechen","description":"A toolkit for matrix sensor data processing 阵列传感器数据处理工具库","archived":false,"fork":false,"pushed_at":"2024-05-27T15:52:53.000Z","size":559,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-12T23:44:00.422Z","etag":null,"topics":["matplotlib","numpy","pypi","python"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/MatSense/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/atomiechen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-12-27T09:41:02.000Z","updated_at":"2024-05-27T13:01:43.000Z","dependencies_parsed_at":"2022-09-26T17:20:57.662Z","dependency_job_id":"b99a0ca2-aeb9-47f8-91bf-4571be2fc675","html_url":"https://github.com/atomiechen/MatSense","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atomiechen%2FMatSense","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atomiechen%2FMatSense/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atomiechen%2FMatSense/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atomiechen%2FMatSense/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/atomiechen","download_url":"https://codeload.github.com/atomiechen/MatSense/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253843165,"owners_count":21972867,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["matplotlib","numpy","pypi","python"],"created_at":"2025-04-01T04:42:08.737Z","updated_at":"2025-05-12T23:44:02.433Z","avatar_url":"https://github.com/atomiechen.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MatSense\n\n[![PyPi](https://img.shields.io/pypi/v/matsense.svg)](https://pypi.org/project/MatSense/)\n\nA toolkit that supports both real-time and off-line matrix sensor data processing and 3D visualization. \n\n![schematic](https://raw.githubusercontent.com/atomiechen/MatSense/main/img/schematic.drawio.svg)\n\nA typical real-time data flow would be in a client-server manner:\n\n- Matrix sensor data: collected (e.g. by Arduino) and transmitted via a serial port to the computer.\n- Data processing: the series of matrix data frames are processed and served by the server.\n- Applications: clients connect to server to get processed data and do further work.\n\nData can also be recorded to and processed from files. \n\n\u003cimg src=\"https://raw.githubusercontent.com/atomiechen/MatSense/main/img/player.png\" alt=\"schematic\" width=\"450\" /\u003e\n\n3D visualization tools are provided to play real-time stream or recorded data.\n\n\n\n\n## Installation\n\nFrom PyPI:\n\n```sh\npip install MatSense\n```\n\nThis will install [Matplotlib](https://matplotlib.org/) to implement 3D visualization tools. \n\nIf you want to further try [PyQtGraph](https://www.pyqtgraph.org/) as visualization method:\n\n```sh\npip install MatSense[pyqtgraph]\n```\n\n\n\n## Usage\n\n### Off-the-shelf tools\n\n3 handy tools are provided. Pass `-h` to get detailed information.\n\n- `matserver` / `python -m matsense.server`\n  - functions:\n    - receive data from serial port, process and serve\n    - process data from file(s) and output to file\n    - other helpful functions\n  - supported processing methods:\n    - voltage-pressure conversion (optional for pressure data)\n    - spatial filter (in-frame denoising): none, ideal, butterworth, gaussian\n    - temporal filter (pixel-wise between-frame denoising): none, moving average, rectangular window\n    - calibration: static or dynamic\n- `matclient` / `python -m matsense.client`: receive server data, process and visualize; or control server via interactive commands\n  - supported processing methods:\n    - interpolation\n    - blob parsing\n- `matdata` / `python -m matsense.data`: visualize file data, or process off-line data\n\n### Configuration\n\nAll 3 tools can be totally configured by a YAML configuration file:\n\n```sh\n## server console\nmatserver --config your_config.yaml\n\n## client console\nmatclient --config your_config.yaml\n\n## off-line data processing\nmatdata --config your_config.yaml\n```\n\nPriority: commandline arguments \u003e config file \u003e program defaults.\n\nA template YAML configuration (unused options can be set to `~` or removed):\n\n```yaml\n## template configurations\n## ~ for defaults\n\n## configurations for matserver mode\nserver_mode:\n  ## enable backend service\n  service: ~\n  ## enable visualization or not (suppress service)\n  visualize: ~\n  ## enumerate all serial ports\n  enumerate: ~\n\n  ## (suppress serial) simulated data source without actual serial connection\n  ## debug mode: true, false\n  debug: ~\n\n  ## (suppress serial) use file as data source or not: true, false\n  use_file: ~\n\n## configurations for matclient mode\nclient_mode:\n  ## make client present raw data\n  raw: ~\n  ## interactive command line mode\n  interactive: ~\n\n## configurations for matdata mode\ndata_mode:\n  ## process file data instead of visualization\n  process: ~\n\n## configurations for file data\ndata:\n  ## input filename(s), filename or a list of filenames: [a.csv, b.csv, ...]\n  in_filenames: ~\n  ## output filename, default filename is used when not provided\n  out_filename: ~\n\n## configurations for matrix sensor\nsensor:\n  ## sensor shape: [16, 16], [8, 8], [6, 24]\n  shape: ~\n  ## total points, can be set to ~\n  total: ~\n  ## 0/1 mask to exclude non-existent points\n  ## |- for multiline without a newline in the end\n  mask: ~\n\n## configurations for serial port\nserial:\n  ## baudrate: 9600, 250000, 500000, 1000000\n  baudrate: ~\n  ## serial port timeout, in seconds\n  timeout: ~\n  ## serial port\n  port: ~\n  ## data transmission protocol: simple, secure\n  protocol: ~\n  ## support IMU data\n  imu: ~\n\n## configurations for client-server connections\nconnection:\n  ## use UDP or UNIX domain socket\n  udp: ~\n  ## udp address format: 127.0.0.1:20503\n  ## UNIX deomain socket address format: /var/tmp/unix.socket.server\n  server_address: ~\n  client_address: ~\n\n## configurations for data processing\nprocess:\n  ### voltage to the reciprocal of resistance\n  ## reference voltage: 255, 255/3.6*3.3\n  V0: ~\n  ## constant factor: 1\n  R0_RECI: ~\n  ## convert voltage to resistance: true\n  convert: ~\n\n  ### server data processing\n  ## no filtering and calibration\n  raw: ~\n  ## time of warming up in seconds: 1\n  warm_up: ~\n  ## spatial filter: none, ideal, butterworth, gaussian\n  filter_spatial: ~\n  ## spatial filter cut-off freq: 3.5\n  filter_spatial_cutoff: ~\n  ## Butterworth filter order: 2\n  butterworth_order: ~\n  ## temporal filter: none, moving average, rectangular window\n  filter_temporal: ~\n  ## temporal filter size: 15\n  filter_temporal_size: ~\n  ## rectangular window filter cut-off frequency: 0.04\n  rw_cutoff: ~\n  ## calibrative frames, 0 for no calibration: 0, 200\n  cali_frames: ~\n  ## calibration frame window size, 0 for static and \u003e0 for dynamic: 0, 10000\n  cali_win_size: ~\n  ## intermediate result: 0, 1, 2\n    ## 0: convert voltage to reciprocal resistance\n    ## 1: convert \u0026 spatial filter\n    ## 2: convert \u0026 spatial filter \u0026 temporal filter\n  intermediate: ~\n\n  ### (optional) client data processing\n  ## interpolation shape, default to sensor.shape\n  interp: ~\n  ## interpolation order: 3\n  interp_order: ~\n  ## filter out blobs: true\n  blob: ~\n  ## total blob number: 3\n  blob_num: ~\n  ## blob filter threshole: 0.1, 0.15\n  threshold: ~\n  ## special check for certain hardwares: false\n  special_check: ~\n\npointing:\n  ## value bound for checking cursor moving state: 0\n  bound: ~\n  ## directly map coordinates or relatively (suppress trackpoint)\n  direct_map: ~\n  ## use ThinkPad's TrackPoint (red dot) control style\n  trackpoint: ~\n  ## smoothing\n  alpha: ~\n\n## configurations for visualization\nvisual:\n  ## using pyqtgraph or matplotlib\n  pyqtgraph: ~\n  ## z-axis limit: 3, 5\n  zlim: ~\n  ## frame rate: 100\n  fps: ~\n  ## scatter plot: false\n  scatter: ~\n  ## show text value: false\n  show_value: ~\n```\n\n### Useful modules\n\n- `matsense.uclient`\n  - `Uclient`: interface to receive data from server\n- `matsense.process`: data processing tools\n  - `DataHandlerPressure`: process pressure data (conversion \u0026 filtering \u0026 calibration)\n  - `BlobParser`\n  - `Interpolator`\n  - `PointSmoother`\n  - `CursorController`\n  - `PressureSelector`\n- `matsense.datasetter`: data setter, using serial port or file data\n  - `DataSetterSerial`\n  - `DataSetterFile`\n\n- `matense.tools`: configuration and other helpful tools\n- `matsense.filemanager`: file I/O tools\n- `matsense.visual`: visualization tools\n  - `from matsense.visual.player_matplot import Player3DMatplot`: 3D player using Matplotlib\n  - `from matsense.visual.player_pyqtgraph import Player3DPyqtgraph`: 3D player using PyQtGraph\n\n\n\n\n## Server-Client Protocol\n\nUse `matclient -i` to control server. \n\nThe underlying server-client communication protocol is：\n\n| Name         | meaning                               | Value              | Format       | Return               | Return format  |\n| ------------ | ------------------------------------- | ------------------ | ------------ | -------------------- | -------------- |\n| CLOSE        | close server                          | 0                  | 1byte        | status               | 1byte          |\n| DATA         | get a data frame                      | 1                  | 1byte        | frame+index          | 256double+1int |\n| RAW          | get a raw data frame                  | 2                  | 1byte        | frame+index          | 256double+1int |\n| REC_DATA     | ask server to record data to file     | 3(+filename)       | 1byte+string | status+filename      | 1byte+string   |\n| REC_RAW      | ask server to record raw data to file | 4(+filename)       | 1byte+string | status+filename      | 1byte+string   |\n| REC_STOP     | ask server to stop recording          | 5                  | 1byte        | status               | 1byte          |\n| RESTART      | restart server with config string     | 6(+config_str)     | 1byte+string | status+config_string | 1byte+string   |\n| RESTART_FILE | restart server with config filename   | 10+config_filename | 1byte+string | status+config_string | 1byte+string   |\n| CONFIG       | get server config                     | 7                  | 1byte        | status+config_string | 1byte+string   |\n| DATA_IMU     | get IMU data                          | 9                  | 1byte        | IMU_frame+index      | 6double+1int   |\n\n- `status` (1 byte): 0 for success and 255 for failure\n\n\n\n## Author\n\nAtomie CHEN: atomic_cwh@163.com\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fatomiechen%2Fmatsense","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fatomiechen%2Fmatsense","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fatomiechen%2Fmatsense/lists"}