{"id":13692096,"url":"https://github.com/jpkli/p4","last_synced_at":"2026-02-18T01:11:25.961Z","repository":{"id":57317528,"uuid":"96662595","full_name":"jpkli/p4","owner":"jpkli","description":"P4: Portable Parallel Processing Pipeline","archived":false,"fork":false,"pushed_at":"2021-01-10T05:16:54.000Z","size":3448,"stargazers_count":40,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-10-19T15:35:58.178Z","etag":null,"topics":["data-processing","gpu","visualizations"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jpkli.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-09T05:11:12.000Z","updated_at":"2024-08-28T02:55:11.000Z","dependencies_parsed_at":"2022-08-25T20:40:25.693Z","dependency_job_id":null,"html_url":"https://github.com/jpkli/p4","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/jpkli/p4","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jpkli%2Fp4","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jpkli%2Fp4/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jpkli%2Fp4/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jpkli%2Fp4/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jpkli","download_url":"https://codeload.github.com/jpkli/p4/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jpkli%2Fp4/sbom","scorecard":{"id":535771,"data":{"date":"2025-08-11","repo":{"name":"github.com/jpkli/p4","commit":"06b704134e1a96780a36133fe7e4ceca143584a2"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Code-Review","score":0,"reason":"Found 2/28 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: GNU General Public License v3.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 4 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-20T06:55:13.821Z","repository_id":57317528,"created_at":"2025-08-20T06:55:13.821Z","updated_at":"2025-08-20T06:55:13.821Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29565024,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T00:47:08.760Z","status":"ssl_error","status_checked_at":"2026-02-18T00:45:26.718Z","response_time":100,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-processing","gpu","visualizations"],"created_at":"2024-08-02T17:00:53.628Z","updated_at":"2026-02-18T01:11:25.945Z","avatar_url":"https://github.com/jpkli.png","language":"JavaScript","readme":"# P4: Portable Parallel Processing Pipelines\n\nP4 is JavaScript library for accelerating data processing and visualization using the GPU. P4 provides an intuitive and declarative API for specifying common data transformations and visualizations, which automatically compile to WebGL shader programs for parallel computing.\n\nFor data processing, P4 is more than 10X faster than codes based on JavaScript Array functions. For visualizing large data, P4 is at least 10X faster than Canvas, and 20X faster than SVG.\n\n\u003c!-- ### Table of Contents\n- [P4: Portable Parallel Processing Pipelines](#p4-portable-parallel-processing-pipelines)\n    - [Installation](#installation)\n  - [Example](#example)\n    - [Data Transformation](#data-transformation)\n    - [Derive](#derive)\n    - [Match](#match)\n    - [Aggregate](#aggregate)\n    - [Visualization](#visualization)\n      - [Visual Channels](#visual-channels)\n        - [Example](#example-1)\n      - [View Composition](#view-composition)\n    - [Custom View Arrangement](#custom-view-arrangement)\n      - [Example](#example-2)\n    - [Current Limitations and Known Issues](#current-limitations-and-known-issues)\n    - [Reference Paper](#reference-paper)\n--\u003e\n\n### Installation\n\nInstall using npm\n```bash\nnpm install p4.js\n```\n\nor include the following line in your html:\n```html\n\u003cscript src=\"https://github.com/jpkli/p4/dist/p4.js\"\u003e\u003c/script\u003e\n```\n\n## Example\n\n```javascript\np4({container: 'body', viewport:[800, 600]})\n.data({\n  format: 'json',\n  values: [{BabyWeight: 9, Gender: 'Girl', MotherAge: 28, FatherAge: 32}, ...]\n})\n.view([{width: 800, height: 600}])\n.derive({ AgeDiff: 'abs(FatherAge - MotherAge)' })\n.match({ AgeDiff: [0, 10] })\n.aggregate({\n  $group: 'AgeDiff',\n  $collect: {\n    Babies: {$count: '*'},\n    AvgWeight: {$avg: 'BabyWeight'}\n  }\n})\n.visualize({\n  mark: 'bar',\n  x: 'AgeDiff',\n  height: 'Babies',\n  color: 'AvgWeight'\n})\n```\nThe above codes process a dataset with 100K records and visualize the result as a bar chart shown below.\n\n\u003cimg width=300 src=\"https://jpkli.github.io/demos/p4/images/colorbars.png\"\u003e\n\n### Data Transformation\n\n### Derive\n*Derive* is for calculating new attributes based on existing data attributes.\n\n```javascript\nderive({\n  NewAttribute1: 'expression with existing attributes and math functions',\n  NewAttribute2: 'expression2',\n  ...\n})\n```\nSupported math functions: abs, ceil, cos, exp, log, log2, max, min, pow, round, sin, sqrt, tan, acos, asin, atan.\nAt default, 4 new numerical attributes can be derived (for saving GPU memory space). To add more, specify it in the config of the pipeline:\n\n```javascript\np4.config({deriveMax: 8}).derive({...}) //now it can derive 8 new attributes.\n```\n\n### Match\n*Match* can be used to filer the data based on multiple data attributes, including both numeric and categorical.\n\n```javascript\nmatch({\n  NumericAttribute1: [start, end],\n  NumericAttribute2: {$in: [number1, number2, ...]},\n  CategoricalAttribute1: {$in: [string1, string2, ...]}\n  ...\n})\n```\n\n### Aggregate\n*Aggregate* is used to group or bin data based on attributes to obtain specified metrics.\n```javascript\naggregate({\n  $group: [attr1, attr2], // up to 3 attributes (current limitation),\n  $collect: {\n    newAttr1Name: {$opt: 'attr3'}, \n    newAttr2Name: {$opt: 'attr4'}, \n  }\n})\n```\nSupported \\$opt: count, sum, avg, min, max\n\n\n### Visualization\n\nP4 can effectively visualize the data stored or processed in GPU using different visual marks and plots.\n\n#### Visual Channels\n\n```javascript\nvisualize({id, mark, channels})\n```\n\n* id (optional)\n  id of the view for this visualization.\n\n* mark\n  A visual mark can be: circle, rect, bar, line.\n\n* channels (x, y, width, height, color, opacity)\n  A channel can be mapped to any original or derived data attributes for visualization. A numeric or string value can be assigned (e.g., {color: 'blue', opacity: 0.5})\n\n##### Example\n\n```javascript\np4(config)\n  .data({ ... })\n  .visualize({\n    mark: 'circle',\n    x: 'MotherWeight',\n    y: 'BabyWeight',\n    color: 'teal',\n    opacity: 0.5,\n    size: 8\n  })\n```\n\n#### View Composition\n\nTo generate more than one views, any array of visual encodings can be specified.\n\n```json\n{\n  \"$visualize\": [\n    {\n      \"mark\": \"bar\",\n      \"color\": \"steelblue\",\n      \"x\": \"MotherEdu\",\n      \"height\": \"Babies\"\n    },\n    {\n      \"mark\": \"line\",\n      \"color\": \"steelblue\",\n      \"x\": \"MotherEdu\",\n      \"y\": \"AvgWeight\",\n      \"size\": 5\n    }\n  ]\n}\n```\nThis will generate two bar charts in a column stack ([see example here](#/play/bar-charts2)).\n\n### Custom View Arrangement\n\nThe *view* function can be used to compose different view arrangements.\n\n```javascript\nview({id, width, height, padding, offset})\n```\n\n#### Example\n```javascript\np4(config).views([\n    {\n      \"id\": \"c1\",\n      \"width\": 360,\n      \"height\": 360,\n      \"padding\": {\"left\": 120, \"right\": 10, \"top\": 10, \"bottom\": 50},\n      \"offset\": [380, 0]\n    },\n    {\n      \"id\": \"c2\",\n      \"width\": 360,\n      \"height\": 360,\n      \"padding\": {\"left\": 120, \"right\": 10, \"top\": 10, \"bottom\": 50},\n      \"offset\": [0, 0]\n    }\n  ],\n);\n```\nSee this example in [Online Editor](#/play/brush-link).\n\n### Current Limitations and Known Issues\n - Only 24-bit, single floating point precision is supported.\n - Data size cannot be larger than the max size supported by WebGL texture, which is typically 8096 x 8096.\n\n### Publication\nP4 was first developed to leverage WebGL for GPU-accelerated data processing. The support for interactive visualizations is added based on the research work documented in the following paper:\n\nLi JK, Ma KL. [P4: Portable Parallel Processing Pipelines for Interactive Information Visualization](https://ieeexplore.ieee.org/abstract/document/8468065). IEEE transactions on visualization and computer graphics. 2018 Sep 19.\n\n## Acknowledgement\nThis research was sponsored in part by the U.S. National Science Foundation through grant NSF IIS-1528203 and U.S. Department of Energy through grant DE-SC0014917.\n","funding_links":[],"categories":["[↑](#contents) Declarative"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjpkli%2Fp4","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjpkli%2Fp4","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjpkli%2Fp4/lists"}