{"id":13425881,"url":"https://github.com/uwdata/arquero","last_synced_at":"2025-05-13T22:02:38.949Z","repository":{"id":38444421,"uuid":"291996157","full_name":"uwdata/arquero","owner":"uwdata","description":"Query processing and transformation of array-backed data tables.","archived":false,"fork":false,"pushed_at":"2025-02-17T03:54:48.000Z","size":1467,"stargazers_count":1390,"open_issues_count":40,"forks_count":67,"subscribers_count":26,"default_branch":"main","last_synced_at":"2025-04-29T13:12:44.104Z","etag":null,"topics":["arrays","data","database","dataframe","query","table","transform"],"latest_commit_sha":null,"homepage":"https://idl.uw.edu/arquero/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uwdata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-01T12:48:01.000Z","updated_at":"2025-04-27T04:17:26.000Z","dependencies_parsed_at":"2024-01-06T14:05:22.097Z","dependency_job_id":"6d2a3014-ff2a-4034-ac51-59f3fbb20ee8","html_url":"https://github.com/uwdata/arquero","commit_stats":{"total_commits":443,"total_committers":17,"mean_commits":"26.058823529411764","dds":"0.23702031602708806","last_synced_commit":"000ef3c1c46a405d78cab5006359acbd800b1f58"},"previous_names":[],"tags_count":54,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uwdata%2Farquero","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uwdata%2Farquero/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uwdata%2Farquero/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uwdata%2Farquero/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uwdata","download_url":"https://codeload.github.com/uwdata/arquero/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254036807,"owners_count":22003652,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arrays","data","database","dataframe","query","table","transform"],"created_at":"2024-07-31T00:01:21.146Z","updated_at":"2025-05-13T22:02:38.878Z","avatar_url":"https://github.com/uwdata.png","language":"JavaScript","readme":"# Arquero \u003ca href=\"https://idl.uw.edu/arquero\"\u003e\u003cimg align=\"right\" src=\"https://raw.githubusercontent.com/uwdata/arquero/main/docs/assets/logo.svg\" height=\"38\"\u003e\u003c/img\u003e\u003c/a\u003e\n\n**Arquero** is a JavaScript library for query processing and transformation of array-backed data tables. Following the [relational algebra](https://en.wikipedia.org/wiki/Relational_algebra) and inspired by the design of [dplyr](https://dplyr.tidyverse.org/), Arquero provides a fluent API for manipulating column-oriented data frames. Arquero supports a range of data transformation tasks, including filter, sample, aggregation, window, join, and reshaping operations.\n\n* Fast: process data tables with million+ rows.\n* Flexible: query over arrays, typed arrays, array-like objects, or [Apache Arrow](https://arrow.apache.org/) columns.\n* Full-Featured: perform a variety of wrangling and analysis tasks.\n* Extensible: add new column types or functions, including aggregate \u0026amp; window operations.\n* Lightweight: small size, minimal dependencies.\n\nTo get up and running, start with the [Introducing Arquero](https://observablehq.com/@uwdata/introducing-arquero) tutorial, part of the [Arquero notebook collection](https://observablehq.com/collection/@uwdata/arquero).\n\nHave a question or need help? Visit the [Arquero GitHub repo](https://github.com/uwdata/arquero) or post to the [Arquero GitHub Discussions board](https://github.com/uwdata/arquero/discussions).\n\nArquero is Spanish for \"archer\": if datasets are [arrows](https://arrow.apache.org/), Arquero helps their aim stay true. 🏹 Arquero also refers to a goalkeeper: safeguard your data from analytic \"own goals\"! 🥅 ✋ ⚽\n\n## API Documentation\n\n* [Top-Level API](https://idl.uw.edu/arquero/api) - All methods in the top-level Arquero namespace.\n* [Table](https://idl.uw.edu/arquero/api/table) - Table access and output methods.\n* [Verbs](https://idl.uw.edu/arquero/api/verbs) - Table transformation verbs.\n* [Op Functions](https://idl.uw.edu/arquero/api/op) - All functions, including aggregate and window functions.\n* [Expressions](https://idl.uw.edu/arquero/api/expressions) - Parsing and generation of table expressions.\n* [Extensibility](https://idl.uw.edu/arquero/api/extensibility) - Extend Arquero with new expression functions or table verbs.\n\n## Example\n\nThe core abstractions in Arquero are *data tables*, which model each column as an array of values, and *verbs* that transform data and return new tables. Verbs are table methods, allowing method chaining for multi-step transformations. Though each table is unique, many verbs reuse the underlying columns to limit duplication.\n\n```js\nimport { all, desc, op, table } from 'arquero';\n\n// Average hours of sunshine per month, from https://usclimatedata.com/.\nconst dt = table({\n  'Seattle': [69,108,178,207,253,268,312,281,221,142,72,52],\n  'Chicago': [135,136,187,215,281,311,318,283,226,193,113,106],\n  'San Francisco': [165,182,251,281,314,330,300,272,267,243,189,156]\n});\n\n// Sorted differences between Seattle and Chicago.\n// Table expressions use arrow function syntax.\ndt.derive({\n    month: d =\u003e op.row_number(),\n    diff:  d =\u003e d.Seattle - d.Chicago\n  })\n  .select('month', 'diff')\n  .orderby(desc('diff'))\n  .print();\n\n// Is Seattle more correlated with San Francisco or Chicago?\n// Operations accept column name strings outside a function context.\ndt.rollup({\n    corr_sf:  op.corr('Seattle', 'San Francisco'),\n    corr_chi: op.corr('Seattle', 'Chicago')\n  })\n  .print();\n\n// Aggregate statistics per city, as output objects.\n// Reshape (fold) the data to a two column layout: city, sun.\ndt.fold(all(), { as: ['city', 'sun'] })\n  .groupby('city')\n  .rollup({\n    min:  d =\u003e op.min(d.sun), // functional form of op.min('sun')\n    max:  d =\u003e op.max(d.sun),\n    avg:  d =\u003e op.average(d.sun),\n    med:  d =\u003e op.median(d.sun),\n    // functional forms permit flexible table expressions\n    skew: ({sun: s}) =\u003e (op.mean(s) - op.median(s)) / op.stdev(s) || 0\n  })\n  .objects()\n```\n\n## Usage\n\n### In Browser\n\nTo use in the browser, you can load Arquero from a content delivery network:\n\n```html\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/arquero@latest\"\u003e\u003c/script\u003e\n```\n\nArquero will be imported into the `aq` global object. The default browser bundle does not include the [Apache Arrow](https://arrow.apache.org/) library. To perform Arrow encoding using [toArrow()](https://idl.uw.edu/arquero/api/#toArrow) or binary file loading using [loadArrow()](https://idl.uw.edu/arquero/api/#loadArrow), import Apache Arrow first:\n\n```html\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/apache-arrow@latest\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdn.jsdelivr.net/npm/arquero@latest\"\u003e\u003c/script\u003e\n```\n\nAlternatively, you can build and import `arquero.min.js` from the `dist` directory, or build your own application bundle. When building custom application bundles for the browser, the module bundler should draw from the `browser` property of Arquero's `package.json` file. For example, if using [rollup](https://rollupjs.org/), pass the `browser: true` option to the [node-resolve](https://github.com/rollup/plugins/tree/master/packages/node-resolve) plugin.\n\nArquero uses modern JavaScript features, and so will not work with some outdated browsers. To use Arquero with older browsers including Internet Explorer, set up your project with a transpiler such as [Babel](https://babeljs.io/).\n\n### In Node.js or Application Bundles\n\nFirst install `arquero` as a dependency, for example via `npm install arquero --save`.\nArquero assumes Node version 18 or higher.\nAs of Arquero version 6, the library uses type `module` and should be loaded using ES module syntax.\n\nImport using ES module syntax, import all exports into a single object:\n\n```js\nimport * as aq from 'arquero';\n```\n\nImport using ES module syntax, with targeted imports:\n\n```js\nimport { op, table } from 'arquero';\n```\n\nDynamic import (e.g., within a Node.js REPL):\n\n```js\naq = await import('arquero');\n```\n\n## Build Instructions\n\nTo build and develop Arquero locally:\n\n- Clone [https://github.com/uwdata/arquero](https://github.com/uwdata/arquero).\n- Run `npm i` to install dependencies.\n- Run `npm test` to run test cases, `npm run perf` to run performance benchmarks, and `npm run build` to build output files.\n","funding_links":[],"categories":["JavaScript","JavaScript Libraries","database","Libraries"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuwdata%2Farquero","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuwdata%2Farquero","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuwdata%2Farquero/lists"}