{"id":16389598,"url":"https://github.com/yuce/js-pilosa","last_synced_at":"2025-06-29T07:37:24.107Z","repository":{"id":57324157,"uuid":"92607224","full_name":"yuce/js-pilosa","owner":"yuce","description":"Javascript/Typescript client for Pilosa distributed bitmap index","archived":false,"fork":false,"pushed_at":"2017-06-10T17:30:24.000Z","size":143,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-09T07:20:35.907Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yuce.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-05-27T15:29:30.000Z","updated_at":"2018-09-25T02:33:40.000Z","dependencies_parsed_at":"2022-09-21T00:53:08.383Z","dependency_job_id":null,"html_url":"https://github.com/yuce/js-pilosa","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/yuce/js-pilosa","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuce%2Fjs-pilosa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuce%2Fjs-pilosa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuce%2Fjs-pilosa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuce%2Fjs-pilosa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yuce","download_url":"https://codeload.github.com/yuce/js-pilosa/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuce%2Fjs-pilosa/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262558435,"owners_count":23328533,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T04:33:44.506Z","updated_at":"2025-06-29T07:37:24.090Z","avatar_url":"https://github.com/yuce.png","language":"JavaScript","funding_links":[],"categories":["Software"],"sub_categories":["Client"],"readme":"# Javascript/Typescript Client for Pilosa\n\n\u003ca href=\"https://github.com/pilosa\"\u003e\u003cimg src=\"https://img.shields.io/badge/pilosa-v0.4.0-blue.svg\"\u003e\u003c/a\u003e\n\u003ca href=\"https://www.npmjs.com/package/pilosa\"\u003e\u003cimg src=\"https://img.shields.io/npm/v/pilosa.svg\"\u003e\u003c/a\u003e\n\u003ca href=\"https://travis-ci.org/yuce/js-pilosa\"\u003e\u003cimg src=\"https://api.travis-ci.org/yuce/js-pilosa.svg?branch=master\"\u003e\u003c/a\u003e\n\u003ca href=\"https://coveralls.io/github/yuce/js-pilosa?branch=master\"\u003e\u003cimg src=\"https://coveralls.io/repos/github/yuce/js-pilosa/badge.svg?branch=master\"\u003e\u003c/a\u003e\n\n\u003cimg src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/a/a7/Cute_Sloth.jpg/320px-Cute_Sloth.jpg\" style=\"float: right\" align=\"right\" height=\"180\"\u003e\n\nJavascript/Typescript client for [Pilosa](https://www.pilosa.com/) high performance distributed bitmap index.\n\n## Change Log\n\n* **v0.4.0** (2017-06-10):\n    * Supports Pilosa Server v0.4.0.\n    * *Breaking Change*: Changed default row ID label to `rowID` and default column ID to `column ID`.\n    * Updated the accepted values for index, frame names and labels to match with the Pilosa server.\n    * `Union` queries accept 0 or more arguments. `Intersect` and `Difference` queries accept 1 or more arguments.\n    * Added `inverse TopN` and `inverse Range` calls.\n    * Inverse enabled status of frames is not checked on the client side.\n    * `https` scheme is allowed.\n\n* **v0.3.3** (2017-05-28):\n    * Initial version.\n    * Supports Pilosa Server v0.3.2.\n\n## Requirements\n\n* NodeJS 4 or later\n* (Optional) Typescript 2.3 and higher\n\n## Install\n\nPilosa client is available as an [npm](https://www.npmjs.com/package/pilosa) package. You can install the library using:\n\n```\nnpm install --save pilosa\n```\n\n## Usage\n\n### Quick overview (Javascript/Typescript using promises)\n\nAssuming [Pilosa](https://github.com/pilosa/pilosa) server is running at `localhost:10101` (the default):\n\n```javascript\nvar pilosa = require(\"pilosa\");\n\n// Create the default client\nvar client = new pilosa.Client();\n\n// Create an Index object\nvar myindex = new pilosa.Index(\"myindex\");\n\n// Create a Frame object\nvar myframe = myindex.frame(\"myframe\");\n\n// Make sure the index exists on the server\nclient.ensureIndex(myindex).then(() =\u003e\n\n// Make sure the frame exists on the server\nclient.ensureFrame(myframe)).then(() =\u003e\n\n// Send a SetBit query\nclient.query(myframe.setBit(5, 42))).then(_ =\u003e {\n    // Send a Bitmap query\n    client.query(myframe.bitmap(5)).then(response =\u003e {\n        if (response.result) {\n            var bits = response.result.bitmap.bits;\n            console.log(\"Got bits: \", bits);\n        }\n    }).catch(err =\u003e console.log(\"ERROR: \", err));\n\n    // You can batch queries to improve throughput\n    client.query(\n        myindex.batchQuery(\n            myframe.bitmap(5),\n            myframe.bitmap(19)\n        )\n    ).then(response =\u003e {\n        response.results.forEach(result =\u003e {\n            console.log(\"Got bits: \", result.bitmap.bits);\n        });\n    }).catch(err =\u003e console.log(\"ERROR: \", err));\n}).catch(err =\u003e console.log(\"ERROR: \", err));\n\n```\n\n### Quick overview (Typescript using async/await)\n\n```typescript\nimport * as pilosa from \"pilosa\";\n\nasync function main() {\n    // Create the default client\n    var client = new pilosa.Client();\n\n    // Create an Index object\n    var myindex = new pilosa.Index(\"myindex\");\n\n    // Create a Frame object\n    var myframe = myindex.frame(\"myframe\");\n\n    // Make sure the index exists on the server\n    await client.ensureIndex(myindex);\n\n    // Make sure the frame exists on the server\n    await client.ensureFrame(myframe);\n\n    // Send a SetBit query\n    await client.query(myframe.setBit(5, 42));\n\n    // Send a Bitmap query\n    var response = await client.query(myframe.bitmap(5));\n    if (response.result) {\n        var bits = response.result.bitmap.bits;\n        console.log(\"Got bits: \", bits);\n    }\n\n    // You can batch queries to improve throughput\n    response = await client.query(\n        myindex.batchQuery(\n            myframe.bitmap(5),\n            myframe.bitmap(19)));\n\n    response.results.forEach(result =\u003e {\n        console.log(\"Got bits: \", result.bitmap.bits);\n    });\n}\n\nmain().catch(err =\u003e console.log(\"ERROR: \", err));\n```\n\n### Data Model and Queries\n\n#### Indexes and Frames\n\n*Index* and *frame*s are the main data models of Pilosa. You can check the [Pilosa documentation](https://www.pilosa.com/docs) for more detail about the data model.\n\n`Index` constructor is used to create an index object. Note that this does not create an index on the server; the index object simply defines the schema.\n\n```javascript\nvar repository = new pilosa.Index(\"repository\")\n```\n\nIndexes support changing the column label and time quantum. `IndexOptions` objects store that kind of data. In order to apply these custom options, pass an `IndexOptions` object as the second argument to `Index`:\n\n```javascript\nvar options = {\n    columnLabel: \"repo_id\",\n    timeQuantum: pilosa.TimeQuantum.YEAR_MONTH\n}\nvar repository = pilosa.Index(\"repository\", options);\n```\n\nFrames are created with a call to `index.frame` method:\n\n```javascript\nvar stargazer = repository.frame(\"stargazer\");\n```\n\nSimilar to index objects, you can pass custom options to the `index.frame` method:\n\n```javascript\nvar options = {\n    rowLabel: \"stargazer_id\",\n    timeQuantum: pilosa.TimeQuantum.YEAR_MONTH_DAY\n}\nvar stargazer = repository.frame(\"stargazer\", options);\n```\n\n#### Queries\n\nOnce you have indexes and frame objects created, you can create queries for them. Some of the queries work on the columns; corresponding methods are attached to the index. Other queries work on rows, with related methods attached to frames.\n\nFor instance, `Bitmap` queries work on rows; use a frame object to create those queries:\n\n```javascript\nvar bitmapQuery = stargazer.bitmap(1, 100);  // corresponds to PQL: Bitmap(frame='stargazer', stargazer_id=1)\n```\n\n`Union` queries work on columns; use the index object to create them:\n\n```javascript\nvar query = repository.union(bitmapQuery1, bitmapQuery2);\n```\n\nIn order to increase througput, you may want to batch queries sent to the Pilosa server. The `index.batchQuery` method is used for that purpose:\n\n```javascript\nvar query = repository.batchQuery(\n    stargazer.bitmap(1, 100),\n    repository.union(stargazer.bitmap(100, 200), stargazer.bitmap(5, 100))\n);\n```\n\nThe recommended way of creating query objects is, using dedicated methods attached to index and frame objects. But sometimes it would be desirable to send raw queries to Pilosa. You can use the `index.rawQuery` method for that. Note that, query string is not validated before sending to the server:\n\n```javascript\nvar query = repository.rawQuery(\"Bitmap(frame='stargazer', stargazer_id=5)\");\n```\n\nCheck [Pilosa documentation](https://www.pilosa.com/docs) for PQL details. Here is a list of methods corresponding to PQL calls:\n\nIndex:\n\n* `union(...bitmaps: Array\u003cPqlBitmapQuery\u003e): PqlBitmapQuery`\n* `intersect(...bitmaps: Array\u003cPqlBitmapQuery\u003e): PqlBitmapQuery`\n* `difference(...bitmaps: Array\u003cPqlBitmapQuery\u003e): PqlBitmapQuery`\n* `count(bitmap: PqlBitmapQuery): PqlQuery`\n* `setColumnAttrs(columnID: number, attrs: AttributeMap): PqlBitmapQuery`\n\nFrame:\n\n* `bitmap(rowID: number): PqlBitmapQuery`\n* `inverseBitmap(columnID: number): PqlQuery`\n* `setBit(rowID: number, columnID: number, timestamp?: Date): PqlQuery`\n* `clearBit(rowID: number, columnID: number): PqlQuery`\n* `topN(n: number, bitmap?: PqlBitmapQuery, field?: string, ...values: Array\u003cany\u003e): PqlBitmapQuery`\n* `inverseTopN(n: number, bitmap?: PqlBitmapQuery, field?: string, ...values: Array\u003cany\u003e): PqlBitmapQuery`\n* `range(rowID: number, start: Date, end: Date): PqlBitmapQuery`\n* `inverseRange(columnID: number, start: Date, end: Date): PqlBitmapQuery`\n* `setRowAttrs(rowID: number, attrs: AttributeMap): PqlBitmapQuery`\n\n### Pilosa URI\n\nA Pilosa URI has the `${SCHEME}://${HOST}:${PORT}` format:\n* **Scheme**: Protocol of the URI. Default: `http`.\n* **Host**: Hostname or ipv4/ipv6 IP address. Default: localhost.\n* **Port**: Port number. Default: `10101`.\n\nAll parts of the URI are optional, but at least one of them must be specified. The following are equivalent:\n\n* `http://localhost:10101`\n* `http://localhost`\n* `http://:10101`\n* `localhost:10101`\n* `localhost`\n* `:10101`\n\nA Pilosa URI is represented by the `pilosa.URI` class. Below are a few ways to create `URI` objects:\n\n```javascript\n// create the default URI: http://localhost:10101\nvar uri1 = new pilosa.URI()\n\n// create a URI from string address\nvar uri2 = pilosa.URI.address(\"db1.pilosa.com:20202\")\n\n// create a URI with the given host and port\nvar URI uri3 = new pilosa.URI(host=\"db1.pilosa.com\", port=20202);\n``` \n\n### Pilosa Client\n\nIn order to interact with a Pilosa server, an instance of `pilosa.Client` should be created. We recommend creating a single instance of the client and share it with other objects when necessary.\n\nIf the Pilosa server is running at the default address (`http://localhost:10101`) you can create the default client with default options using:\n\n```javascript\nvar client = new pilosa.Client()\n```\n\nTo use a a custom server address, pass the address in the first argument:\n\n```javascript\nvar client = new pilosa.Client(\"http://db1.pilosa.com:15000\")\n```\n\nIf you are running a cluster of Pilosa servers, you can create a `pilosa.Cluster` object that keeps addresses of those servers:\n\n```javascript\nvar cluster = new pilosa.Cluster(\n    pilosa.URI.address(\":10101\"),\n    pilosa.URI.address(\":10110\"),\n    pilosa.URI.address(\":10111\"),\n);\n\n// Create a client with the cluster\nvar client = new pilosa.Client(cluster)\n```\n\nOnce you create a client, you can create indexes, frames and start sending queries. All client methods return a `Promise` object.\n\nHere is how you would create an index and frame:\n\n```javascript\n// materialize repository index instance initialized before\nclient.createIndex(repository).then(() =\u003e\n// materialize stargazer frame instance initialized before\nclient.create_frame(stargazer).then(() =\u003e {\n    // actions on the frame.\n})).catch(err =\u003e {\n    // act on the error\n});\n```\n\nUsing async/await syntax is obviously preferable in case you are using Javascript with NodeJS 7 and higher or Typescript:\n\n```javascript\ntry {\n    // materialize repository index instance initialized before\n    await client.createIndex(repository);\n    // materialize stargazer frame instance initialized before\n    await client.create_frame(stargazer);\n\n    // actions on the frame...\n}\ncatch (e)  {\n    // act on the error\n}\n```\n\nIf the index or frame exists on the server, you will receive a `PilosaError`. You can use `ensureIndex` and `ensureFrame` methods to ignore existing indexes and frames.\n\nYou can send queries to a Pilosa server using the `query` method of client objects.\n\nUsing promises:\n\n```javascript\nclient.query(frame.bitmap(5)).then(response =\u003e {\n    // act on the response\n})\n```\n\nUsing async/await:\n```javascript\nvar response = await client.query(frame.bitmap(5));\n// act on the response\n```\n\n`query` method accepts an optional argument of type `QueryOptions`:\n\n```javascript\nvar queryOptions = {\n    columns: true  // return column data in the response\n}\nclient.query(frame.bitmap(5), queryOptions).then(response =\u003e {\n    // act on the response\n});\n```\n\n### Server Response\n\nWhen a query is sent to a Pilosa server, the server either fulfills the query or sends an error message. In the case of an error, `PilosaError` is returned, otherwise a `QueryResponse` object is returned.\n\nA `QueryResponse` object may contain zero or more results of `QueryResult` type. You can access all results using the `results` property of `QueryResponse` (which returns a list of `QueryResult` objects) or you can use the `result` property (which returns either the first result or `null` if there are no results):\n\n```javascript\nclient.query(frame.bitmap(5)).then(response =\u003e {\n    // check that there's a result and act on it\n    var result = response.result\n    if result {\n        // act on the result\n    }\n\n    // iterate on all results\n    response.results.forEach(result =\u003e {\n        // act on the result\n    });\n}).catch(err =\u003e {\n    // act on the error\n});\n```\n\nSimilarly, a `QueryResponse` object may include a number of column objects, if `columns=true` query option was used:\n\n```javascript\n// check that there's a column object and act on it\nvar column = response.column\nif column {\n    // act on the column\n}\n    \n// iterate on all columns\nresponse.columns.forEach(column =\u003e {\n    // act on the column\n})\n```\n\n`QueryResult` objects contain:\n\n* `bitmap` property to retrieve a bitmap result,\n* `countItems` property to retrieve column count per row ID entries returned from `topN` queries,\n* `count` property to retrieve the number of rows per the given row ID returned from `count` queries.\n\n```javascript\nvar bitmap = response.bitmap\nvar bits = bitmap.bits\nvar attributes = bitmap.attributes\n\nvar countItems = response.countItems\n\nvar count = response.count\n```\n\n## Contribution\n\n1. Fork this repo and add it as upstream: `git remote add upstream git@github.com:yuce/js-pilosa.git`.\n2. Make sure all tests pass (use `make test-all`) and be sure that the tests cover all statements in your code (we aim for 100% test coverage).\n3. Commit your code to a feature branch and send a pull request to the `master` branch of our repo.\n\nThe sections below assume your platform has `make`. Otherwise you can view the corresponding steps of the `Makefile`.\n\n### Running tests\n\nYou can run unit tests with:\n```\nmake test\n```\n\nAnd both unit and integration tests with:\n```\nmake test-all\n```\n\n### Generating protobuf classes\n\nProtobuf classes are already checked in to source control, so this step is only needed when the upstream `public.proto` changes.\n\nBefore running the following step, make sure you have the [Protobuf compiler](https://github.com/google/protobuf) installed:\n\n```\nmake generate\n```\n\n## License\n\n```\nCopyright 2017 Yuce Tekol\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions\nare met:\n\n1. Redistributions of source code must retain the above copyright\nnotice, this list of conditions and the following disclaimer.\n\n2. Redistributions in binary form must reproduce the above copyright\nnotice, this list of conditions and the following disclaimer in the\ndocumentation and/or other materials provided with the distribution.\n\n3. Neither the name of the copyright holder nor the names of its\ncontributors may be used to endorse or promote products derived\nfrom this software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND\nCONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES,\nINCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF\nMERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\nDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR\nCONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,\nSPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,\nBUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\nSERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS\nINTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,\nWHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING\nNEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\nOF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH\nDAMAGE.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuce%2Fjs-pilosa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyuce%2Fjs-pilosa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuce%2Fjs-pilosa/lists"}