{"id":17059951,"url":"https://github.com/nisaacson/riak-streaming-node","last_synced_at":"2025-10-15T18:34:07.355Z","repository":{"id":11868157,"uuid":"14428066","full_name":"nisaacson/riak-streaming-node","owner":"nisaacson","description":"Streaming riak client for node.js","archived":false,"fork":false,"pushed_at":"2014-01-13T19:29:16.000Z","size":659,"stargazers_count":5,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-12T18:06:20.485Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nisaacson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-11-15T15:40:02.000Z","updated_at":"2019-07-15T12:59:01.000Z","dependencies_parsed_at":"2022-09-18T11:50:29.698Z","dependency_job_id":null,"html_url":"https://github.com/nisaacson/riak-streaming-node","commit_stats":null,"previous_names":[],"tags_count":51,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nisaacson%2Friak-streaming-node","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nisaacson%2Friak-streaming-node/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nisaacson%2Friak-streaming-node/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nisaacson%2Friak-streaming-node/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nisaacson","download_url":"https://codeload.github.com/nisaacson/riak-streaming-node/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248610345,"owners_count":21132921,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T10:35:48.580Z","updated_at":"2025-10-15T18:34:02.291Z","avatar_url":"https://github.com/nisaacson.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Overview\n\nBasic riak client that is fully streaming\n\n[![Build Status](https://travis-ci.org/nisaacson/riak-streaming-node.png?branch=master)](https://travis-ci.org/nisaacson/riak-streaming-node) [![Dependency Status](https://david-dm.org/nisaacson/riak-streaming-node.png)](https://david-dm.org/nisaacson/riak-streaming-node) [![Code Climate](https://codeclimate.com/github/nisaacson/riak-streaming-node.png)](https://codeclimate.com/github/nisaacson/riak-streaming-node)\n\n[![NPM](https://nodei.co/npm/riaks.png)](https://nodei.co/npm/riaks/)\n\n**Table of Contents**\n\n- [Overview](#overview)\n- [Installation](#installation)\n- [Usage](#usage)\n- [API](#api)\n    - [bucketKeys](#bucketkeys)\n    - [bucketKeysStream](#bucketkeysstream)\n    - [bucketStream](#bucketstream)\n    - [bucketDeleteAll](#bucketdeleteall)\n    - [getWithKey](#getwithkey)\n    - [saveWithKey](#savewithkey)\n    - [deleteWithKey](#deletewithkey)\n    - [search](#search)\n    - [searchStream](#searchStream)\n    - [valueStreamWithQueryRange](#valuestreamwithqueryrange)\n    - [queryRangeStream](#queryrangestream)\n    - [mapReduceStream](#mapreducestream)\n    - [purgeDB](#purgedb)\n    - [disconnect](#disconnect)\n- [Test](#test)\n\n# Installation\n\n```bash\nnpm install -S riaks\n```\n\n# Usage\n\nThe client supports both the riak `http` interface as well as `protocol buffers`. Specify the interface you wish to use via the `protocol` parameter in the configuration option.  The api of the resulting client object is the same regardless of which protocol is used.\n\n\n* http Client\n\n```javascript\nvar Client = require('riaks')\nvar opts = {\n  host: 'localhost',\n  protocol: 'http',\n  port: 8098,\n  timeout: 10000 // optional\n}\n\nvar client = new Client(opts)\n```\n\n\n* protocol buffer client. Use the `protocol: 'protobuf'` setting.\n\n```javascript\nvar Client = require('riaks')\nvar opts = {\n  host: 'localhost',\n  protocol: 'protobuf',\n  port: 8087,\n  timeout: 10000 // optional\n}\n\nvar client = new Client(opts)\n```\n\n* https Client. If you want all traffic to be encrypted over https, use the `protocol: 'https'` setting\n\n```javascript\nvar Client = require('riaks')\nvar opts = {\n  host: 'localhost',\n  protocol: 'https',\n  port: 443,\n  timeout: 10000 // optional\n}\n\nvar client = new Client(opts)\n```\n\nAftter the client is created, you can verify the connection to the riak server is valid by calling the `connect` method on the client. This returns a promise which is resolved after the client succesfully communicates with the riak server and gets a response back.\n\n```javascript\nvar promise = client.connect()\npromise.then(connectHandler).fail(failHandler)\n\nfunction connectHandler() {\n  console.log('client connected correctly')\n}\n\nfunction failHandler(err) {\n  console.error('connection failed')\n  console.error(err)\n  throw err\n}\n```\n\n# API\n\nOnce you have a client object, the following api is available\n\n## bucketKeys\n\nGet all keys from a bucket (returns a promise). According to Riak this should not be used in production since it is very slow\n\n```javascript\nvar opts = {\n  bucket: 'test_bucket_name' // name of the bucket to look in\n}\nvar promise = client.bucketKeys(opts)\npromise.then(function(keys) {\n  console.dir(keys)\n})\n```\n\n## bucketKeysStream\n\nGet all the keys in a bucket, but stream them back as they come back from Riak\n\n```javascript\nvar opts = {\n  bucket: 'test_bucket_name' // name of the bucket to look in\n}\nvar keyStream = client.bucketKeysStream(opts)\nkeyStream.on('data', function(key) {\n  console.dir(key)\n})\n```\n\n## bucketStream\n\nGet a list of all buckets and stream back the bucket names as they come back from Riak\n\n```javascript\nvar bucketStream = client.bucketStream()\nbucketStream.on('data', function(bucketName) {\n  console.dir(bucketName)\n})\n\n```\n\n## bucketDeleteAll\n\ndelete all keys in a bucket (returns a promise)\n\n```javascript\nvar opts = {\n  bucket: 'test_bucket', // name of bucket\n  concurrency: 5 // optional, limits max number of simultanous delete requests to riak\n}\nvar promise = client.bucketDeleteAll(opts)\npromise.then(function() {\n  console.dir('bucket emptied')\n})\n```\n\n## getWithKey\n\nGet value for key (returns a promise).\n\nIf the object was saved with `content-type: application/json`, the client will call JSON.parse and return an actual javascript object.\n\n```javascript\nvar opts = {\n  bucket: 'test_bucket',\n  key: 'test_key'\n}\nvar promise = client.getWithKey(opts)\npromise.then(function(value) {\n  // if key is not found value will be undefined\n  console.dir(value)\n})\n```\n\nIf you need both the value as well as the secondary index key value pairs, specify the `returnMeta: true` parameter.\n\n```javascript\nvar opts = {\n  bucket: 'test_bucket',\n  key: 'test_key',\n  returnMeta: true // optional, return 2i indexes, headers, etc\n}\nvar promise = client.getWithKey(opts)\npromise.then(function(reply) {\n  // note that reply is null if no key-value pair exists in riak\n  var value = reply.value\n  console.dir(value)\n\n  var indices = reply.indices\n  console.dir(indices)\n})\n```\n\nIn the example above, the reply from getWithKey will look like\n\n```javascript\n{\n  value: { foo: 'bar'}, // actual javascript object but stored as json in riak\n  indices: [\n    {\n      key: 'first_index_bin',\n      value: 'first index value here'\n    },\n    {\n      key: 'second_index_bin',\n      value: 'second index value here'\n    }\n  ]\n}\n```\n\n## saveWithKey\nsave value for key with optionaly secondary indices(returns a promise). In the following example, we set two different secondary indices `test_index_one` with value `45` and `test_index_two` with value `foo`\n\n* String value\n\n```javascript\nvar opts = {\n  bucket: 'test_bucket',\n  key: 'test_key',\n  indices: {\n    test_index_one: '45'\n    test_index_two: 'foo'\n  },\n  returnBody: true, // (optional) whether to return the contents of the stored object. defaults to false\n  value: 'test_value_here'\n}\nvar promise = client.saveWithKey(opts)\npromise.then(function() {\n  // if key is not found value will be undefined\n  console.dir('key saved')\n})\n```\n\n* Object value. `JSON.stringify` will be called on the value before it is saved to riak and the header `content-type: application/json` header will be applied. When you get this value back from riak via the `client.getWithKey` method, `JSON.parse` will be called so you receive an actual object back.\n\n```javascript\nvar opts = {\n  bucket: 'test_bucket',\n  key: 'test_key',\n  indices: {\n    test_index_one: '45'\n    test_index_two: 'foo'\n  },\n  returnBody: true, // (optional) whether to return the contents of the stored object. defaults to false\n  value: { foo: 'bar' }\n}\nvar promise = client.saveWithKey(opts)\npromise.then(function() {\n  // if key is not found value will be undefined\n  console.dir('key saved')\n})\n```\n\n## deleteWithKey\n\ndelete a key in a given bucket, returning a promise\n\n```javascript\nvar opts = {\n  bucket: 'test_bucket',\n  key: 'test_key'\n}\nvar promise = client.deleteWithKey(opts)\npromise.then(function() {\n  // if key is not found value will be undefined\n  console.dir('key deleted')\n})\n```\n\n## search\n\nSearch via the solr-compatible interface.\n\n[riak reference](http://docs.basho.com/riak/latest/dev/using/search/#Query-Interfaces)\n\n```javascript\nvar opts = {\n  index: bucket,\n  q: 'value_*', // query\n  df: 'bar',    // default field\n  start: 0,\n  rows: 1,      // limit number of results\n  sort: 'bar',\n  filter: '',\n  presort: 'key',\n}\nvar promise = client.search(opts)\npromise.then(function (reply) {\n  console.dir(reply)\n})\n```\n\nThe reply object in the example above will look like\n\n```\n{\n  numFound: 4,\n  start: 0,\n  maxScore: '0.00000e+0',\n  docs: [\n    {\n      id: '1_key',\n      index: 'http_test',\n      fields: { bar: 'value_1' },\n      props: {}\n    }\n  ]\n}\n```\n\n## searchStream\n\nSearch via the solr-compatible interface and automatically search over paginated results via streaming interface\n\n[riak reference](http://docs.basho.com/riak/latest/dev/using/search/#Query-Interfaces)\n\n```javascript\nvar opts = {\n  index: bucket,\n  q: 'value_*', // query\n  df: 'bar',    // default field\n  start: 0,\n  rows: 1,      // limit number of results per page, (optional, defaults to 20)\n  maxRows: 20   // end the stream when reached, if not set client will stream all matching results\n  sort: 'bar',\n  filter: '',\n  presort: 'key',\n}\nvar readStream = client.searchStream(opts)\nreadStream.on('data', function dataHandler(reply) {\n  console.dir(reply)\n})\nreadStream.on('finish', function finishHandler() {\n  console.log('got all rows from search stream')\n})\n```\n\nThe reply object in the `dataHandler` function in the example above will look like\n\n```\n{\n  id: '1_key',\n  index: 'http_test',\n  fields: { bar: 'value_1' },\n  props: {}\n}\n```\n\n## queryRangeStream\n\nStream back keys from a secondary index query. The appriate suffix of either `_int` or `_bin` will appended to the index key based on the type of value in the `start` field. This maps the to [http://docs.basho.com/riak/latest/dev/references/http/secondary-indexes/](http://docs.basho.com/riak/latest/dev/references/http/secondary-indexes/) http interface in Riak.\n\nIf you want the secondary index values in the output, specify `returnTerms: true` in the options object. Note that the `returnTerms` corresponds to `return_terms` in the riak http options.\n\nIf you want limit the number of results returned, specify `maxResults: \u003cinteger value`. The `maxResults` options maps the riak `max_results` url parameter.\n\n* Keys only\n\n```javascript\nvar opts = {\n  bucket: 'test_bucket',\n  indexKey: 'test_index_key', // `_bin` or `_int` suffix automatically added based on start value type\n  start: '/x00',\n  end: '/xff',\n  returnTerms: false, // false by default if not specified\n  maxResults: 10 // optional, limits the number of results returned\n}\nvar keyStream = client.queryRangeStream(opts)\nkeyStream.on('data', function(key) {\n  console.dir(key)\n})\n```\n\n* Keys And Index values\n\n```javascript\nvar inspect = require('eyespect').inspector()\nvar opts = {\n  bucket: 'test_bucket',\n  indexKey: 'test_index_key', // `_bin` or `_int` suffix automatically added based on start value type\n  start: '/x00',\n  end: '/xff',\n  returnTerms: true, // false by default if not specified\n  maxResults: 10 // optional, limits the number of results returned\n}\nvar keyStream = client.queryRangeStream(opts)\nkeyStream.on('data', function(data) {\n  var key = data.key\n  var value = data.value\n  inspect(key, 'secondary index value')\n  inspect(value, 'key for key-\u003evalue pair in riak')\n})\n```\n\nThe data objects emitted by the stream in the above example look like\n\n```javascript\n{\n  key: 'secondary index value'\n  value: 'key to actual object stored in riak'\n}\n```\n\n## mapReduceStream\n\nRun mapreduce jobs with arbitrary javascript functions and stream back the results. In the example below, the actual javascript functions for the map and reduce phase as passed in as paramters. The client will handle stringifying these functions before sending the `mapred` request to riak.\n\n```javascript\nvar opts = getMapReduceOpts()\nvar readStream = client.mapReduceStream(opts)\nreadStream.on('data', function(data) {\n  console.dir(data)\n})\nreadStream.on('error', function(err) {\n  console.dir('mapReduce stream error', err)\n  throw err\n})\n\nreadStream.on('end', function() {\n  console.dir('all mapreduce results streamed back and readStream closed')\n})\n\nfunction getMapReduceOpts() {\n  var mapPhaseOpts = {\n    map: {\n      fn: mapFunction,\n      keep: false,\n      arg: 'foo'\n    }\n  }\n  var reducePhaseOpts = {\n    reduce: {\n      fn: reduceFunction,\n      keep: true,\n      arg: 'foo'\n    }\n  }\n  var opts = {}\n  var query = [mapPhaseOpts, reducePhaseOpts]\n\n  // use a secondary index query as an input to the map reduce job\n  var inputs = {\n    bucket: 'test_bucket',\n    index: 'test_index_key_bin',\n    start: 'test_start',\n    end: 'test_end'\n  }\n  opts.query = query\n  opts.inputs = inputs\n  opts.timeout = 1000 // optional, set to 1000 milliseconds timeout or 1 second\n  return opts\n}\n\nfunction mapFunction(value, keyData, arg) {\n  var data = Riak.mapValuesJson(value)[0]\n  var output = [data]\n  return output\n}\n\nfunction reduceFunction(list) {\n  // riak runs reduce jobs in batchs of 20. If this job is the result of a previous reduce, return it directly.\n  // see http://stackoverflow.com/questions/16359656/riak-map-reduce-in-js-returning-limited-data/20058732\n  if (typeof list === 'number') {\n    return list\n  }\n  var sum = list.reduce(function(a, b) {\n    a += b\n    return a\n   }, 0)\n  return sum\n}\n```\n\n\n## purgeDB\n\nCompletely clear out all buckets in Riak. (returns a promise)\n\n```javascript\nvar promise = client.purgeDB()\npromise.then(function() {\n  console.log('All Riak buckets completely cleared')\n})\n```\n\n## disconnect\n\nClose existing open connects to riak server. This is really only needed for the `protobuf` protocol option, and is a `no-op` for the `http` protocol option to maintain a consistent public api.\n\n```javascript\nclient.disconnect()\n```\n\n# Test\n\nMake sure you have riak running on the default port before running the test suite.\n\n```bash\nnpm install\nmake test # or npm test\n```\n\nTo make testing easier, a `Vagrantfile` is included in the root of this repo. Bringing up this virtual machine will install the required dependencies so that you can test the riaks client without having to install nodejs and riak on your local machine.\n\n```bash\nvagrant up --provision\nvagrant ssh\ncd /vagrant\nnpm install\nnpm test\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnisaacson%2Friak-streaming-node","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnisaacson%2Friak-streaming-node","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnisaacson%2Friak-streaming-node/lists"}