Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/yildizdb/bigtable

TypeScript Bigtable Client with :battery::battery: included.
https://github.com/yildizdb/bigtable

big-data bigtable google google-cloud hbase key-value nodejs typescript

Last synced: 3 months ago
JSON representation

TypeScript Bigtable Client with :battery::battery: included.

Awesome Lists containing this project

README 

        
# bigtable-client



`yarn add bigtable-client`



## Intro



This is a TypeScript Bigtable client, it acts as wrapper around the official 

Google package [@google-cloud/bigtable](https://github.com/googleapis/nodejs-bigtable).

When working with Bigtable we almost always had the urge to wrap the API to add a pinch

of convenience to it, as well implement a way to get TTL (per cell basis), as well as metadata

information such as a simple count more efficiently.



This client automatically manages a `metadata` table and `ttl jobs` for every table that you manage

through it. Additionally it aims to mimic a simple CRUD interface, that is offered by a lot of redis packages

like `ioredis` for example.



Additionally the setup and all operation (except for scan) are optimized for sub-millisecond response times

(depending on your Google Cloud Bigtable Instance configuration), which helps you to develop real-time

applications based on this Bigtable. This client is not ment to be used for analytical purposes, although

it is fairly possible through scan operations.



## Before you get started



Make sure to follow the setup described [here](https://github.com/googleapis/nodejs-bigtable#before-you-begin).

You will need a Google Cloud Project with enabled billing, as well as a

setup authentication flow for this client to work.



## Using



Using it is fairly simple:



First, you have to setup a factory instance, which gets the general

configuration to connect to your Bigtable instance.

NOTE: If the instance you describe does not exist, it will be created.



```javascript

const {BigtableFactory} = require("bigtable-client");

const bigtableFactory = new BigtableFactory({



  projectId: "my-project-1", // -> see @google-cloud/bigtable configuration

  instanceName: "my-bigtable-cluster", // -> see @google-cloud/bigtable configuration

  //keyFilename: "keyfile.json", // -> see @google-cloud/bigtable configuration



  // optional:

  ttlScanIntervalMs: 5000,

  minJitterMs: 2000,

  maxJitterMs: 30000,

});

await bigtableFactory.init();

```



Then, using the factory you can create handles for you tables very easily.

You can see that we are taking away the complexity of handling columnFamilies and

columns in general, by assuming default values in the API that can be set via config optionally.

However the API always allows you to access cells (by passing a column name as parameter) directly,

as well as accessing and deleting whole rows. Please bear in mind that the number is TTL in seconds, 

and will be deleted on the next job run.



```javascript

const myTable = await bigtableFactory.get({

  name: "mytable",



  // optional:

  columnFamily: "myfamily",

  defaultColumn: "default",

  maxVersions: 1,

});



const rowKey = "myrowkey";

const value = "myvalue";



await myTable.set(rowKey, value);

await myTable.set(rowKey, value, 10, "newColumn");



await myTable.ttl(rowKey);



await myTable.multiSet(rowKey, {testColumn: "hello", anotherColumn: "yes"}, 5);



await myTable.increase(rowKey);

await myTable.decrease(rowKey);



await myTable.bulkInsert([

    {

    row: "jean-paul",

    column: "sartre",

    data: "france",

    },

    {

    row: "emmanuel",

    column: "kant",

    data: "germany",

    },

    {

    row: "baruch",

    column: "spinoza",

    data: "netherland",

    },

], 5);



await myTable.multiAdd(rowKey, {foo: 1, bar: -5}, 7);

await myTable.get(rowKey);



await myTable.ttl(rowKey);

await myTable.count();



await myTable.getRow(rowKey)

await myTable.deleteRow(rowKey);



myTable.close(); // or bigtableFactory.close();

```



You can also scan tables (be carefull as these operations are slow).



```javascript

const filters = [

    {

        // -> check out the official api for bigtable filters: https://cloud.google.com/nodejs/docs/reference/bigtable/0.13.x/Filter#interleave

    }

];



const etl = (row) => {

    return row.id || null;

};



const cells = await myTable.scanCells(filters, etl);

```



You can activate debug logs via env variable `DEBUG=yildiz:bigtable:*`.



You can find additional implementation examples here:

* [Example File](example/index.ts)

* [Integration Test](test/int/Service.test.js)



# License



License is MIT



# Disclaimer



This project is not associated with Google.