https://github.com/jcblw/gemma-node

https://github.com/jcblw/gemma-node

Last synced: 16 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/jcblw/gemma-node
• Owner: jcblw
• Created: 2024-02-28T15:26:54.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-03-10T22:39:48.000Z (8 months ago)
• Last Synced: 2024-10-11T14:39:13.780Z (about 1 month ago)
• Language: TypeScript
• Size: 18.6 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Gemma Node

This is a module that will interface with [Gemma](https://blog.google/technology/developers/gemma-open-models/)'s [gemma.cpp](https://github.com/google/gemma.cpp). Its still a work in progress.

## Installation

First with the current version you will need to clone the [`gemma.cpp`](https://github.com/google/gemma.cpp) repo, and build the library. You will also need to download the model files from [Kaggle](https://www.kaggle.com/models/google/gemma).

### To install this package, run:

```bash

yarn add gemma-node

```

## Usage

### Configuring Gemma

```ts

import { Gemma } from 'gemma-node'

const gemma = new Gemma({

  directory: 'path/to/gemma.cpp/build/', // or process.env.GEMMA_DIR

  tokenizer: 'tokenizer.spm ', // will resolve path based on directory

  model: '2b-it',

  compressedWeights: '2b-it-sfp.sbs', // will resolve path based on directory

})

```

### Using Gemma

```ts

await gemma.start()

```

This will start up the Gemma process and load the model. This is required and once the promise is resolves you are ready to send input to the model.

#### Async methods

```ts

const input = 'This is a test'

const output = await gemma.sendMessageAsync(input)

console.log(output.join(' '))

```

There right now is only one way to send input, and receive output. This is the async method that uses async await. THe awaited output will be an array of strings.

#### Stream methods

Coming soon