An open API service indexing awesome lists of open source software.

https://github.com/feder-cr/jev

jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.
https://github.com/feder-cr/jev

ai classification cpu-inference decision-model edge-ai gguf jev jev-alternative llama-cpp llm llm-inference local-ai local-llm microsoft-decision microsoft-decision-1 offline on-device-ai open-source-alternative self-hosted yes-no

Last synced: 29 days ago
JSON representation

jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.

Awesome Lists containing this project

README

          






jevos, decisions on a laptop CPU. An animation: a text and a question sent to /v1/systemone, the probability that comes back, 28 ms against 129, 311 and 3,060 for the others, and the one binary that serves it.

**Decisions on a laptop CPU in 28โ€“130 ms.** Send a text and a question (yes/no, multiple choice or a
score) and get back the probability of each answer, in TypeSafe Jev's API, on your own machine.


jevos playing a Chrome Dino-style game on the CPU, answering two yes/no questions per step (recording at 2ร— speed)

## Quickstart

From the [release](https://github.com/feder-cr/jev/releases/tag/jevos-v5), download the archive for your
system and `jevos-v5-openvino-int8.zip`, then:

```bash
tar -xzf jev-linux-x64.tar.gz # Windows: unzip jev-windows-x64.zip
cd jev
unzip ../jevos-v5-openvino-int8.zip # creates model/
./jev serve # Windows: jev.exe serve
```

The model is also on Hugging Face, [feder-cr/jev](https://huggingface.co/feder-cr/jev):
`hf download feder-cr/jev --include "model/*" --local-dir jev` puts it in `jev/model/`.

```bash
curl http://127.0.0.1:8017/v1/systemone -H 'Content-Type: application/json' -d '{
"model": "jev-latest",
"state": "My invoice shows a charge I do not recognize.",
"questions": {"billing": {"type": "noul", "instructions": "Is this a billing problem?"}}}'
```

```json
{"model": "jevos-v5", "answers": {"billing": {"type": "noul", "noul": 0.97}}, "usage": {"input_tokens": 28, "output_tokens": 0}}
```

One binary, CPU only, no Python. The release and the Hugging Face repo also have the model as GGUF for
llama.cpp.

## Benchmarks

Latency on a short / long request: jevos-v5 30 / 132 ms; Jev 311 / 314 ms; Qwen3.5-4B 3,060 / 4,761 ms; Laya 129 / 480 ms. Accuracy on 7 tasks (Admission policy, Rental policy, Rules and scenarios, Agent rules, Authority rules, Fraud points, Patent phrases): jevos-v5 0.92, 0.81, 0.76, 0.74, 0.79, 0.53, 0.36; Jev 1.00, 0.91, 0.88, 0.85, 0.98, 0.69, 0.59; Qwen3.5-4B 0.82, 0.60, 0.72, 0.60, 0.78, 0.45, 0.18; Laya 0.54, 0.31, 0.55, 0.45, 0.64, 0.25, 0.32

Same questions for every system, through the same HTTP client. Latency is the median of 10 requests (for
jevos, of 40 over four sessions) on an Intel Core Ultra 7 255H laptop, 16 threads, each text read from
scratch. No task text was used to train jevos; all seven sets were among those used to choose the released
checkpoint.

| | **jevos-v5** | Jev | Laya |
|---|:---:|:---:|:---:|
| Yes/no questions | โœ“ | โœ“ | โœ“ |
| Multiple choice | โœ“ | โœ“ | โœ“ |
| Scores | โœ“ (early) | โœ“ | โœ“ |
| Runs on | your machine | cloud | your machine |
| Cost | free | per token | free |
| Context | 8,192 tokens | not stated | 512 tokens |

## API

`POST /v1/systemone`, in TypeSafe Jev's wire format: code written for Jev's SDK works unchanged for yes/no
questions. One request can ask several questions about the same text, which is read once:

```json
{
"model": "jev-latest",
"state": {"item": "wireless mouse", "customer_message": "The box arrived empty. This is the second time!"},
"questions": {
"refund": {"type": "noul", "instructions": "Our policy refunds items reported missing within 30 days of delivery. Should this customer get a refund?"},
"team": {"type": "choice", "instructions": "Which team should handle this message?",
"criteria": {"billing": "payments, refunds", "shipping": "deliveries, missing parcels", "tech": "a product that does not work"}},
"anger": {"type": "score", "instructions": "How angry is the customer?", "criteria": ["calm", "annoyed", "angry", "furious"]}
}
}
```

A `noul` returns P(yes); a `choice` the most probable option; a `score` the expected level. Choices and scores
also return a probability per answer and a `confidence`: when it is low, send the case to a person. Put the
rule in the question, and do sums in code.

`GET /health` reports readiness and the SHA-256 of the model files; `./jev decide request.json` answers a
request file without a server.

## Options

| Option | Default | |
|---|---|---|
| `--threads` | all logical CPUs | fewer if other heavy apps are running |
| `--host`, `--port` | `127.0.0.1`, `8017` | where the server listens |
| `--model-dir` | `model` beside the binary | the model folder |

With `JEV_API_KEY` set, every call but `/health` needs `Authorization: Bearer `. About 1 GB of memory,
1.4 GB with the text cache full. Every option is described at the top of [src/main.cpp](src/main.cpp).

## Build from source

```bash
python -m pip install -r requirements.txt # OpenVINO's SDK, CMake, Ninja, the tests' packages
python scripts/build.py # dist/jev; on Windows, from a Visual Studio developer prompt
python tests/check.py # with the model in dist/jev/model
```

## More

Guides and measurements are in the [wiki](https://github.com/feder-cr/jev/wiki). Built together with
[Loris Salsi (@LosaLosSantos)](https://github.com/LosaLosSantos).