https://github.com/moeki0/kiruke
Kiruke is a tool for testing prompts for LLM.
https://github.com/moeki0/kiruke
Last synced: 10 months ago
JSON representation
Kiruke is a tool for testing prompts for LLM.
- Host: GitHub
- URL: https://github.com/moeki0/kiruke
- Owner: moeki0
- Created: 2024-10-02T17:42:59.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-10-04T13:29:53.000Z (over 1 year ago)
- Last Synced: 2025-07-31T13:37:10.234Z (11 months ago)
- Language: TypeScript
- Homepage:
- Size: 9.77 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Kiruke
Kiruke is a tool for testing prompts for LLM.
## Installation
```bash
npm install -D kiruke
```
```
yarn add -D kiruke
```
```bash
pnpm add -D kiruke
```
## Usage
```typescript
// example.ts
import OpenAI from 'openai';
import dotenv from 'dotenv';
dotenv.config();
const client = new OpenAI({
apiKey: process.env['OPENAI_API_KEY'],
});
const example = async (prompt: string) => {
const chatCompletion = await client.chat.completions.create({
messages: [{ role: 'user', content: `
Please proofread the following text. If you correct any typos or errors, enclose them in .
Text: ${prompt}
` }],
model: 'gpt-4o',
});
return chatCompletion.choices[0].message.content
}
```
```typescript
// example.kiruke.ts
import { example } from "./example";
import { describe, test } from "kiruke";
describe(example, () => {
test(`Text: Hello, ChatGPT! How is the weather today?`, async (prompt, result) => {
expect(prompt).not.include("weather");
expect(prompt).match(/today/);
expect(result).include("weather");
});
test(`Text: Please tell me the characteristics of humans.`, async (prompt, result) => {
expect(prompt).not.include("characteristics");
expect(prompt).match(/humans/);
expect(result).include("characteristics");
});
}, { threshold: 0.7 });
```
```json
{
"scripts": {
"test:prompt": "kiruke"
}
}
```
Output example:
```bash
🧪example
Prompt: Text: Hello, ChatGPT! How is the whether today?
Latency: 875.019ms
Result: Text: Hello, ChatGPT! How is the weather today?
✅ Assertion passed: does not include weather
✅ Assertion passed: matches /today/
✅ Assertion passed: includes weather
Prompt: Text: Please tell me the characteeristics of humans.
Latency: 1.017s
Result: Text: Please tell me the characteristics of humans.
✅ Assertion passed: does not include characteristics
✅ Assertion passed: matches /humans/
✅ Assertion passed: includes characteristics
✅ Test passed: 100% / 70%
```
## Matchers
- `include("weather")`
- `not.include("weather")`
- `match(/today/)`
- `not.match(/today/)`
- `toBeLike("It should be positive.")`
## Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/kawakamimoeki/kiruke. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/kawakamimoeki/kiruke/blob/main/CODE_OF_CONDUCT.md).
## License
The npm package is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
## Code of Conduct
Everyone interacting in the Kiruke project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/kawakamimoeki/kiruke/blob/main/CODE_OF_CONDUCT.md).