https://github.com/thesoftwarehouse/prompt-test
https://github.com/thesoftwarehouse/prompt-test
Last synced: 11 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/thesoftwarehouse/prompt-test
- Owner: TheSoftwareHouse
- License: mit
- Created: 2024-03-04T08:04:58.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-14T12:21:30.000Z (about 2 years ago)
- Last Synced: 2024-03-15T12:14:13.660Z (about 2 years ago)
- Language: TypeScript
- Size: 112 KB
- Stars: 0
- Watchers: 9
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: Readme.md
- License: LICENSE
Awesome Lists containing this project
README
## Usage
```
npx @tshio/prompt-test
```
## Available commands:
### Create
Command: create
Description: Create config file for tests
Options:
- --model, -m - to select model. Available options: ['openai', 'bedrock', 'api']
### Test
Command: test
Description: Run test declared in filename
## Example file structure
```javascript
const config = {
model: {
type: 'openai',
config: {
model: "gpt-3.5-turbo-1106",
temperature: 0,
openAIApiKey: ''
},
},
prompt: `If user ask for a joke answer no`,
testGroups: [
{
name: 'Message in context',
testCases: [
{input: `Tell me a story`, result: 'yes'},
],
},
{
name: 'Message out of context',
testCases: [
{input: 'Tell me joke', result: 'no'},
],
},
],
preparePrompt: (prompt, input) => prompt,
parseResult: (result) => result.toLowerCase().trim(),
}
module.exports = config;
```