Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/meteatamel/opentrivia-llm-testing
A test suite to see if LLMs can correctly guess the correct answer in OpenTrivia quizzes
https://github.com/meteatamel/opentrivia-llm-testing
Last synced: 2 days ago
JSON representation
A test suite to see if LLMs can correctly guess the correct answer in OpenTrivia quizzes
- Host: GitHub
- URL: https://github.com/meteatamel/opentrivia-llm-testing
- Owner: meteatamel
- License: apache-2.0
- Created: 2024-06-03T13:20:55.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-06-06T09:48:46.000Z (7 months ago)
- Last Synced: 2024-12-27T05:41:55.574Z (7 days ago)
- Language: Python
- Size: 29.3 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# OpenTrivia LLM testing
A test suite to see if LLMs can correctly guess the correct answer in quizzes from [Open
Trivia database](https://opentdb.com/).
Install dependencies:
```sh
pip install -r requirements.txt
```
Minimal run:
```sh
python main.py your-project-id model-id
```
Run with all options:
```sh
python main.py your-project-id model-id --num_iterations=4 --no_questions=25 --google_search_grounding
```
Example runs:
```sh
python main.py genai-atamel gemini-1.0-pro
python main.py genai-atamel gemini-1.5-pro
python main.py genai-atamel gemini-1.5-flash
```
You can find some run outputs in [runs](runs) folder.