{"id":18304657,"url":"https://github.com/yukinagae/genkit-promptfoo-sample","last_synced_at":"2025-08-15T14:16:25.627Z","repository":{"id":256489175,"uuid":"854998296","full_name":"yukinagae/genkit-promptfoo-sample","owner":"yukinagae","description":"Sample implementation demonstrating how to use Firebase Genkit with Promptfoo","archived":false,"fork":false,"pushed_at":"2024-09-11T04:44:55.000Z","size":2415,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-01T00:12:05.745Z","etag":null,"topics":["evaluation","evaluation-framework","genkit","llm","llm-eval","llm-evaluation","llm-evaluation-framework","llmops","prompt-testing","promptfoo","prompts","testing"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yukinagae.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-10T06:02:40.000Z","updated_at":"2024-09-11T04:44:58.000Z","dependencies_parsed_at":"2024-09-11T03:43:27.438Z","dependency_job_id":"83a6701b-3f48-4bdc-a514-13052cd08f16","html_url":"https://github.com/yukinagae/genkit-promptfoo-sample","commit_stats":null,"previous_names":["yukinagae/genkit-promptfoo-sample"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/yukinagae/genkit-promptfoo-sample","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yukinagae%2Fgenkit-promptfoo-sample","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yukinagae%2Fgenkit-promptfoo-sample/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yukinagae%2Fgenkit-promptfoo-sample/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yukinagae%2Fgenkit-promptfoo-sample/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yukinagae","download_url":"https://codeload.github.com/yukinagae/genkit-promptfoo-sample/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yukinagae%2Fgenkit-promptfoo-sample/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270579825,"owners_count":24610083,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-15T02:00:12.559Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["evaluation","evaluation-framework","genkit","llm","llm-eval","llm-evaluation","llm-evaluation-framework","llmops","prompt-testing","promptfoo","prompts","testing"],"created_at":"2024-11-05T15:29:52.344Z","updated_at":"2025-08-15T14:16:25.605Z","avatar_url":"https://github.com/yukinagae.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# genkit-promptfoo-sample\n\nThis project is a sample implementation demonstrating how to use [Firebase Genkit](https://firebase.google.com/docs/genkit) with [Promptfoo](https://www.promptfoo.dev/), a test framework designed to evaluate the output of generative AI models.\n\n- [Requirements](#requirements)\n- [Setup](#setup)\n- [Run Genkit](#run-genkit)\n- [Evaluation](#evaluation)\n- [Making Changes](#making-changes)\n- [License](#license)\n\n## Requirements\n\nBefore you start, make sure you have these installed:\n\n- **Node.js** version 22 or later\n- **npm**\n- **Genkit**\n\nFor Genkit installation, see the [official guide](https://firebase.google.com/docs/genkit/get-started).\n\nCheck your installations by running:\n\n```bash\n$ node --version # the below version is on my environment\nv22.4.1\n$ npm --version # the below version is on my environment\n10.7.0\n$ genkit --version # the below version is on my environment\n0.5.4\n```\n\n## Setup\n\n**Install Project Dependencies**: Open your terminal, navigate to this project's folder, and run:\n\n```bash\n$ npm install\n```\n\n## Run Genkit\n\n1. **Set the `GOOGLE_GENAI_API_KEY` Environment Variable**\n\nBefore running the project, you need to provide your Google GenAI API key.\n\n```bash\n$ export GOOGLE_GENAI_API_KEY=your_api_key\n```\n\n2. **Run the Genkit server locally**\n\nThis command starts the project and automatically opens your default web browser to `http://localhost:4000`.\n\n```bash\n$ npm run genkit\n```\n\nTo test the functionality, run the following command:\n\n```bash\n$ curl -X POST -H \"Content-Type: application/json\" -d '{\"data\":\"French\"}' http://127.0.0.1:3400/menuSuggestionFlow\n{\"result\":\"## **Steak Frites with Béarnaise Sauce**\\n\\nThis classic French dish is simple, elegant, and always a crowd-pleaser. \\n\\n**Here's why it's a good menu choice:**\\n\\n* **Familiar yet elevated:** It's a familiar comfort food with a touch of sophistication.\\n* **Versatile:** It can be adapted to different dietary needs (e.g., gluten-free fries, vegetarian alternative like a mushroom steak).\\n* **Visually appealing:** A beautifully seared steak with crispy fries and a luscious Béarnaise sauce is very photogenic.\\n* **High-quality ingredients:**  It allows you to showcase premium ingredients like a high-quality cut of beef and fresh herbs for the sauce.\\n* **French authenticity:** It's a true classic French dish, reinforcing the restaurant's theme.\\n\\n**To add a unique twist:**\\n\\n* **Try different cuts of beef:**  Filet mignon, ribeye, or even a hangar steak.\\n* **Offer different sauce options:**  Mushroom sauce, peppercorn sauce, or even a light truffle oil.\\n* **Add a side:**  A small salad, a side of roasted vegetables, or a creamy potato gratin.\\n\\n**This dish can be a signature item on your French-themed restaurant menu, appealing to a wide range of diners.** \\n\"}\n```\n\n## Evaluation\n\nTo evaluate the flow using [Promptfoo](https://www.promptfoo.dev/), follow these steps:\n\n1. Run the evaluation command:\n\n```bash\n$ npm run eval:run\n```\n\n2. To view the results, run:\n\n```bash\n$ npm run eval:view\n```\n\nThis will automatically open your browser to `http://localhost:15500`.\n\nYou can view the results table here:\n\n![Result Table](https://raw.githubusercontent.com/yukinagae/genkit-promptfoo-sample/main/docs/1.png)\n\nClick the 🔍 icon to see detailed information for each test:\n\n![Test Details](https://raw.githubusercontent.com/yukinagae/genkit-promptfoo-sample/main/docs/2.png)\n\nIn the details section, you can review the input and output:\n\n![Input and Output](https://raw.githubusercontent.com/yukinagae/genkit-promptfoo-sample/main/docs/3.png)\n\nAdditionally, explanations for each assertion are provided:\n\n![Assertions](https://raw.githubusercontent.com/yukinagae/genkit-promptfoo-sample/main/docs/4.png)\n\nFor more extensive testing with Promptfoo, you can customize the configuration by editing the `eval/promptfooconfig.yaml` file. For detailed instructions and additional information, please refer to the [official Promptfoo documentation](https://www.promptfoo.dev/docs/intro/).\n\n## Making Changes\n\n### Building the Project\n\nAfter making changes, you might need to build the project to see your changes in action:\n\n```bash\n$ npm run build\n```\n\n### Formatting and Linting\n\nTo ensure your code follows the project's coding standards, run the formatting and linting tools:\n\n```bash\n$ npm run typecheck # type check without modifying files\n$ npm run check     # scan without modifying files\n$ npm run fix       # modify files\n```\n\n### Kill Existing Processes\n\nSometimes existing processes are still running, preventing you from running genkit locally because the ports are already in use. In that case, run the following command to kill the processes tied to the ports:\n\n```bash\n$ npm run kill\n```\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyukinagae%2Fgenkit-promptfoo-sample","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyukinagae%2Fgenkit-promptfoo-sample","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyukinagae%2Fgenkit-promptfoo-sample/lists"}