{"id":15012801,"url":"https://github.com/ryanhaticus/pretzelduck","last_synced_at":"2026-03-15T20:11:45.244Z","repository":{"id":255121794,"uuid":"848606018","full_name":"ryanhaticus/pretzelduck","owner":"ryanhaticus","description":"Use natural language to write end-to-end tests in Playwright.","archived":false,"fork":false,"pushed_at":"2024-11-05T05:16:54.000Z","size":300,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-02-24T18:44:01.956Z","etag":null,"topics":["ai","e2e","hacktoberfest","playwright","testing"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/pretzelduck","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ryanhaticus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-08-28T04:22:24.000Z","updated_at":"2026-02-05T14:49:46.000Z","dependencies_parsed_at":"2025-04-12T04:01:19.128Z","dependency_job_id":"e88589fe-8dd3-4a5c-bc8f-373f4aa53152","html_url":"https://github.com/ryanhaticus/pretzelduck","commit_stats":{"total_commits":76,"total_committers":3,"mean_commits":"25.333333333333332","dds":"0.10526315789473684","last_synced_commit":"b87069c996e8ff9e81c2f166e235fad434b988fe"},"previous_names":["ryanhaticus/pretzelduck"],"tags_count":29,"template":false,"template_full_name":null,"purl":"pkg:github/ryanhaticus/pretzelduck","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryanhaticus%2Fpretzelduck","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryanhaticus%2Fpretzelduck/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryanhaticus%2Fpretzelduck/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryanhaticus%2Fpretzelduck/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ryanhaticus","download_url":"https://codeload.github.com/ryanhaticus/pretzelduck/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryanhaticus%2Fpretzelduck/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30550789,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-15T15:03:43.933Z","status":"ssl_error","status_checked_at":"2026-03-15T15:03:37.630Z","response_time":61,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","e2e","hacktoberfest","playwright","testing"],"created_at":"2024-09-24T19:43:14.455Z","updated_at":"2026-03-15T20:11:45.228Z","avatar_url":"https://github.com/ryanhaticus.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pretzel Duck\nUse natural language to write end-to-end tests in Playwright.\n\n```typescript\nimport { PretzelDuck, openai } from \"pretzelduck\";\nimport { test } from \"@playwright/test\";\n\nconst { describe, beforeEach } = test;\n\nconst { it } = new PretzelDuck(test, openai(\"gpt-4o\"));\n\ndescribe(\"search functionality\", () =\u003e {\n\tbeforeEach(async ({ page }) =\u003e {\n\t\tawait page.goto(\"https://github.com/\");\n\t});\n\n\tit(\n\t\t\"should find and favorite the Pretzel Duck repository\",\n\t\t\"the favorite button should be highlighted yellow on a GitHub repository titled Pretzel Duck\",\n\t);\n});\n```\n\n## Why\nEnd-to-end (E2E) testing in its current form is often not practical for various reasons:\n- Writing end-to-end tests takes a long time.\n- Often, in order to use end-to-end tests effectively, you need to maintain a mock API or have control over your environment's data.\n- Seemingly small UI changes can break end-to-end tests, requiring you spend additional time fixing them.\n\nFrom the author:\n\u003e As an engineer at John Deere, my team tirelessly works to meet feature deadlines and deliver bug fixes. Moreover, we don't control the data in our quality assurance environment. Though we wanted to save integration testing time, the investment required to write end-to-end tests has never proven to be worth it. The Pretzel Duck project was a shower thought of mine to help my team overcome this opportunity cost. Though it can be quite expensive to run an LLM, especially in this fashion, the engineering time outweighs this cost ten-fold.\n\n## Features\n\n- Simple to use with good defaults\n    - Just provide a natural language `goal` and `assertion` for every test\n- Interactions such as clicking, inputting text, scrolling, and hitting enter\n- Highly configurable with a BYOM (bring your own model) approach\n- Use existing Playwright DX features like tracing and UI\n- **Saves engineering time**\n\n## Getting Started\n- [I don't have an existing Playwright test suite](#i-dont-have-an-existing-playwright-test-suite)\n- [I have an existing Playwright test suite](#i-have-an-existing-playwright-test-suite)\n\n### I don't have an existing Playwright test suite\nBefore proceeding, please follow the [introductory Playwright documentation](https://playwright.dev/docs/intro) to create your first test suite. This usually takes around 5-10 minutes.\n\nNext, modify your `playwright.config.ts` to increase the test timeout time.\nBecause Pretzel Duck repeatedly makes calls to your model of choice, testing times can become lengthy. As you refine your goal prompts, you may be able to decrease the testing suite timeout. We recommend a minute to start.\n```typescript\nexport default defineConfig({\n    // ...\n    timeout: 60000\n    // ...\n});\n```\n\n### I have an existing Playwright test suite\nYou'll first need to install Pretzel Duck via [npm](https://www.npmjs.com/).\n```bash\nnpm i pretzelduck\n```\n\nYou'll also need to install the AI SDK of your choice. This example uses OpenAI.\n\nNext, create a `pretzelduck.ts` file that's accessible from your testing suite.\n```typescript\nimport { PretzelDuck } from 'pretzelduck';\nimport { openai } from '@ai-sdk/openai';\nimport { test } from '@playwright/test';\n\n/*\n    The `model` expected by Pretzel Duck is identical to those created by the Vercel AI SDK.\n    We support any model that allows for multi-modal input (image and text).\n    \n    This example uses `openai`, but others are available.\n    You will need to provide an API key using the respective environment variables.\n*/\nconst model = openai('gpt-4o');\n\nexport const pretzelDuck = new PretzelDuck(test, model);\n```\n\nIn your testing suite, you're now ready to write your first natural language end-to-end test!\nOpen up an existing `.spec.ts` or `.test.ts` file, or use the one `example.spec.ts` generated by Playwright.\n```typescript\nimport { test } from \"@playwright/test\";\nimport { pretzelDuck } from \"./pretzelduck\";\n\n// Setup the page using the base Playwright `test` utility.\ntest.beforeEach(async ({ page }) =\u003e {\n\tawait page.goto(\"https://www.example.com/\");\n});\n\npretzelDuck.test(\n\t\"find more information about protocol registries\", // goal\n\t\"you should be on a page titled Protocol Registries\", // assertion indicating the goal has been reached\n);\n```\n\nThat's it, test away! In your console, enter the following:\n```bash\nnpx playwright test --ui\n```\n\nFrom the UI, run the respective test and see the model perform in real-time.\n\n## Advanced\nThough Pretzel Duck strives to be a 0-configuration solution, more advanced use cases may require some fine-tuning. Fortunately, you can configure Pretzel Duck instance-by-instance and test-by-test.\n\nThe following options can be provided as a trailing parameter on both the `PretzelDuck` class and the `test` function:\n```typescript\ntype TestOptions = {\n\tinteractions: {\n\t\tmaxInteractions: number;\n\t\tdisabled: InteractionLabels[];\n\t\tinteractables: {\n\t\t\tdisabledElements: InteractableElements[];\n\t\t\tdisabledRoles: InteractableRoles[];\n\t\t};\n\t};\n\tassertions: {\n\t\tmaxRetries: number;\n\t\ttemperature: number;\n\t\timmediate: boolean;\n\t};\n\tdecisions: {\n\t\tmaxRetries: number;\n\t\tprogressions: {\n\t\t\tenabled: boolean;\n\t\t\ttype: 'forced';\n\t\t};\n\t\ttemperature: number;\n\t\tmaxEntropy: number;\n\t\tuseScreenshots: boolean;\n\t\tuseVisibleHtml: boolean;\n\t\tuseHistory: boolean;\n\t};\n};\n```\nHere's an example of the configuration in use:\n```typescript\nimport { PretzelDuck } from \"pretzelduck\";\nimport { test } from \"@playwright/test\";\n\nexport const pretzelDuck = new PretzelDuck(test, /* Your Model Here */, {\n\tinteractions: {\n\t\tdisabled: [\"submit\"],\n\t\tinteractables: {\n\t\t\tdisabledElements: [\"select\"],\n\t\t\tdisabledRoles: [\"menuitem\"],\n\t\t},\n\t},\n\tdecisions: {\n\t\ttemperature: 0.8,\n\t},\n});\n\npretzelDuck.test(\n\t\"find the e-mail address of the website owner\",\n\t\"an e-mail address should be visible on screen\",\n\t{\n\t\tdecisions: {\n\t\t\tprogressions: {\n\t\t\t\tenabled: false,\n\t\t\t},\n\t\t\tuseVisibleHtml: false,\n\t\t},\n\t},\n);\n```\n\n## Architecture\n![image](https://github.com/user-attachments/assets/cd056925-29c8-4fff-bad3-3af6bc66da5e)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryanhaticus%2Fpretzelduck","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fryanhaticus%2Fpretzelduck","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryanhaticus%2Fpretzelduck/lists"}