{"id":13512885,"url":"https://github.com/Pythagora-io/pythagora","last_synced_at":"2025-03-31T00:30:43.221Z","repository":{"id":65776176,"uuid":"591742672","full_name":"Pythagora-io/pythagora","owner":"Pythagora-io","description":"Generate automated tests for your Node.js app via LLMs without developers having to write a single line of code.","archived":false,"fork":false,"pushed_at":"2025-03-24T23:03:48.000Z","size":104571,"stargazers_count":1767,"open_issues_count":23,"forks_count":98,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-03-25T09:50:35.037Z","etag":null,"topics":["api-testing","api-testing-framework","automated-testing","database-testing","developer-tools","integration-testing","nodejs","test-coverage","testing"],"latest_commit_sha":null,"homepage":"https://pythagora.ai","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Pythagora-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-21T18:13:27.000Z","updated_at":"2025-03-22T19:27:46.000Z","dependencies_parsed_at":"2024-01-19T00:24:21.134Z","dependency_job_id":"442ede30-9754-406a-90cf-52f64edcb832","html_url":"https://github.com/Pythagora-io/pythagora","commit_stats":{"total_commits":457,"total_committers":7,"mean_commits":65.28571428571429,"dds":0.5733041575492341,"last_synced_commit":"eaecd1bba4ab3cab4f02cc1cad35abc2bc50d389"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pythagora-io%2Fpythagora","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pythagora-io%2Fpythagora/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pythagora-io%2Fpythagora/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Pythagora-io%2Fpythagora/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Pythagora-io","download_url":"https://codeload.github.com/Pythagora-io/pythagora/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246399816,"owners_count":20770907,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-testing","api-testing-framework","automated-testing","database-testing","developer-tools","integration-testing","nodejs","test-coverage","testing"],"created_at":"2024-08-01T04:00:33.807Z","updated_at":"2025-03-31T00:30:42.726Z","avatar_url":"https://github.com/Pythagora-io.png","language":"JavaScript","readme":"\u003ch1 align=\"center\"\u003eThis repo is deprecated - we're working on \u003ca href=\"https://github.com/Pythagora-io/gpt-pilot\"\u003eGPT Pilot\u003c/h1\u003e\n\n\n\u003cp align=center\u003e\n  \u003cpicture\u003e\n    \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://user-images.githubusercontent.com/10895136/228003796-7e3319ad-f0b1-4da9-a2d0-6cf67ccc7a32.png\"\u003e\n    \u003cimg height=\"200px\" alt=\"Pythagora Logo\" src=\"https://user-images.githubusercontent.com/10895136/228003796-7e3319ad-f0b1-4da9-a2d0-6cf67ccc7a32.png\"\u003e\n  \u003c/picture\u003e\n\u003c/p\u003e\n\u003ch2 align=\"center\"\u003ePythagora is on a mission to make automated tests\u003cbr\u003e🤖 fully autonomous 🤖\u003c/h2\u003e\n\u003ch3 align=\"center\"\u003eJust run one command and watch the tests being created with GPT-4\u003c/h3\u003e\n\u003cbr\u003e\n\nThe following details are for generating unit tests. To view the docs on how to generate **integration tests**, click [here](./src/docs/integration/README.md).\n\n\u003cbr\u003e\n\n# \u003cimg src=\"https://s3.amazonaws.com/assets.pythagora.ai/vscode/vscode_icon.png\" alt=\"Visual Studio Code Logo\" width=\"24\" height=\"24\"\u003e Visual Studio Code Extension\n\nIf you want to try out Pythagora using Visual Studio Code extension you can download it \u003ca href=\"https://marketplace.visualstudio.com/items?itemName=PythagoraTechnologies.pythagora-vscode-extension\"\u003ehere\u003c/a\u003e.\n\n# 🏃💨️ Quickstart\n\nTo install Pythagora run:\n```bash\nnpm i pythagora --save-dev\n```\nThen, [add your API key](#%EF%B8%8F-config) and you're ready to get tests generated. After that, just run the following command from the root directory of your repo:\n\n```bash\nnpx pythagora --unit-tests --func \u003cFUNCTION_NAME\u003e\n```\n\nWhere `\u003cFUNCTION_NAME\u003e` is the name of the function you want to generate unit tests for. Just make sure that your function is exported from a file. You can see other options like generating tests for multiple files or folders [below in the Options section](#-options).\n\n\u003cbr\u003e\u003cbr\u003e\nIf you wish to expand your current test suite with more tests to get better code coverage you can run:\n\n```bash\nnpx pythagora --expand-unit-tests --path \u003cPATH_TO_YOUR_TEST_SUITE\u003e\n```\nfor more details on expanding existing tests see [below in the Expanding existing tests section](#-expand-existing-tests).\n\n\u003cbr\u003e\u003cbr\u003e\n\n**NOTE:** on Windows make sure to run all commands using `Git Bash` and not `Power Shell` or anything similiar\n\n\u003cbr\u003e\n\n# 🎞 Demo\n\nHere are some demo videos that can help you get started.\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://youtu.be/NNd08XgFFw4\"\u003e\u003cimg src=\"https://github-production-user-asset-6210df.s3.amazonaws.com/10895136/244031887-02f19eb9-dba1-4e62-a670-744c7d3423ae.gif\" alt=\"Pythagora Alpha Demo\"\u003e\u003c/a\u003e\n\u003c/div\u003e\n\u003cp align=center\u003e\n  \u003ca target=\"_blank\" href=\"https://youtu.be/NNd08XgFFw4\"\u003ePythagora Unit Tests Demo (2 min)\u003c/a\u003e\n\u003c/p\u003e\n\u003cbr\u003e\n\n# 🔎 Examples\n\nHere are examples of open sourced repositories that we forked and created tests with Pythagora so you can easily see it in action. \n\n- [Lodash](https://github.com/Pythagora-io/pythagora-demo-lodash)\n  - 📝 1604 tests generated\n  - 🐞 11 bugs found (1 edge case and 10 bugs)\n  - ⏳️ 4 hour run time\n\n  ![lodash pythagora tests results](https://github.com/Pythagora-io/pythagora/assets/10895136/c0a2a589-e3ef-4812-9ea2-545307fd1a1d)\n\n\n- [node-fs-extra](https://github.com/Pythagora-io/pythagora-demo-node-fs-extra)\n  - 📝 98 tests generated\n  - 🐞 2 bugs found\n  - ⏳️ 30 minutes run time\n \n  ![node-fs-extra pythagora tests results](https://github.com/Pythagora-io/pythagora/assets/10895136/a3d8ec9e-2881-4b97-9d95-57440c1932e4)\n\n\u003cbr\u003e\n\n# 🔬 How does it work?\nWhen Pythagora generates unit tests, it uses the following approach:\n1. Find the function you want to test\n2. Find all the functions that are called from within that function\n  - This is done with AST (Abstract Syntax Tree) parsing\n3. Send the function you want to test and all the related functions to the Pythagora server which then generates the unit tests with GPT-4\n  - the Pythagora server is open sourced as well [here](https://github.com/Pythagora-io/api)\n  - You can find the prompts [in this folder](https://github.com/Pythagora-io/api/tree/main/prompts) on the Pythagora server\n    \n\u003cbr\u003e\n\n# 📈 Expand existing tests\nIf you already have generated tests for your codebase but you just want to increase your code coverage or cover more edge cases, simply run:\n\n```bash\nnpx pythagora --expand-unit-tests --path \u003cPATH_TO_YOUR_TEST_SUITE\u003e\n```\nWhen running command `PATH_TO_YOUR_TEST_SUITE` can be path to a single test file or to a folder and all test files inside of that folder will be processed and expanded.\n\nThat's all, enjoy your new code coverage!\n\n# 📖 Options\n- To generate unit tests for **one single function**, run:\n\n    ```bash\n    npx pythagora --unit-tests --func \u003cFUNCTION_NAME\u003e\n    ```\n\n- To generate unit tests for **one single function** in a specific file, run:\n\n    ```bash\n    npx pythagora --unit-tests --func \u003cFUNCTION_NAME\u003e --path ./path/to/file.js\n    ```\n\n- To generate unit tests for **all functions in a file**, run:\n\n    ```bash\n    npx pythagora --unit-tests --path ./path/to/file.js\n    ``` \n\n- To generate unit tests for **all functions in all files in a folder**, run:\n\n    ```bash\n    npx pythagora --unit-tests --path ./path/to/folder/\n    ```\n\n\u003cbr\u003e\n\n# ⚙️ Config\nPythagora uses GPT-4 to generate tests so you either need to have OpenAI API Key or Pythagora API Key. You can get your [Pythagora API Key here](https://mailchi.mp/f4f4d7270a7a/api-waitlist) or [OpenAI API Key here](https://platform.openai.com/account/api-keys). Once you have it, add it to Pythagora with:\n```bash\nnpx pythagora --config --pythagora-api-key \u003cAPI_KEY\u003e\n```\nor\n```bash\nnpx pythagora --config --openai-api-key \u003cAPI_KEY\u003e\n```\n\u003cbr\u003e\n\n# ▶️ How to run unit tests\nTo run the generated tests, you can simply run\n```bash\nnpx jest ./pythagora_tests/\n```\nor to run tests from a specific file or a folder, run `npx jest \u003cPATH_TO_FILE_OR_FOLDER\u003e`. Currently, Pythagora supports only generating Jest tests but if you would like it to generate tests in other frameworks, let us know at [hi@pythagora.ai](mailto:hi@pythagora.ai).\n\n\u003cbr\u003e\n\n# 📌️ Notes\n\n- The best unit tests that Pythagora generates are the ones that are standalone functions (eg. helpers). Basically, the parts of the code that actually can be unit tested. For example, take a look at this [Pythagora file](./src/utils/common.js) - it contains helper functions that are a perfect candidate for unit tests. When we ran `npx pythagora --unit-tests --path ./src/utils/common.js` - it generated 145 tests from which only 17 failed. What is amazing is that only 6 tests failed because they were incorrectly written and the other 11 tests caught bugs in the code itself. You can view these tests [here](./pythagora_tests/unit/src/utils/common/).\n- We don't store any of your code on our servers. However, the code is being sent to GPT and hence OpenAI. Here is their [privacy policy](https://openai.com/policies/privacy-policy).\n- a function you want to generate tests for needs to be exported from the file. For example, if you have a file like this:\n  ```javascript\n  function mongoObjToJson(originalObj) {\n      ...\n  }\n  \n  module.exports = {\n      mongoObjToJson\n  };\n  ```\n  \n  Then, to generate unit tests for the `mongoObjToJson` function, you can run:\n  ```bash\n  npx pythagora --unit-tests --func mongoObjToJson\n  ```\n\n\u003cbr\u003e\n\n# 🤔️ FAQ\n\n- **How accurate are these tests?**\n  - The best unit tests that Pythagora generates are the ones that are standalone functions. Basically, the parts of the code that actually can be unit tested. For example, take a look at this [Pythagora file](./src/utils/common.js) - it contains helper functions that are a perfect candidate for unit tests. When we ran `npx pythagora --unit-tests --path ./src/utils/common.js` - it generated 145 tests from which only 17 failed. What is amazing is that only 6 tests failed because they were incorrectly written and the other 11 tests caught bugs in the code itself. You can view these tests [here](./pythagora_tests/unit/src/utils/common/).\n  - Here are a couple of observations we've made while testing Pythagora:\n    1. It does a great job at testing edge cases. For many repos we created tests for, the tests found bugs right away by testing edge cases.\n    2. It works best for testing standalone helper functions. For example, we tried generating tests for the Lodash repo and it create 1000 tests from which only 40 needed additional review. For other, non standalone functions, we're planning to combine recordings from integration tests to generate proper mocks so that should expand Pythagora's test palette.\n    3. It's definitely not perfect but the tests it created I wanted to keep and commit them. So, I encourage you to try it out and see how it works for you. If you do that, please let us know via [email](mailto:hi@pythagora.ai) or [Discord](https://discord.gg/npC5TAfj6e). We're super excited to hear how it went for you.\n\u003cbr\u003e\u003cbr\u003e\n- **Should I review generated tests?**\n  - Absolutely. As mentioned above, some tests might be incorrectly written so it's best for you to review all tests before committing them. Nevertheless, I think this will save you a lot of time and will help you think about your code in a different way.\n\u003cbr\u003e\u003cbr\u003e\n- **Tests help me think about my code - I don't want to generate them automatically**\n    - That's the best thing about Pythagora - it actually does help you think about the code. Just, you don't need to spend time writing tests. This happened to us, who created Pythagora - we coded it as fast as possible but when we added unit test generation, we realized that it cannot create tests for some functions. So, we refactored the code and made it more modular so that unit tests can be generated for it. \n\u003cbr\u003e\u003cbr\u003e\n- **Is Pythagora limited to a specific programming language or framework?**\n  - Pythagora primarily generates unit tests for JavaScript code. However, it's designed to work with code written in JavaScript, TypeScript, and similar languages. If you'd like to see support for other languages or frameworks, please let us know at hi@pythagora.ai.\n\u003cbr\u003e\u003cbr\u003e\n\n- **Can Pythagora generate integration tests as well?**\n  - Pythagora is currently focused on generating unit tests. For generating integration tests, you might need to combine the recordings from integration tests to generate proper mocks. We are actively exploring options to expand its capabilities in the future.\n\u003cbr\u003e\u003cbr\u003e\n\n- **Is Pythagora compatible with all JavaScript testing frameworks?**\n  - Currently, Pythagora generates tests using the Jest testing framework. While we are open to expanding compatibility to other testing frameworks, Jest is the primary framework supported at the moment. If you have a specific framework in mind, feel free to share your suggestions with us.\n\u003cbr\u003e\u003cbr\u003e\n\n- **How does Pythagora handle sensitive or proprietary code?**\n  - Pythagora doesn't store your code on its servers, but it sends code to GPT and OpenAI for test generation. It's essential to review the generated tests, especially if your code contains sensitive or proprietary information, before committing them to your repository. Be cautious when using Pythagora with sensitive code.\n\u003cbr\u003e\u003cbr\u003e\n\n- **Is Pythagora suitable for all types of projects?**\n  - Pythagora works best for projects with well-structured code and standalone functions (such as helper functions). It excels at generating tests for these types of code. For more complex or non-standalone functions, manual review and modifications may be necessary.\n\n\n\n# 🏁 Alpha version\nThis is an alpha version of Pythagora. To get an update about the beta release or to give a \u003cb\u003esuggestion on tech (framework / database) you want Pythagora to support\u003c/b\u003e you can 👉 \u003ca href=\"http://eepurl.com/ikg_nT\" target=\"_blank\"\u003eadd your email / comment here\u003c/a\u003e 👈 .\n\u003cbr\u003e\n\n# 🔗 Connect with us\n💬 Join the discussion on \u003ca href=\"https://discord.gg/npC5TAfj6e\" target=\"_blank\"\u003eour Discord server\u003c/a\u003e.\n\u003cbr\u003e\u003cbr\u003e\n📨 Get updates on new features and beta release by \u003ca href=\"http://eepurl.com/ikg_nT\" target=\"_blank\"\u003eadding your email here\u003c/a\u003e.\n\u003cbr\u003e\u003cbr\u003e\n🌟 As an open source tool, it would mean the world to us if you starred the Pythagora repo 🌟\n\u003cbr\u003e\u003cbr\u003e\n\u003cbr\u003e\u003cbr\u003e\n\n\u003cbr\u003e\u003cbr\u003e\n\n","funding_links":[],"categories":["Code Quality","JavaScript"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPythagora-io%2Fpythagora","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPythagora-io%2Fpythagora","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPythagora-io%2Fpythagora/lists"}