Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/keploy/keploy
Unit and Integration Test generation for Developers. Generate tests and stubs for your application that actually work!
https://github.com/keploy/keploy
api api-testing go go-test golang java-test mock mock-data-generator mock-framework productivity test-automation test-automation-framework test-generation testing testing-library testing-tools unit-test unit-testing unit-testing-framework
Last synced: 3 days ago
JSON representation
Unit and Integration Test generation for Developers. Generate tests and stubs for your application that actually work!
- Host: GitHub
- URL: https://github.com/keploy/keploy
- Owner: keploy
- License: apache-2.0
- Created: 2022-01-19T10:40:31.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2024-10-29T09:16:06.000Z (about 1 month ago)
- Last Synced: 2024-10-29T09:25:25.262Z (about 1 month ago)
- Topics: api, api-testing, go, go-test, golang, java-test, mock, mock-data-generator, mock-framework, productivity, test-automation, test-automation-framework, test-generation, testing, testing-library, testing-tools, unit-test, unit-testing, unit-testing-framework
- Language: Go
- Homepage: https://keploy.io
- Size: 44.8 MB
- Stars: 5,011
- Watchers: 31
- Forks: 467
- Open Issues: 169
-
Metadata Files:
- Readme: README-UnitGen.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Citation: CITATION.cff
- Security: SECURITY.md
Awesome Lists containing this project
- awesome-testing - Keploy - API Testing Platform that automatically generates unit test cases along with dependency mocks(test data) from API calls. (Software / Make your life easier)
- awesome-golang-repositories - keploy
- trackawesomelist - keploy (⭐3.9k) - Generate Testcase and Data Mocks from API calls automatically. (Recently Updated / [Sep 15, 2024](/content/2024/09/15/README.md))
- awesome-go - keploy - Generate Testcase and Data Mocks from API calls automatically. Stars:`5.1K`. (Testing / Testing Frameworks)
README
⚡️ Generate unit tests with LLMs, that actually works ⚡️
🌟 The must-have tool for developers in the AI-Gen era 🌟
---
[](https://join.slack.com/t/keploy/shared_invite/zt-2poflru6f-_VAuvQfCBT8fDWv1WwSbkw)
[](https://www.linkedin.com/company/keploy/)
[](https://twitter.com/Keployio)
---
Keploy-gen uses LLMs to understand code semantics and generates meaningful **unit tests**. It's inspired by the [Automated Unit Test Improvement using LLM at Meta](https://arxiv.org/pdf/2402.09171).
### Objectives
- **Automate unit test generation (UTG)**: Quickly generate comprehensive unit tests and reduce the redundant manual effort.
- **Improve edge cases**: Extend and improve the scope of tests to cover more complex scenarios that are often missed manually.
- **Boost test coverage**: As codebase grows, ensuring exhaustive coverage should become feasible.
## Core Components
| **Phase** | **Activities** | **Tools/Technologies** |
| ----------------------------- | ------------------------------------------------------------------------------------------------- | ---------------------------------------- |
| **Code Analysis** | Analyze the code structure and dependencies. | Static analysis tools, LLMs |
| **Prompt Engineering** | Generation of targeted prompts to guide the LLM in producing relevant tests. | LLMs, Custom scripts |
| **Iterative Test Refinement** | Cyclic process of refining tests by running them, analyzing coverage, and incorporating feedback. | Testing frameworks (e.g., JUnit, pytest) |
### Process Overview
Referred from [Meta's research](https://arxiv.org/pdf/2402.09171), TestGen-LLM top level architecture.
## Prerequisites
**AI model Setup** - Set the environment variable **API_KEY**.
```
export API_KEY=xxxx
```
**API_KEY** can be from either of one these:
- **OpenAI's GPT-4o** directly **[preferred]**.
- Alternative LLMs via [litellm](https://github.com/BerriAI/litellm?tab=readme-ov-file#quick-start-proxy---cli).
- Azure OpenAI
## Installation
Install Keploy locally by running the following command:
#### ➡ Linux/Mac
```shell
curl --silent -O -L https://keploy.io/install.sh && source install.sh
```
#### ➡ Windows
- [Download](https://github.com/keploy/keploy/releases/latest/download/keploy_windows_amd64.tar.gz) and **move the keploy.exe file** to `C:\Windows\System32`
###  ➡ Running with Node.js/TypeScript applications
- Ensure you've set the API key, as mentioned in pre-requisites above:
```shell
export API_KEY=xxxx
```
- Ensure **Cobertura** formatted coverage reports, edit `jest.config.js` or `package.json`:
```json
// package.json
"jest": {
"coverageReporters": ["text", "cobertura"],
}
```
or
```javascript
// jest.config.js
module.exports = {
coverageReporters: ["text", "cobertura"],
};
```
#### Generating Unit Tests
- Run the following command in the root of your application.
- **For Single Test File:** If you prefer to test a smaller section of your application or to control costs, consider generating tests for a single source and its corresponding test file:
```shell
keploy gen --sourceFilePath="" --testFilePath="" --testCommand="npm test" --coverageReportPath=""
```
- **For Entire Application** use the following command to generate tests across:
⚠️ **Warning:** Executing this command will generate unit tests for all files in the application. Depending on the size of the codebase, this process may take between 20 minutes to an hour and will incur costs related to LLM usage.
```bash
keploy gen --testCommand="npm test" --testDir="test" --coverageReportPath=""
```
🎉 You should see improved test cases and code-coverage. ✅ Enjoy coding with enhanced unit test coverage! 🫰
###  → Running with Golang applications
- Ensure you've set the API key, as mentioned in pre-requisites above:
```shell
export API_KEY=xxxx
```
- To ensure **Cobertura** formatted coverage reports, add:
```bash
go install github.com/axw/gocov/[email protected]
go install github.com/AlekSi/[email protected]
```
#### Generating Unit Tests
- Run the following command in the root of your application.
- **For Single Test File:** If you prefer to test a smaller section of your application or to control costs, consider generating tests for a single source and its corresponding test file:
```shell
keploy gen --sourceFilePath="" --testFilePath="" --testCommand="go test -v ./... -coverprofile=coverage.out && gocov convert coverage.out | gocov-xml > coverage.xml" --coverageReportPath=""
```
- **For Entire Application** use the following command to generate tests across:
⚠️ **Warning:** Executing this command will generate unit tests for all files in the application. Depending on the size of the codebase, this process may take between 20 minutes to an hour and will incur costs related to LLM usage.
```bash
keploy gen --testDir="." --testCommand="go test -v ./... -coverprofile=coverage.out && gocov convert coverage.out | gocov-xml > coverage.xml" --coverageReportPath=""
```
🎉 You should see improved test cases and code-coverage. ✅ Enjoy coding with enhanced unit test coverage! 🫰
### → Setup for Other Languages
- Ensure you've set the API key, as mentioned in pre-requisites above:
```shell
export API_KEY=xxxx
```
- Ensure that your unit test report format is **Cobertura**(it's very common).
- Generate tests using keploy-gen:
```bash
keploy gen --sourceFilePath="" --testFilePath="" --testCommand="" --coverageReportPath=""
```
## Configuration
Configure Keploy using command-line flags:
```bash
--sourceFilePath ""
--testFilePath ""
--coverageReportPath "coverage.xml"
--testCommand ""
--coverageFormat "cobertura"
--expectedCoverage 100
--maxIterations 5
--testDir ""
--llmBaseUrl "https://api.openai.com/v1"
--model "gpt-4o"
--llmApiVersion "
```
- `sourceFilePath`: Path to the source file for which tests are to be generated.
- `testFilePath`: Path where the generated tests will be saved.
- `coverageReportPath`: Path to generate the coverage report.
- `testCommand` (required): Command to execute tests and generate the coverage report.
- `coverageFormat`: Type of the coverage report (default "cobertura").
- `expectedCoverage`: Desired coverage percentage (default 100%).
- `maxIterations`: Maximum number of iterations for refining tests (default 5).
- `testDir`: Directory where tests will be written.
- `llmBaseUrl`: Base url of the llm.
- `model`: Specifies the AI model to use (default "gpt-4o").
- `llmApiVersion`: API version of the llm if any (default "")
# Frequently Asked Questions
1. What is Keploy's Unit Test Generator (UTG)?
- Keploy's UTG automates the creation of unit tests based on code semantics, enhancing test coverage and reliability.
2. Does Keploy send your private data to any cloud server for test generation?
- No, Keploy does not send any user code to remote systems, except when using the unit test generation feature. When using the UT gen feature, only the source code and the unit test code will be sent to the Large Language Model (LLM) you are using. By default, Keploy uses - litellm to support vast number of LLM backends. Yes, if your organization has its own LLM(a private one), you can use it with Keploy. This ensures that data is not sent to any external systems.
3. How does Keploy contribute to improving unit test coverage?
- By providing a zero code platform for automated testing, Keploy empowers developers to scale up their unit test coverage without extensive coding knowledge. This integration enhances testing reports, ultimately boosting confidence in the product's quality.
4. Is Keploy cost-effective for automated unit testing?
- Yes, Keploy optimizes costs by automating repetitive testing tasks and improving overall test efficiency.
5. How does Keploy generate coverage reports?
- Keploy generates detailed Cobertura format reports, offering insights into test effectiveness and code quality.
6. Can Keploy handle large codebases efficiently?
- Yes, Keploy is designed to handle large codebases efficiently, though processing time may vary based on project size and complexity.
# 🙋🏻♀️ Questions? 🙋🏻♂️
Reach out to us. We're here to answer!
[](https://join.slack.com/t/keploy/shared_invite/zt-2poflru6f-_VAuvQfCBT8fDWv1WwSbkw)
[](https://www.linkedin.com/company/keploy/)
[](https://www.youtube.com/channel/UC6OTg7F4o0WkmNtSoob34lg)
[](https://twitter.com/Keployio)
# 📝 Sample QuickStarts
-  : Try a unit-gen on [Mux-SQL](https://github.com/keploy/samples-go/tree/main/mux-sql#create-unit-testcase-with-keploy) app
-  : Try a unit-gen on [Express-Mongoose](https://github.com/keploy/samples-typescript/tree/main/express-mongoose#create-unit-testcase-with-keploy) app
## 🌐 Language Support
Other language may be supported, we've not tested them yet. If your **coverage reports** are of **Cobertura format** then you should be able to use keploy-gen in any language.
## Dev Support
Keploy-gen is not just a project but an attempt to make developers life easier testing applications.
It aims to simplify the creation and maintenance of tests, ensuring high coverage, and adapts to the complexity of modern software development.
#### Prompt Generation
Referred from [Meta's research](https://arxiv.org/pdf/2402.09171), the four primary prompts used in the deployment for the December 2023 Instagram and Facebook app test-a-thons
| Prompt Name | Template |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| extend_test | Here is a Kotlin unit test class: {`existing_test_class`}. Write an extended version of the test class that includes additional tests to cover some extra corner cases. |
| extend_coverage | Here is a Kotlin unit test class and the class that it tests: {`existing_test_class`} {`class_under_test`}. Write an extended version of the test class that includes additional unit tests that will increase the test coverage of the class under test. |
| corner_cases | Here is a Kotlin unit test class and the class that it tests: {`existing_test_class`} {`class_under_test`}. Write an extended version of the test class that includes additional unit tests that will cover corner cases missed by the original and will increase the test coverage of the class under test. |
| statement_to_complete | Here is a Kotlin class under test {`class_under_test`} This class under test can be tested with this Kotlin unit test class {`existing_test_class`}. Here is an extended version of the unit test class that includes additional unit test cases that will cover methods, edge cases, corner cases, and other features of the class under test that were missed by the original unit test class: |
Limitation: This project currently doesn't generate quality fresh tests if there are no existing tests to learn from.
Enjoy coding with enhanced unit test coverage! 🫰