https://github.com/ltbringer/fastest
Watches python code and creates test cases
https://github.com/ltbringer/fastest
Last synced: about 1 year ago
JSON representation
Watches python code and creates test cases
- Host: GitHub
- URL: https://github.com/ltbringer/fastest
- Owner: ltbringer
- License: mit
- Created: 2019-01-13T02:58:21.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2019-03-29T06:37:14.000Z (about 7 years ago)
- Last Synced: 2025-03-25T09:18:47.703Z (about 1 year ago)
- Language: Jupyter Notebook
- Size: 1.04 MB
- Stars: 10
- Watchers: 3
- Forks: 1
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Fastest
Creates unit tests from examples in the docstring and more.
[](https://app.codacy.com/app/AmreshVenugopal/fastest?utm_source=github.com&utm_medium=referral&utm_content=AmreshVenugopal/fastest&utm_campaign=Badge_Grade_Dashboard)
[](https://scrutinizer-ci.com/g/AmreshVenugopal/fastest/)
[](https://coveralls.io/github/AmreshVenugopal/fastest?branch=master)
[](https://travis-ci.org/AmreshVenugopal/fastest)
[](https://pypi.org/project/fastest/)
[](https://pypi.org/project/fastest/)
## Install
```bash
$ pip install fastest
```
## Usage
```bash
$ fastest
```
watches all .py files and creates coverage for entire project.
```bash
$ fastest --path=$(pwd) --source=py_module
```
where `path` is the the project root, and [`source`](https://coverage.readthedocs.io/en/coverage-4.3.4/source.html#source)
is same as the value passed to the command `coverage run -m unittest --source=$source test`
```bash
$ fastest --exclude=dont_check_this_dir/*,these__*.py
```
To exclude files/folders use `--exclude` and the file watcher will ignore them.
The `test/*` folder that `faster` creates is excluded by default.
```bash
$ fastest --poll-duration=10
```
Builds files, runs tests and coverage every `10s`, default = `1s`
Things that happen when you run `python main.py --path=$(pwd)`:
1. Checks for a `test` file at the project root, it creates if it doesn't find one.
2. Watches `.py` files for changes.
3. Creates unittests if a function has examples in its docstrings like so:
```python
# .
# ├──module_a
# ├──module_b
# └── utils.py
#
def add(x, y):
"""
----
examples:
1) add(3, 4) -> 7
"""
return x + y
```
This will create a unittest in the `test` directory, `assertEqual(add(3, 4), 7)`
within `Class test__(self)`
(for the given directory, tree: `Class test_utils_add(self)`)
4. Runs all tests that are created.
5. Creates a coverage report (in html format).
6. Print the link to the coverage reports' index.html.
## How to make best use of Fastest
1. Keep your `functions` light:
- Be paranoid about separation of concerns.
- Too many conditions are a hint that you might need another function.
- Complex loops and `if-else` are not scalable code, a single mistake would
take that tower down and feature additions would involve someone going through
that brain-teaser.
2. Use libraries but wrap them with your own functions. Like: Use `requests` or the inevitable database?
wrap them with your own functions.
- Helps with adding customizations in one place (configuring things like base url, and similar configs)
- Helps mocking so that entire code-base can be unit tested.
3. Docstrings may get outdated if your work pace is too fast to maintain quality documentation.
Now adding examples now would help you create
tests which prevents your descriptions from going stale, **if the tests fail,
probably the documentation needs a second look too**. This is enforced within Fastest, as documentation **IS**
contributing to tests.
## Examples:
1. Allows creation of variables within the docstrings, which includes lambda functions!
```python
def quick_maths(a, b):
"""
----
examples:
@let
a = {
'apples': 3,
'oranges': 4
}
@end
1) quick_maths(a['apples'], a['oranges']) -> 7
----
"""
return a + b
```
2. You can run any valid python code within `@let--@end` blocks.
3. Can include installed modules external to your project.
```python
def current_time():
"""
---
examples:
@need
from datetime import datetime
@end
1) current_time() -> datetime.now()
"""
return datetime.now()
```
4. If types are added to docstring, Fastest will create tests
for checking type of the value returned against empty of arguments.
```python
def chain_strings(str1, str2):
"""
:param str1: str
:param str2: str
:return: str
"""
return str1 + str2
```
Fastest will create a `assertInstanceIs(chain_strings('', ''), str)` for the above snippet.
5. To create an `assertRaises` test-case:
```python
def crashes_sometimes(input_string):
"""
----
examples:
!! crashes_sometimes(None) -> ValueError
----
"""
if not input_string:
raise ValueError
return input_string
```
The syntax marked with `!! crashes_sometimes(None) -> ValueError` handles exceptions
that the code throws
## Fuzzing json apis
*Note: work in progress*
An experimental feature that can test your api's (currently supporting json request bodies only!).
A happy-case example is required and fastest will figure out the rest, it will e-mail the errors to you.
*example-case:*
```js
router.post('/test', function(req, res, next) {
const { messageObject } = req.body;
res.json({ prop: messageObject.message });
});
```
The above is a snippet from an express-server's post request. It expects a `messageObject` within the `req.body`.
This api returns a response containing `messageObject.message` without ever checking if it is present.
Hate mistakes like these?
Make sure to check out the [fuzz.log](https://github.com/AmreshVenugopal/fastest/blob/master/fuzz.log).
```text
url: /test
request_body: {'messageObject': None}
response:
Cannot read property 'message' of null
TypeError: Cannot read property 'message' of null
at /home/ltbringer/programming/js/sample-json-api/routes/index.js:11:34
at Layer.handle [as handle_request] (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/layer.js:95:5)
at next (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/route.js:137:13)
at Route.dispatch (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/layer.js:95:5)
at /home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/index.js:281:22
at Function.process_params (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/index.js:335:12)
at next (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/index.js:275:10)
at Function.handle (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/index.js:174:3)
at router (/home/ltbringer/programming/js/sample-json-api/node_modules/express/lib/router/index.js:47:12)
status_code: 500
```
This feature is still under progress because:
- Signatures that have resulted in errors need to be avoided.
- A simple to use format in which server request-bodies and other config should be shared is not yet formalized.
- Error-Grouping: ability to detect when multiple signatures are leading to similar error or affects same area of code is not present.
- Optimizing happy-case: signatures that don't cause errors and an appropriate response is returned by the server under test, need not be sent back to the server again.
# Goals for Fastest
- [x] Help maintaining tests, code-coverage and documentation.
- [ ] Help with performance issues within code.
- [ ] Provide testability score for code.
- [x] Test functions for auto-generated inputs where the code would crash.
Fastest uses itself for creating tests and manages a 100% on the coverage!