Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/bgadrian/pseudoservice
A (micro) service that generates large amounts of fake data (users, json, yaml, ecommerce ...)
https://github.com/bgadrian/pseudoservice
fake faker go golang json microservice yaml
Last synced: 3 months ago
JSON representation
A (micro) service that generates large amounts of fake data (users, json, yaml, ecommerce ...)
- Host: GitHub
- URL: https://github.com/bgadrian/pseudoservice
- Owner: bgadrian
- License: gpl-3.0
- Created: 2018-10-30T16:16:14.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2018-12-06T11:21:17.000Z (about 6 years ago)
- Last Synced: 2024-06-19T04:11:11.001Z (8 months ago)
- Topics: fake, faker, go, golang, json, microservice, yaml
- Language: Go
- Homepage:
- Size: 3.54 MB
- Stars: 17
- Watchers: 4
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Pseudo service  [](https://goreportcard.com/report/github.com/bgadrian/pseudoservice) [](https://github.com/bgadrian/pseudoservice/issues)
Pseudo random (deterministic) data generator as a (micro) service.
It is a wrapper for the [FastFaker Go package](https://github.com/bgadrian/fastfaker).
#### The problem
The need to generate random (user or other type) data, to mockup, test or stress your own systems.
#### The solution
I've built this project to be used as an internal service, to generate large amounts of fake data.
### Performance
To run your own benchmarks see [call_benchmark.sh](./call_benchmark.sh). For the best performance I recommend the following:
* Count (batch) value: **100**
* concurrent requests: CPUCores * 1.5
#### Benchmarks
Tests done on Pseudoservice 2.0.0 using ApacheBench 2.3 `ab -n 8000 -c 6 -kdq -m GET http://localhost:8080/users/100?token=SECRET42`. The tests ignore any network latency, does not unmarshall the response, reuses HTTP connections (keep-alive) and were made from the local machine (where pseudoservice was running), meaning you will have **at least a 20% lower performance** in a real-world scenario.
* **t2.micro** (~1 CPU), AWS Linux
```bash
Time taken for tests: 7.649 seconds
Complete requests: 8000
Requests per second: 1045.86 [#/sec] (mean)
Time per request: 5.737 [ms] (mean)
Time per request: 0.956 [ms] (mean, across all concurrent requests)
Transfer rate: 45236.55 [Kbytes/sec] received
Translation: 45Mb/s fake data = 104500 fake users/s
```
* **t2.xlarge** (~4 CPU), AWS Linux
```bash
Time taken for tests: 2.825 seconds
Requests per second: 2832.02 [#/sec] (mean)
Time per request: 2.119 [ms] (mean)
Time per request: 0.353 [ms] (mean, across all concurrent requests)
Transfer rate: 122314.75 [Kbytes/sec] received
Translation: 122Mb/s fake data = 283200 users/s
```
* **c5.2xlarge** (~8 CPU), AWS Linux
```bash
Time taken for tests: 2.035 seconds
Total transferred: 353697150 bytes
HTML transferred: 352137150 bytes
Requests per second: 3931.71 [#/sec] (mean)
Time per request: 1.526 [ms] (mean)
Time per request: 0.254 [ms] (mean, across all concurrent requests)
Transfer rate: 169755.12 [Kbytes/sec] received
Translation: 170Mb/s fake data = 393100 users/s
```
### How it works
You run it as a simple HTTP server, in a private network, and the clients can make requests to get random data, based on their custom templates/needs.
Global optional parameters:
* `?seed=42` - if given, the result will be deterministic, as in for all calls the same data will be returned.
* `token=SECRET42` - the APIKEY
* `{count}` - how many results should be generated, integer [1,500]
### Endpoints (data types)
##### /docs
Contains the OpenID/swagger documentation for this API.
##### /health
Returns `200` if the service is available.
##### /custom/{count}?template="Hello ~name~!"
Generate random data based on a given template. Supports any string, including JSON schema.
The template can contains variables like `~name~` or `~email~`. For a full list see the [FastFaker variables](https://github.com/bgadrian/fastfaker/blob/master/TEMPLATE_VARIABLES.md), but instead of `{}` you must use the `~` delimiter (it is more URL friendly).
```bash
#the template is URL encoded: Hello ~name~!
curl -X GET "http://localhost:8080/custom/3?token=SECRET42&seed=42&template=Hello%20~name~%21"
{"results":[
"Hello Jeromy Schmeler!",
"Hello Kim Steuber!",
"Hello Jacky Borer!"
],"seed":42}
#template: {name:"~name~",age:~digit~~digit~}
curl -X GET "http://localhost:8080/custom/3?token=SECRET42&seed=42&template=%22%7Bname%3A%22~name~%22%2Cage%3A~digit~%7D%22"
{"results":[
"\"{name:\"Jeromy Schmeler\",age:53}\"",
"\"{name:\"Dustin Jones\",age:62}\"",
"\"{name:\"Keely Hartmann\",age:12}\""
],"seed":42}
#template: ~country~
curl -X GET "http://localhost:8080/custom/6?token=SECRET42&seed=42&template=~country~"
{"results":[
"Tajikistan",
"Cameroon",
"Cote Divoire",
"Turkmenistan",
"Ethiopia",
"Afghanistan"
],"seed":42}
```
For more examples see the FastFaker [examples](https://github.com/bgadrian/fastfaker/tree/master/example) and [GoDoc](https://godoc.org/github.com/bgadrian/fastfaker/faker#pkg-examples).
##### /users/{count}
Generate random users based on a preconfigured set of properties, with a friend relationship between them.
This endpoint was created to simulate a Social network and test a Graph database: [davidescus/10minFor300Gb](https://github.com/davidescus/10minFor300Gb).
```bash
#fast and random user
curl "http://localhost:8080/api/v1/users/1?token=SECRET42"
{"seed":6124420007038740542,"nextseed":0,"users":[
{
"id":"a4419f4f-f7f9-4c99-a746-5799b281a5d6","
age":209,
"name":"Jonas Borer",
"company":"core frictionless Inc",
"position":"Senior Paradigm Associate",
"email":"[email protected]",
"country":"Yemen"
}
]}
#slower but deterministic call (seed=42)
curl "http://localhost:8080/api/v1/users/1?token=SECRET42&seed=42"
{"seed":42,"nextseed":43,"users":[
{"
id":"538c7f96-b164-4f1b-97bb-9f4bb472e89f",
"age":62,
"name":"Jacky Borer",
"company":"Ameliorated mindshare Inc",
"position":"International Assurance Orchestrator",
"email":"[email protected]",
"country":"Greece"
}
]}
```
### How to install
a). Download the [binaries from a Release](https://github.com/bgadrian/pseudoservice/releases)
```bash
wget https://github.com/bgadrian/pseudoservice/releases/download/v2.0.0/pseudoservice.tar.gz
tar -xzf pseudoservice.tar.gz
./pseudoservice/linux/pseudoservice --api-key=SECRET42 --read-timeout=1s --write-timeout=1s --keep-alive=15s --listen-limit=1024 --max-header-size=3KiB --host=0.0.0.0 --port=8080
```
OR b). Get the [docker image from bgadrian/pseudoservice/](https://hub.docker.com/r/bgadrian/pseudoservice/)
```bash
docker run --name pseudoservice -p 8080:8080 -d -e APIKEY=MYSECRET bgadrian/pseudoservice
curl "http://localhost:8080/api/v1/users/1?token=MYSECRET&seed=42"
```
OR c). Hard way, requires Go 1.11, make and git
```bash
git clone [email protected]:bgadrian/pseudoservice.git
cd pseudoservice
make run
#OR
make build
env PORT=8080 ./build/pseudoservice
```
### How to use
The binary has the following env variables:
* `PORT` - http listening port (8080)
* `APIKEY` - secret string to be served at `?token=SECRET42`
* for more see `pseudoservice --help`
The server accepts `gzip` and it was generated using swagger (open API), to see a full documentation of the service access the [127.0.0.1:8080/docs](http://127.0.0.1:8080/docs) endpoint.
### TODO
All the [issues](https://github.com/bgadrian/pseudoservice/issues)
### Copyright
Bledea Georgescu Adrian https://coder.today