Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tinybirdco/mockingbird
Mockingbird is a mock streaming data generator
https://github.com/tinybirdco/mockingbird
data-generation data-generator fakerjs generator http kafka streaming-data tinybird typescript
Last synced: about 6 hours ago
JSON representation
Mockingbird is a mock streaming data generator
- Host: GitHub
- URL: https://github.com/tinybirdco/mockingbird
- Owner: tinybirdco
- License: apache-2.0
- Created: 2023-01-13T10:59:13.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-12-22T13:23:20.000Z (19 days ago)
- Last Synced: 2025-01-02T05:44:16.885Z (8 days ago)
- Topics: data-generation, data-generator, fakerjs, generator, http, kafka, streaming-data, tinybird, typescript
- Language: TypeScript
- Homepage: https://mockingbird.tinybird.co
- Size: 2.39 MB
- Stars: 109
- Watchers: 6
- Forks: 13
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
![Mockingbird Logo](assets/logo/logo_white.png)
# Mockingbird
Mockingbird is mock streaming data generator built by [Tinybird](https://tinybird.co).
It can be used as a library in other projects, or on its own via the UI or CLI.
Mockingbird can send data to any downstream HTTP endpoint through Destination plugins. If you don't see a Destination that you want, feel free to request it or contribute it!
If you simply want to use the Web UI, you use the hosted one here: [https://mockingbird.tinybird.co/](https://mockingbird.tinybird.co/)
## Docs
Find the docs at [https://mockingbird.tinybird.co/docs](https://mockingbird.tinybird.co/docs)
## Usage
### Web UI
The Web UI provides and easy to use, guided experience. It should be pretty easy to get started with, so give it a go!
If you need help with defining schemas, or configuring particular Destinations, you can find more complete documentation [here](https://mockingbird.tinybird.co/docs).
### Passing params in the URL
If you want to re-use configurations from a previous session, you can simply save the URL. All settings are saved as parameters in the URL, so you can re-use and share configs with your team. For example: [http://localhost:3000/?schema=z_sales&eps=1&host=gcp_europe_west3&datasource=sales_dg&token=p.eyJ1IjogIjg4Nzk5NGUxLWZmNmMtNGUyMi1iZTg5LTNlYzBmNmRmMzlkZCIsICJpZCI6ICIwN2RlZThhMS0wNGMzLTQ4OTQtYmQxNi05ZTlkMmM3ZWRhMTgifQ.p_N4EETK7dbxOgHtugAUue3BUWwyGHT461Ha8P-d3Go](http://localhost:3000/?schema=z_sales&eps=1&host=gcp_europe_west3&datasource=sales_dg&token=p.eyJ1IjogIjg4Nzk5NGUxLWZmNmMtNGUyMi1iZTg5LTNlYzBmNmRmMzlkZCIsICJpZCI6ICIwN2RlZThhMS0wNGMzLTQ4OTQtYmQxNi05ZTlkMmM3ZWRhMTgifQ.p_N4EETK7dbxOgHtugAUue3BUWwyGHT461Ha8P-d3Go)
**Warning**: all settings are saved in the URL including senstive field such as tokens & passwords! This is helpful in many occasions for demos, POCs and tests where these credentials are short-lived and disposable - but take care if you are using credentials that must not be shared!
### CLI
Mockingbird is available as a standalone CLI.
Install the CLI with:
```
npm install @tinybirdco/mockingbird-cli
```Here is an example of sending data to the Tinybird Events API:
```sh
> mockingbird-cli tinybird
--schema schema.json \
--datasource "my_data_source" \
--token "e.Pdjdbfsbhksd...." \
--endpoint gcp_europe_west3 \
--eps 50 \
--limit 200
```## Contributing
All contributions are welcome! We encourages individuals & commerical vendors to contribute to Mockingbird to build a data generator that works for everyone.
The repository has the following structure:
```bash
├── apps
│ ├── cli
│ ├── docs
│ └── web
└── packages
└── mockingbird
```### Generator
The core Mockingbird generator is under `./packages/mockingbird`. All new Data Types, Schemas and Destinations are added here.
The generator is written in TypeScript & uses [Faker.js](https://fakerjs.dev/) under the hood to power much of the fake data generation. Custom Data Types are added on-top of Faker to supplement where needed.
#### Adding new Data Types
DataTypes are defined in [/packages/mockingbird/src/extendedFaker.ts](./packages/mockingbird/src/extendedFaker.ts).
To add a new Data Type, add a new item to the `mockingbirdModule` object.
They key of the item will become the name of the Data Type. Types added to this module are automatically added to the `mockingbird` namespace, meaning that they are referenced in schemas like `mockingbird.myTypeName`, this avoids clashes with Faker.js types.
The value of the item must be a function that returns the desired value. The function can have 1 optional parameter, which allows the function to accept incoming parameters.
For example, a custom Data Type that takes no input params:
```javascript
latitudeString: () => faker.address.latitude().toString(),
```A custom Data Type that accepts incoming parameters:
```javascript
pick: (params: { values: unknown[] }) =>
params.values[Math.floor(Math.random() * params.values.length)],
```#### Adding new preset schemas
Preset schemas are defined in [/packages/mockingbird/src/schemas/](./packages/mockingbird/src/schemas).
Create a new file in the `./packages/mockingbird/src/schemas/` directory.
The contents of the file should look like this:
```javascript
import { Schema } from "../types";const newSchema: Schema = {
timestamp: {
type: "mockingbird.timestampNow",
},
};export default newSchema;
```Ensure a new schema is exported in [./packages/mockingbird/src/schemas/index.ts](./packages/mockingbird/src/schemas/index.ts)
### CLI
The CLI is under `./apps/cli`
The CLI is Node based and uses [`yargs`](https://github.com/yargs/yargs).
### Web UI
The Web UI is under `./apps/web`
The Web UI is a Next.js application written in TypeScript.
#### Running the local dev server
To run the Mockingbird UI locally, first install Node.js (developed using v18).
Then, use these commands:
```bash
git clone https://github.com/tinybirdco/mockingbird.git
cd mockingbird
npm install
npm run dev
```This will serve both the Web UI & the documentation site locally. By default, the UI is available on `http://localhost:3001` and the docs are on `http://localhost:3000`.
### Docs
The Docs are under `./apps/docs`
The Docs are written in MDX using [Nextra](https://nextra.site/) as a static site generator.
To run the docs locally, see the instructions for the running the [Web UI](#running-the-local-dev-server).