{"id":49341409,"url":"https://github.com/redborder/synthetic-producer","last_synced_at":"2026-04-27T04:04:07.008Z","repository":{"id":2202390,"uuid":"41410418","full_name":"redBorder/synthetic-producer","owner":"redBorder","description":"Generates random JSON messages and sends them to Apache Kafka","archived":false,"fork":false,"pushed_at":"2024-10-24T15:09:49.000Z","size":177,"stargazers_count":7,"open_issues_count":5,"forks_count":8,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-10-25T06:12:56.829Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/redBorder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-08-26T07:10:38.000Z","updated_at":"2024-10-01T13:42:16.000Z","dependencies_parsed_at":"2024-11-16T14:16:34.710Z","dependency_job_id":null,"html_url":"https://github.com/redBorder/synthetic-producer","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/redBorder/synthetic-producer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redBorder%2Fsynthetic-producer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redBorder%2Fsynthetic-producer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redBorder%2Fsynthetic-producer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redBorder%2Fsynthetic-producer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/redBorder","download_url":"https://codeload.github.com/redBorder/synthetic-producer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/redBorder%2Fsynthetic-producer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32321945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-26T23:26:28.701Z","status":"online","status_checked_at":"2026-04-27T02:00:06.769Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-27T04:04:05.647Z","updated_at":"2026-04-27T04:04:07.001Z","avatar_url":"https://github.com/redBorder.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# synthetic-producer\n[![Codacy Badge](https://api.codacy.com/project/badge/grade/75478e4c803148978811f05f732f3f3a)](https://www.codacy.com/app/redBorder/synthetic-producer)\nGenerates random JSON messages and sends them to Apache Kafka.\nIt uses Apache Zookeeper to discover the available brokers and partitions for a given topic.\n\nUsage:\n\n```bash\n$ java -jar target/synthetic-producer-1.4.0-selfcontained.jar\nusage: java -jar synthetic-producer.jar\n -c,--config \u003carg\u003e      config file path\n -h,--help              show this help\n -r,--rate \u003carg\u003e        messages rate per second\n -t,--threads \u003carg\u003e     number of producer threads\n -z,--zookeeper \u003carg\u003e   zookeeper connect string\n```\n\n## Config file\n\nThe config file specifies the Zookeeper connection string, the rate of \nmessages generated, the threads to use, the topic where you want to produce,\nand the messages' schema.\n\nFor example, take a look at the following config:\n\n```yaml\ntopic: testTopic\npartitionKey: myPartitionKeyField\nfields:\n  myConstant:\n    type: constant\n    value: 'Hello, World!'\n  randomNumber:\n    type: integer\n    min: 0\n    max: 100\n```\n\nUsing a file like that will make the producer will connect to Zookeeper at \"localhost:2181\" in order create\ntwo producing threads, and send messages to the topic \"testTopic\" at a constant rate of 100 msgs per\nsecond, which would look like the following:\n\n```json\n{ \"myConstant\": \"Hello, World!\", \"randomNumber\": 4}\n{ \"myConstant\": \"Hello, World!\", \"randomNumber\": 12}\n{ \"myConstant\": \"Hello, World!\", \"randomNumber\": 94}\n{ \"myConstant\": \"Hello, World!\", \"randomNumber\": 33}\n```\n\nYou can find a full example of config file in [this file](https://github.com/redBorder/synthetic-producer/blob/master/configProducer.yml)\n\n## Fields types\n\nYou can use a variety of different field types to generate your random message.\nSome types accepts one or more parameters that modify the field randomness.\nFor example, the constant type always generates the same value for a field, but the integer\ntype generates a random number between 'min' and 'max' for each field.\n\nThe available types, with their parameters, are the following:\n\n### constant\nGenerates a constant value\n\nParameters:\n - value: The value to be generated\n \nExample:\n```yaml\nmyField:\n type: constant\n value: \"This is a constant\"\n```\n \n### integer\nGenerates a random integer\n\nParameters:\n- min: The minimum integer to be generated (Default 0)\n- max: The maximum integer to be generated (Default Integer.MAX_VALUE)\n- negative: True if the generated number must be negative (It will be multiplied by -1)\n\nExample:\n```yaml\nmyField:\n  type: integer\n  min: 0\n  max: 50\n```\n\n### double\nGenerates a random double\n\nParameters:\n- min: The minimum double to be generated (Default 0.00)\n- max: The maximum double to be generated (Default Double.MAX_VALUE)\n- negative: True if the generated number must be negative (It will be multiplied by -1)\n- truncate: The decimals numbers (Default 10000, four decimals) \n\nExample:\n```yaml\nmyField:\n  type: double\n  min: 0.00\n  max: 50.00\n```\n\n### counter\nGenerates an integer counter between two values\n\nParameters:\n- min: Minimum value that counter can return (Default 0)\n- max: Maximum value that counter can return (Default Integer.MAX_VALUE)\n- direction: True if the counter goes from minimum to maximum. False if it goes from maximum to minimum. (Default true)\n\nExample:\n```yaml\nmyField:\n  type: counter\n  min: 1\n  max: 10\n```\n\n### ip\nGenerates a random IP address\n\nParameters:\n- min: The minimum ip to be generated (Default 0.0.0.0)\n- max: The maximum ip to be generated (Default 255.255.255.255)\n\nExample:\n```yaml\nmyField:\n  type: ip\n  min: 192.168.0.1\n  max: 192.168.0.255\n```\n\n### mac\nGenerates a random MAC address\n\nParameters:\n- min: The minimum ip to be generated (Default 00:00:00:00:00:00)\n- max: The maximum ip to be generated (Default FF:FF:FF:FF:FF:FF)\n- separator: The string using to split the mac on the output.  \n\nExample:\n```yaml\nmyField:\n  type: mac\n  min: 00:00:00:00:00:11\n  max: 00:00:00:00:00:44\n```\n\n### timestamp\nGenerates a field with the current timestamp of the machine\n\nExample:\n```yaml\nmyField:\n  type: timestamp\n```\n\n### collection\nGenerates a random element from the given collection of elements\n\nParameters:\n- values: An array of elements\n\nExample:\n```yaml\nmyField:\n  type: collection\n  values:\n    - Campus A\n    - Campus B\n    - Campus C\n    - Campus D\n    - Campus E\n```\n\n### composition\nGenerates a random field based on a composition of two or more random fields\n\nParameters:\n- separator: The character that will be used as the separator between the two generated values\n- components: An array of types definitions that will be generated before joining the result with the separator\n\nExamples:\n```yaml\nmyField:\n  type: composition\n  separator: ':'\n  components:\n    - type: integer\n      min: 0\n      max: 11\n    - type: integer\n      min: 0\n      max: 101\n```\n\nThis will create values like the following: 2:34, 6:11, 1:75, 10:62...\nOf course, you can use a composition type as a component for another composition type.\n\n### set\nGenerate a fixed number of sets with multiple fixed field per set.\n\nParameters:\n- numbers: The different numbers of sets\n- components: The components that form a set\n```yaml\n  set123:\n    type: set\n    numbers: 10\n    components:\n      client_mac:\n          type: mac\n          min: 00:00:00:00:00:11\n          max: 00:00:00:00:00:44\n\n      src:\n          type: ip\n          min: 80.82.34.12\n          max: 80.82.34.72\n```\n\nThis will create 10 sets form by a **client_mac** of type *mac* and **src** of type *ip*. The sets are making on the init and them don't change after.\nA message has 1 set that is choose randomly from 10 sets.\n\n### random\nChoose one component randomly among all the components inside it.\n\nParameters:\n- components: List of the components that will be picked randomly.\n\nYou can find a simple example below:\n\n```yaml\n  mySimpleRandom:\n    type: random\n    components:\n      -\n        myField:\n          type: integer\n          min: 101\n          max: 110\n      -\n        myField:\n          type: integer\n          min: 201\n          max: 210\n      -\n        myField:\n          type: integer\n          min: 301\n          max: 310\n```\n\nThis will generate values from 101 to 110, from 201 to 210 and from 301 to 310. Also you can **nest** random types in order to get more complex combinations.\n\n### json\nGenerate a new json inside other json.\n\nParameters:\n- components: The components that form a set\n```yaml\n  engine_id_name:\n    type: constant\n    value: IANA-L4\n  myNewJson:\n    type: json\n    components:\n        src:\n          type: ip\n          min: 192.168.0.1\n          max: 192.168.255.255\n        application_id:\n          type: composition\n          separator: ':'\n          components:\n             - type: integer\n               min: 0\n               max: 11\n             - type: integer\n               min: 0\n               max: 101\n        myNewJsonInside:\n          type: json\n          components:\n             dst:\n               type: ip\n               min: 192.168.0.1\n               max: 192.168.255.255\n```\n\nThis will generate messages like this:\n\n```json\n{\"myNewJson\":{\"myNewJsonInside\":{\"dst\":\"192.168.231.69\"},\"src\":\"192.168.161.220\",\"application_id\":\"9:32\"},\"engine_id_name\":\"IANA-L4\"}\n```\n\n### coordinates\nGenerate a random latitude and longitude inside a circle that is defined using radius.\n- latPoint: The latitude center point.\n- longPoint: The longitude center point.\n- radius: The radius of the circle.\n- latitudeDim: The field name of the latitude value.\n- longitudeDim: The field name of the longitude value.\n- truncate: The decimals numbers (Default 10000, four decimals) \n\nExample:\n```yaml\n  coordinates:\n    type: coordinate\n    latPoint: 30.000\n    longPoint: -40.000\n    radius: 1000\n    latitudeDim: latitude\n    longitudeDim: longitude\n    truncate: 1000\n```\n\n### array\nGenerate array with different components.\n- components: The different types that build the array\n\nExample:\n```yaml\nmyArray:\n    type: array\n    components:\n       field1:\n         type: json\n         components:\n            dst:\n               type: ip\n               min: 192.168.0.1\n               max: 192.168.255.255\n       field2:\n         type: constant\n         value: 5\n```\n\nThis example generate an array like:\n\n```json\n{\"myArray\":[5,{\"dst\":\"192.168.142.39\"}]}\n{\"myArray\":[5,{\"dst\":\"192.168.121.126\"}]}\n{\"myArray\":[5,{\"dst\":\"192.168.189.54\"}]}\n```\n\n### sequential\nGenerate one component each time. The sequential restart when reach the end.\n- components: The different types that build the sequence\n\n```yaml\nsequentialField:\n type: sequential\n components:\n   -\n     seq1:\n       type: set\n       numbers: 1\n       components:\n         longitude:\n            type: constant\n            value: 1\n         latitude:\n            type: constant\n            value: -1\n   -\n     seq2:\n       type: set\n       numbers: 1\n       components:\n         longitude:\n            type: constant\n            value: 2\n         latitude:\n            type: constant\n            value: -2\n   -\n     seq3:\n       type: set\n       numbers: 1\n       components:\n         longitude:\n            type: constant\n            value: 3\n         latitude:\n            type: constant\n            value: -3\n```\n\nThis example generate an array like:\n\n```json\n{\"longitude\":1,\"latitude\":-1}\n{\"longitude\":2,\"latitude\":-2}\n{\"longitude\":3,\"latitude\":-3}\n{\"longitude\":1,\"latitude\":-1}\n{\"longitude\":2,\"latitude\":-2}\n```\n\n## Contributing\n\n1. [Fork it](https://github.com/redborder/synthetic-producer/fork)\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create a new Pull Request\n\n## License\n\n[AGPL v3](http://www.gnu.org/licenses/agpl-3.0.html)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredborder%2Fsynthetic-producer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fredborder%2Fsynthetic-producer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredborder%2Fsynthetic-producer/lists"}