{"id":16643431,"url":"https://github.com/sonots/dummer","last_synced_at":"2025-07-17T08:33:40.733Z","repository":{"id":11915560,"uuid":"14480967","full_name":"sonots/dummer","owner":"sonots","description":"Generates dummy log data","archived":false,"fork":false,"pushed_at":"2019-11-13T12:41:28.000Z","size":86,"stargazers_count":78,"open_issues_count":2,"forks_count":19,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-07-14T12:49:30.076Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sonots.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-11-18T02:49:20.000Z","updated_at":"2025-04-10T19:04:14.000Z","dependencies_parsed_at":"2022-09-14T21:50:27.536Z","dependency_job_id":null,"html_url":"https://github.com/sonots/dummer","commit_stats":null,"previous_names":["sonots/dummy_log_generator"],"tags_count":17,"template":false,"template_full_name":null,"purl":"pkg:github/sonots/dummer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonots%2Fdummer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonots%2Fdummer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonots%2Fdummer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonots%2Fdummer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sonots","download_url":"https://codeload.github.com/sonots/dummer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonots%2Fdummer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265585315,"owners_count":23792716,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-12T08:08:32.236Z","updated_at":"2025-07-17T08:33:40.716Z","avatar_url":"https://github.com/sonots.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Dummer\n\nDummer is a set of tools to generate dummy log data. I made this for Fluentd benchmark. \n\nThis gem includes three executable commands\n\n1. dummer\n2. dummer\\_simple\n3. dummer\\_yes\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n    gem 'dummer'\n\nAnd then execute:\n\n    $ bundle\n\nOr install it yourself as:\n\n    $ gem install dummer\n\nRun as\n\n    $ dummer -c dummer.conf\n    $ dummer_simple [options]\n    $ dummer_yes [options]\n\n## dummer\n\n`dummer` allows you to\n\n1. specify a rate of generating messages per second, \n2. determine a log format, and\n3. generate logs randomly\n\n### Usage (1) - Write to a file\n\nCreate a configuration file. A sample configuration is as follows:\n\n```ruby\n# dummer.conf\nconfigure 'sample' do\n  output \"dummy.log\"\n  rate 500\n  delimiter \"\\t\"\n  labeled true\n  field :id, type: :integer, countup: true, format: \"%04d\"\n  field :time, type: :datetime, format: \"[%Y-%m-%d %H:%M:%S]\", random: false\n  field :level, type: :string, any: %w[DEBUG INFO WARN ERROR]\n  field :method, type: :string, any: %w[GET POST PUT]\n  field :uri, type: :string, any: %w[/api/v1/people /api/v1/textdata /api/v1/messages]\n  field :reqtime, type: :float, range: 0.1..5.0\n  field :foobar, type: :string, length: 8\nend \n```\n\nRunning\n\n```\n$ dummer -c dummer.conf\n```\n\nOutputs to the `dummy.log` (specified by `output` parameter) file like: \n\n```\nid:0422  time:[2013-11-19 02:34:58]  level:INFO  method:POST uri:/api/v1/textdata  reqtime:3.9726677258569842  foobar:LFK6XV1N\nid:0423  time:[2013-11-19 02:34:58]  level:DEBUG method:GET  uri:/api/v1/people    reqtime:0.49912949125272277 foobar:DcOYrONH\nid:0424  time:[2013-11-19 02:34:58]  level:WARN  method:POST uri:/api/v1/textdata  reqtime:2.930590441869852   foobar:XEZ5bQsh\n```\n\n### Usage (2) - Post to Fluentd process\n\n(experimental)\n\nCreate a configuration file. Assume that a fluentd process is running on localhost:24224.\nA sample configuration is as follows:\n\n```ruby\n# dummer.conf\nconfigure 'sample' do\n  host \"localhost\" # define `host` and `port` instead of `output`\n  port 24224\n  rate 500\n  tag type: :string, any: %w[raw.syslog raw.message raw.nginx] # configure tag\n  field :id, type: :integer, countup: true, format: \"%04d\"\n  field :level, type: :string, any: %w[DEBUG INFO WARN ERROR]\n  field :method, type: :string, any: %w[GET POST PUT]\n  field :uri, type: :string, any: %w[/api/v1/people /api/v1/textdata /api/v1/messages]\n  field :reqtime, type: :float, range: 0.1..5.0\n  field :foobar, type: :string, length: 8\nend \n```\n\nRunning\n\n```\n$ dummer -c dummer.conf\n```\n\nData is posted to fluentd process like (below is the fluentd log generated by out_stdout)\n\n```\n2014-01-31 00:55:32 +0900 raw.message: {\"id\":\"1377\",\"level\":\"INFO\",\"method\":\"POST\",\"uri\":\"/api/v1/people\",\"reqtime\":1.678867810409548,\"foobar\":\"paOIWxhQ\"}\n2014-01-31 00:55:32 +0900 raw.syslog: {\"id\":\"1378\",\"level\":\"INFO\",\"method\":\"GET\",\"uri\":\"/api/v1/people\",\"reqtime\":4.8412816521873445,\"foobar\":\"kUvnC0MK\"}\n2014-01-31 00:55:32 +0900 raw.message: {\"id\":\"1379\",\"level\":\"WARN\",\"method\":\"GET\",\"uri\":\"/api/v1/people\",\"reqtime\":3.584494903998221,\"foobar\":\"KD78mpjX\"}\n```\n\n### CLI Options\n\nYou can specify some configuration parameters on CLI without writing them on a configuration file.\n\n```\n$ dummer help start\nUsage:\n  dummer start\n\nOptions:\n  -c, [--config=CONFIG]            # Config file\n                                   # Default: dummer.conf\n  -r, [--rate=N]                   # Number of generating messages per second\n  -o, [--output=OUTPUT]            # Output file\n  -h, [--host=HOST]                # Host of fluentd process\n  -p, [--port=N]                   # Port of fluentd process\n  -m, [--message=MESSAGE]          # Output message\n  -d, [--daemonize]                # Daemonize. Stop with `dummer stop`\n  -w, [--workers=N]                # Number of parallels\n      [--worker-type=WORKER_TYPE]\n                                   # Default: process\n  -p, [--pid-path=PID_PATH]\n                                   # Default: dummer.pid\n```\n\n### Configuration Parameters\n\nFollowing parameters in the configuration file are available:\n\n* output\n\n    Specify a filename to output, or IO object (STDOUT, STDERR)\n\n* host\n\n    Post a data to a fluentd process on the specified host. Either of `output` or `host` can be specified.\n\n* port\n\n    Post a data to a fluentd process on the specified post. Default is 24224. \n\n* rate\n\n    Specify how many messages to generate per second. Default: 500 msgs / sec\n\n* workers\n\n    Specify number of processes for parallel processing. \n\n* delimiter\n\n    Specify the delimiter between each field. Default: \"\\t\" (Tab)\n\n* labeled\n\n    Whether add field name as a label or not. Default: true\n\n* label_delimiter\n\n    Specify the delimiter between the label and the value. Default: \":\" (column)\n\n* tag\n\n    Define tag field to generate. This is effective only for posting data to fluentd process with `host` and `port`.\n\n* field\n\n    Random field generator mode. Define data fields to generate. `message` and `input` options are ignored. See also `Field Data Types` section below.\n\n* message\n\n    Specific message generation mode. See [message.conf](./example/message.conf) as an example. This mode works pretty fast because it does not require to generate values randomly. \n\n* input\n\n    Messages taken from an input file mode.  Use this if you want to write messages by reading lines of an input file in rotation. `message` option is ignored. See [input.conf](./example/input.conf) as an example. This mode also works fast.\n\n### Field Data Types\n\nYou can specify following data types to your `tag` and `field` parameters:\n\n* :datetime\n\n  * :format\n\n    You can specify format of datetime as `%Y-%m-%d %H:%M:%S`. See [Time#strftime](http://www.ruby-doc.org/core-2.0.0/Time.html#method-i-strftime) for details. \n\n  * :random\n\n    Generate datetime randomly. Default: false (Time.now)\n\n  * :value\n\n    You can specify a fixed Time object. \n\n* :string\n\n  * :any\n\n    You can specify an array of strings, then the generator picks one from them randomly\n\n  * :length\n\n    You can specify the length of string to generate randomly\n\n  * :value\n\n    You can specify a fixed string\n\n* :integer\n\n  * :format\n\n    You can specify a format of string  as `%03d`. \n\n  * :range\n\n    You can specify a range of integers, then the generator picks one in the range (uniform) randomly\n\n  * :countup\n\n    Generate countup data. Default: false\n\n  * :value\n\n    You can specify a fixed integer\n\n* :float\n\n  * :format\n\n    You can specify a format of string  as `%03.1f`. \n\n  * :range\n\n    You can specify a range of float numbers, then the generator picks one in the range (uniform) randomly\n\n  * :value\n\n    You can specify a fixed float number\n\n## dummer\\_simple\n\nI created a simple version of `dummer` since `dummer` could not achieve the maximum system I/O throughputs because of its rich features.\nThis simple version, `dummer_simple` could  achieve the system I/O limit in my environment. \n\nSorry, but this simple script cannot post data to fluentd process, supports only writing to a file.\n\n### Usage\n\n```\n$ dummer_simple [options]\n```\n\n### Options\n\n```\nUsage:\n  dummer_simple\n\nOptions:\n      [--sync]             # Set `IO#sync=true`\n  -s, [--second=N]         # Duration of running in second\n                           # Default: 1\n  -p, [--parallel=N]       # Number of processes to run in parallel\n                           # Default: 1\n  -o, [--output=OUTPUT]    # Output file\n                           # Default: dummy.log\n  -i, [--input=INPUT]      # Input file (Output messages by reading lines of the file in rotation)\n  -m, [--message=MESSAGE]  # Output message\n                           # Default: time:2013-11-20 23:39:42 +0900    level:ERROR     method:POST     uri:/api/v1/people      reqtime:3.1983877060667103\n```\n\n## dummer\\_yes\n\nI created a wrapped version of `yes` command, `dummer_yes`, to confrim that `dummer_simple` achieves the maximum system I/O throughputs. \n\nI do not use `dummer_yes` command anymore because I verified that `dummer_simple` achieves the I/O limit, but I will keep this command so that users can do verification experiments with it. \n\n### Usage\n\n```\n$ dummer_yes [options]\n```\n\n### Options\n\n```\nUsage:\n  dummer_yes\n\nOptions:\n  -s, [--second=N]         # Duration of running in second\n                           # Default: 1\n  -p, [--parallel=N]       # Number of processes to run in parallel\n                           # Default: 1\n  -o, [--output=OUTPUT]    # Output file\n                           # Default: dummy.log\n  -m, [--message=MESSAGE]  # Output message\n                           # Default: time:2013-11-20 23:39:42 +0900  level:ERROR method:POST uri:/api/v1/people  reqtime:3.1983877060667103\n```\n\n## Relatives\n\nThere is a [fluent-plugin-dummydata-producer](https://github.com/tagomoris/fluent-plugin-dummydata-producer), but I wanted to output dummy data to a log file, \nand I wanted a standalone separated tool to do benchmark.\n\n## Related Articles\n\n* [Fluentd のベンチマークテストに使える dummer (旧称 dummy_log_generator)](http://qiita.com/sonots/items/750da77a18e62852a02f)\n\n## ToDO\n\n1. write tests\n\n## Contributing\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new Pull Request\n\n## Licenses\n\nSee [LICENSE.txt](LICENSE.txt)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsonots%2Fdummer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsonots%2Fdummer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsonots%2Fdummer/lists"}