{"id":13748621,"url":"https://github.com/tdunning/log-synth","last_synced_at":"2025-04-06T06:11:23.801Z","repository":{"id":6738963,"uuid":"7985103","full_name":"tdunning/log-synth","owner":"tdunning","description":"Generates more or less realistic log data for testing simple aggregation queries.","archived":false,"fork":false,"pushed_at":"2023-12-01T15:34:17.000Z","size":3439,"stargazers_count":257,"open_issues_count":8,"forks_count":88,"subscribers_count":24,"default_branch":"master","last_synced_at":"2025-03-30T05:06:56.318Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tdunning.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2013-02-03T01:57:28.000Z","updated_at":"2024-08-15T10:44:38.000Z","dependencies_parsed_at":"2023-12-01T16:45:49.788Z","dependency_job_id":null,"html_url":"https://github.com/tdunning/log-synth","commit_stats":{"total_commits":214,"total_committers":11,"mean_commits":"19.454545454545453","dds":"0.40654205607476634","last_synced_commit":"1088028036bf82b296db0d95790dcf00936b69fe"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tdunning%2Flog-synth","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tdunning%2Flog-synth/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tdunning%2Flog-synth/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tdunning%2Flog-synth/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tdunning","download_url":"https://codeload.github.com/tdunning/log-synth/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247441053,"owners_count":20939239,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T07:00:45.897Z","updated_at":"2025-04-06T06:11:23.784Z","avatar_url":"https://github.com/tdunning.png","language":"Java","funding_links":[],"categories":["Java","测试"],"sub_categories":[],"readme":"log-synth\n=========\n\nThe basic idea here is to have a random data generator build fairly\nrealistic files for analysis. The primary use of log-synth has been to\ngenerate data based on a specified schema, but there is an older\nsystem to generate data that looks like a particular kind of web\nserver log. See web-log.md in this directory for more information\nabout the web log generator.\n\nPlease note that contributions are very welcome, but you will need to agree to a contributor license agreement similar to the Apache ICLA before any significant changes are accepted. Ping me (ted.dunning@gmail.com) for details. Ping me if you need a different language or output structure as well.\n\nSchema-driven Data Generation\n=====================\n\nLog-synth allows you to generate data based on a rough schema.  Schema-driven generation supports the generation of addresses, dates, foreign key references, unique id numbers, random integers, sort of realistic personal names and fanciful street names.\n\nIn addition to these primitive generators of strings and numbers, nested structures of arrays and objects can also be generated. You can also generate files that link together via ID's so that complex star schema structures can be build.  In a future release, it is anticipated that the generator will execute arbitrary Javascript in order to allow arbitrary dependencies and transformations of data as it is generated.\n\nTo generate data, follow the compilation directions below, that will\ncreate a standalone executable that you can use to generate data. This\ngenerates a million records using the schema in the file `schema.synth`.\n\n    target/log-synth -count 1M -schema schema.synth\n\n## Command line arguments\n\nThe allowable arguments include:\n\n `-count n`    Defines how many lines of data to emit.  Default value is 1000.  Suffixes including k, M, and G have customary meanings.\n\n `-schema file` Defines where to get the schema definition from.  The schema is in JSON format and consists of a list of field specifications.  Each field specification is a JSON object and is required to have the following value\n\n * `class` - Defines the distribution that is used to sample values for this field.  Possible values include `address`, `date`, `foreign-key`, `id`, `int`, and `street-name`.  Additional values that may be allowed or required for specific generators are detailed below.\n\nCommonly, field specifications also need to give a name\n\n * `name` - This is the name of the field.  The output will consist of fields ordered as in the schema definition and any header file will contain the names for each field as defined by this value.\n\nSee the longer examples below for more information.\n\n`-format CSV | TSV | JSON | XML` Defines what format the output should\nuse.  Note that XML format assumes that the root element is called\n\"root\" and each record element should be callled \"OBJECT_NODE\".  There\nis no way that is what most people really want.  If you need a better\nkind of XML format, file an issue or send a pull request with what is\nreally needed.\n\nNote that the JSON output is a list of individual JSON maps, each one\nto a line, to convert such data to a single JSON array, use the following sed command:\n``` \nsed -i .bak -e '1 s/^/[\\n/' -e '$ s/$/\\n]/' -e '$ ! s/$/,/' oldfile.json  \n``` \nNote that this only works with gnu sed and won't work with the version\nof sed you will find by default on Macs as part of OSX. On OSX, try\nthis instead\n```\nsed -i .bak -e '1 s/^/[\\'$'\\n/' -e '$ s/$/\\'$'\\n]/' -e '$ ! s/$/,/' oldfile.json \n```\nThis last form should work on Linux as well, but I haven't tested it.\n\n `-output output-directory-name`    Designates an output\n directory. Output files will be created in this directory named\n according to the pattern `synth-\u003cthread\u003e.\u003cext\u003e` where `\u003cthread\u003e` part\n is replaced by the thread number that created the file and `\u003cext\u003e` is\n replaced by an appropriate file extension.\n\n `-threads n`  Indicates how many threads to use for generating data.  Requires `-output`.  Note that the schema is\nshared across all of the threads so a schema with an id sampler will still generate all consecutive values in order, but the values will be distributed pretty much randomly across the output files.\n\nNote also that the number of threads that gives best throughput is somewhat surprisingly larger than you might think.  \u003e100 threads can be useful. Having lots of threads can, on the other hand, consume a lot of memory.  Trust, but verify.\n \n## Samplers Allowed in a Schema\n\nCheck the [end of this README](https://github.com/tdunning/log-synth/blob/master/README.md#longer-examples) for worked examples.\n\nThe following classes of values are allowed (in approximately alphabetical order):\n\n#### `address`\nThis distribution generates fairly plausible, if somewhat fanciful street addresses.  There are no additional parameters allowed.\n\n    {\"name\":\"address\", \"class\":\"address\"},\n        \n#### `array-flattener`\nThis sampler converts a nested list of lists into a flat list.  This can be useful if used in conjunction with the `sequence` sampler (see the example for `sequence`).\n\n#### `browser`\nSamples from browser types with kind of plausible frequency distribution.\n\n```json\n{\"name\":\"br\", \"class\":\"browser\"}\n```\n#### `commuter`\nSamples simulated automotive data from commuters.\n\nThe idea here is that we have some number of commuters who each have a home and work location. These commuters\ntend to drive to work in the morning rush hour and home in the evening rush hour, although they may do either \ncommute at other times. While at home, these simulated commuters may decide to run some errands.\n\nUnderneath this life-style model is a traffic model that has each driver pick a route either on local roads or\non a highway. Speeds on local roads are lower and more variable than on highways. Highways also go nearly directly to\nthe destination while local roads are bound to north-south or east-west directions. The choice of which kind of \nsegment to pick depends mostly on the distance to the destination. Note that there is no pre-defined set of roads,\nthe model just makes it up as segments are chosen. This means that there really isn't any sort of congestion modeling\nhappening here, just variable speeds.\n\nBelow the route planning model is a physical model that involves cars that respond to the control inputs generated\nat the higher levels to try to maintain desired speeds. The cars look roughly like they have moderately powerful\nengines with 7 speed automatic transmissions. The shift points are set to roughly match a diesel engine. All shifting\nis done based on fixed shift points and all throttle settings are done using a simple closed loop model that tries\nto match the desired road speed. The performance level of the cars is chosen to be moderately good in that they can\ndo 0-60 MPH in about 7-8 seconds.\n\nThe output from the `commuter` model can be either in nested or flattened form according to whether `flat: true` is \nused in the schema. In nested form there is one record per simulated vehicle. Inside each of these vehicle traces is \na history of what the vehicle did during the test in the form of a list of trips. Each trip has descriptive information\nabout the trip such as distance, start time, duration and type (`errand`, `to_home`, `to_work`).\n\nIn the flattened form, each sample in the nested form is retained, but all nesting is removed with all of the\nfields from the outer structures being repeated in each sample record.\n\nHere is a sample schema for the `commuter` model:\n\n```json\n[\n    {\n        \"name\": \"vehicle\",\n        \"class\": \"vin\",\n        \"verbose\": \"true\"\n    },\n    {\n        \"name\": \"trip\",\n        \"class\": \"commuter\",\n        \"home\": {\n            \"class\":\"zip\"  ,\n            \"fields\":\"latitude, longitude, zip\"\n        },\n        \"work\": 20,\n        \"start\": \"2015-09-03 0:00:00\",\n        \"end\": \"2015-09-04 0:00:00\"\n    }\n]\n```\nThe commuter model also allows you to add some extra fields to every sample in the generated trips. The way that this works is that you define a field called `extras` that contains a complete `log-synth` schema. \n\nIn this next example, our basic commuter schema is augmented so that three extra fields are generated for each sample point.\n\n```json\n[\n    {\n        \"name\": \"vehicle\",\n        \"class\": \"vin\",\n        \"verbose\": \"true\"\n    },\n    {\n        \"name\": \"trip\",\n        \"class\": \"commuter\",\n        \"home\": {\n            \"class\":\"zip\"  ,\n            \"fields\":\"latitude, longitude, zip\"\n        },\n        \"work\": 20,\n        \"start\": \"2015-09-03 0:00:00\",\n        \"end\": \"2015-09-04 0:00:00\",\n        \"extras\": [\n            {\n                \"name\": \"door-open\",\n                \"class\": \"int\",\n                \"min\": 0,\n                \"max\": 0\n            },\n            {\n                \"name\": \"warning-light\",\n                \"class\": \"string\",\n                \"dist\": {\n                    \"on\": 1,\n                    \"off\": 100\n                }\n            },\n            {\n                \"name\": \"temperature\",\n                \"class\": \"random-walk\",\n                \"mean\":100,\n                \"sd\":1\n            }\n        ]\n    }\n]\n```\nNote that the commuter model produces a lot of data per record due to the frequent sampling of engine data. \nThis means that you won't get very many output records per second of simulator run-time, especially if \nyou ask for long histories. This also means that some tools may choke on the output due to the size of \neach input records. To deal with this, you can produce flattened data, you can generate just a single day \nof data at a time or you can request a feature to make the engine sampling frequency to be extended. Your \nfeedback would be helpful here if you need this model.\n\n#### `country`\nSamples from ISO country codes.\n```json\n{\"name\":\"co\", \"class\":\"country\"}\n```\n#### `date`\nThis distribution generates dates which are some time before an epoch date.  Dates shortly before the epoch are more common than those long before.  On average, the dates generated are 100 days before the epoch.  A format field is allowed which takes a format for the data in the style of Java's SimpleDateFormatter.  Note that the order of options is significant in that the format will apply to the start and end options if it comes before them.  By default, these are formatted like yyyy-MM-dd, but you can specify a different format using the format option.  Dates are selected by default to be before July 1, 2013.  The amount before is selected exponentially with mean of 100 days.  If you specify start or end dates, the dates will be sampled uniformly between your dates. The default start is January 1, 1970.  The default end is July 1, 2013.\n```json\n[\n  {\"name\":\"first_visit\", \"class\":\"date\", \"format\":\"MM/dd/yyyy\"},\n  {\"name\":\"second_date\", \"class\":\"date\", \"start\":\"2014-01-31\", \"end\":\"2014-02-07\"},\n  {\"name\":\"third_date\", \"class\":\"date\", \"format\":\"MM/dd/yyyy\", \"start\":\"01/31/1995\", \"end\":\"02/07/1999\"}]\n\n```    \n#### `dns`\nSamples DNS queries with some details about domain and source of the query. The basic idea is\nthat each sample is a history of a particular source of DNS queries. The domains queried\nare sampled from a very skewed distribution (technically, a shared Pitman-Yor sampler) that generates very long tails. The most commonly queried\ndomains are taken from the actual domains of Fortune 500 companies, but others are jibberish generated\nas they are needed.\n\nEach query source is sampled for the time period of interest, but the rates for different sources can\nbe very different. Each source is modeled as being either 'active' or 'inactive'. During active times, DNS queries\nare generated at a higher rate than during inactive times. Different query sources can have active rates that\ndiffer by a factor of 1000 or more to emulate the difference between, say, a lone person using a phone or an office full of people sharing an Internet connection. The transition from inactive to active and back is done by picking an abstract interval until \nthe next transition. This abstract interval represents a warped sort of time that passes more quickly during the local 'daytime' for a source and more slowly during local 'nighttime'. The definition of 'daytime' and 'nighttime' varies by source. The overall model thus has parameters for night time dilation, average \nactive and inactive times and query rates during active and inactive periods. Each of these parameters are sampled for each\nquery source and the distributions for these parameters are controlled by the following hyper-parameters:\n```\nName        Controls                         Default      Description\ndilation    Night-time interval dilation         6            Night dilation is distributed as\n                                                              `rexp(rgamma(shape=dilation))` \nactive      Average active time (seconds)       10 minutes    Activity duration is distributed as\n                                                              `rexp(rate=1/t_active)` where `t_active ~\n                                                              rexp(rate=1/active)` \ninactive    Average inactive time (seconds)      4 hours      Duration of inactivity is distributed as \n                                                              `rexp(rate=1/t_inactive)` where `t_inactive`\n                                                              is sampled for each source independently\n                                                              according to `t_inactive ~\n                                                              rexp(rate=1/inactive)`.  \nrate        Average query rate per second       10 queries/h  Queries are Poisson distributed with rate \n                                                              while active distributed as rexp(rate=1/rate)\nidle        How much does query rate slow        6            Query rate is decreased by ratio distributed \n            when inactive                                     as exp(rgamma(shape=idle))\nalpha       Growth scale of unique domains   10000            Pitman-Yor cale factor for number of unique\n                                                              domains. \ndiscount    Growth rate of unique domains        0.5          Number of unique domains grows with `alpha *\n                                                              N^discount` if `discount \u003e 0` and `alpha *\n                                                              log(N)` if `discount == 0` \n```\nIn addition to these parameters, you can add `json 'flat'=true` to force the output to consist of individual DNS\nqueries. If `json 'flat'=false`, the output consists of one row per source IP with a list of JSON objects containing\nall requested fields.\n\nThe `start`, and `end` required parameters can be used to set the beginning and ending time for the simulation. \nThe `format` parameter determines the  format for the `start` and `end` as well as the format for the `time`\nfield in the output. The `format` parameter can be used after `start` and `end` are set to produce output\nin a different format than that used to specify the `start` and `end` parameters. By default all time expressions \nparsed as \"yyyy-MM-dd\", or, if that fails, as \"yyyy-MM-dd HH:mm:ss\". By default, output is produced as \n\"yyyy-MM-dd HH:mm:ss.\" \n\nYou can also set `fields` to a comma-separated list of fields that should be included in the output of the sampler. \nAvailable fields include `domain`, `source_ip`, `reverse_domain`, `time`.\n \nThe following examples show how this sampler can be used\n```json\n{\"class\": \"dns\", \"start\": \"2014-01-01\", \"end\": \"2014-03-01\", \"format\": \"MM/dd/yyyy HH:mm:ss\",\n \"fields\": \"domain, reverse_domain, source_ip\"} \n```\n#### `event`\nSamples Poisson distributed event times with specified rates.\n\n```json\n{\"name\":\"foo1\", \"class\":\"event\", \"rate\": \"0.1/d\"}\n{\"name\":\"foo2\", \"class\":\"event\", \"start\": \"2014-01-01\", \"format\":\"yyyy-MM-dd HH:mm:ss\", \"rate\": \"10/s\"}\n{\"name\":\"foo3\", \"class\":\"event\", \"format\": \"MM/dd/yyyy HH:mm:ss\", \"start\": \"02/01/2014 00:00:00\", \"rate\": \"0.5/s\"}\n```\n#### `flatten`\nTurns an object into fields.\n\nSome samplers such as `zip` or `vin` return complex objects with many fields.  If you want to output each of these fields\nas a separate field in CSV format, you could post process a JSON output file or you can use the `flatten` sampler\nto promote these fields to the level above. As an example, the snippet below results in samples with fields like `zipType`,\n`zip`, `latitude`, `longitude` and others.\n\n```json\n{\n   \"class\": \"flatten\",\n   \"value\": { \"class\": \"zip\" },\n   \"prefix\": \"\"\n}\n```\nNotice how there is no name here for the `flatten` sampler.  This is because the resulting values are named using the \nprefix (empty in this example) and the names fo the fields from the sub-sampler (the `zip` sampler in this case).\n\nThe prefix used to form the flattened variable names can be specified explicitly, but unless you want it to be empty \nit is usually simpler to just name the `flatten` sampler.  This gives you a default prefix that is the name of the \nflattener with a dash appended.  For instance, this snippet\n\n```json\n[\n {\n   \"name\": \"x\",\n   \"class\": \"flatten\",\n   \"value\": { \"class\": \"zip\", \"fields\": \"latitude, longitude\" }\n },\n {\n   \"name\": \"y\",\n   \"class\": \"flatten\",\n   \"value\": { \"class\": \"zip\", \"fields\": \"latitude, longitude\" }\n }\n]\n```\nwould give samples with fields named `x-latitude`, `x-longitude`, `y-latitude` and `y-longitude` which makes it\neasy to keep track of which fields are associated with each other.\n\n#### `foreign-key`\nThis distribution generates randomized references to an integer key from another table.  You must specify the size of the table being referenced using the size parameter. The default value of size is 1000.  You may optionally specify a skewness factor in the range [0,3].  A value of 0 gives uniform distribution.  A value of 1 gives a classic Zipf distribution.  The default skew is 0.5.  Values are biased towards smaller values.  This sampler uses space proportional to size so be slightly cautious.\n\n#### `id`\nThis distribution returns consecutive integers starting at the value of the start parameter.\n\n```json\n{\"name\":\"id\", \"class\":\"id\"}\n```\n#### `int`\nSamples values from min (inclusive) to max (exclusive) with an adjustable skew toward small values.  If you set skew to a negative number, larger values will be preferred.\n\n```json\n[\n  {\"name\":\"size\", \"class\":\"int\", \"min\":10, \"max\":99},\n  {\"name\": \"z\", \"class\": \"int\", \"min\": 10, \"max\": 20, \"skew\": -1},\n  {\"name\":\"x\", \"class\":\"lookup\", \"resource\":\"data.json\", \"skew\":1}\n]\n```\n\nYou can also specify an explicit distribution for the values returned from this sampler by using the `dist` parameter. For instance, this gives you two values, `a` and `b`. The `a` value will have values 1, 2, 3, 4 distributed 50%, 25%, 15%, 10% while `b` will have values distributed 10%, 15%, 25%, 50%.\n```json\n[\n  {\"name\":\"a\", \"class\":\"int\", \"dist\":[1, 50, 2, 25, 3, 15, 4, 10]},\n  {\"name\":\"b\", \"class\":\"int\", \"dist\":[4, 50, 3, 25, 2, 15, 1, 10]}\n]\n```\n#### `join`\nGlues together an array of strings.  You can specify a separator that goes between the joined strings with the `separator` parameter.  The `value` parameter specifies how to generate the arrays of strings.\n\nThis snippet will generate silly file names nested three deep:\n\n```json\n{\n \"name\": \"filename\",\n \"class\": \"join\",\n \"separator\": \"/\",\n \"value\": {\n   \"class\":\"sequence\",\n   \"length\":3,\n   \"array\":[\n     {\"class\":\"string\", \"dist\":{\"top1\":10, \"top2\":5, \"top3\":2}},\n     {\"class\":\"string\", \"dist\":{\"mid1\":10, \"mid2\":5, \"mid3\":2}},\n     {\"class\":\"string\", \"dist\":{\"alice\":10, \"bob\":5, \"charles\":2, \"dahlia\":1, \"ephraim\":1}}\n   ]\n }\n}\n```\n#### `language`\nSamples from ISO language codes according to prevalence on the web.\n\n```json\n{\"name\":\"la\", \"class\":\"language\"}\n```        \n#### `lookup`\nSamples from lines of a file located in the resources folder. The filename must have .csv, .tsv, or .json suffix and the first line is expected to be a header row containing names for each field.\n\n```\n{\"name\":\"bmw-models\", \"class\":\"lookup\", \"resource\":\"bmw-models.tsv\"}\n```\n\n#### `map`\nSamples from complex objects, fields of which are sampled according to a recursive schema you specify.\n\n```json\n{\n \"name\": \"stuff\",\n \"class\": \"map\",\n \"value\": [\n   {\"name\": \"a\", \"class\": \"int\", \"min\": 3, \"max\": 4},\n   {\"name\": \"b\",\"class\": \"int\",\"min\": 4,\"max\": 5}\n ]\n}\n```\n#### `name`\nSamples from (slightly) plausible names.  The allowable types are\n`first`, `last`, `first_last` and `last_first`.  The default type is `first_last`.\n\n```json\n{\"name\":\"name\", \"class\":\"name\", \"type\":\"first_last\"}\n```\n\n#### `normal`\nSamples from a normal distribution. You can set `mean` and either `sd` or `precision`. \nIf you want a predictable sequence set the `seed`. You can also trim the distribution by setting `min` and/or `max`.\n\n```json\n{\"name\":\"name\", \"class\":\"normal\", \"mean\":3, \"sd\": 2, \"min\":0}\n```\n\n#### `sequence`\nRepeatedly samples from a single distribution and returns an array of the results.\n\nThis example produces variable length results with exponentially distributed lengths.  Some of the results have length 0.\n\n```json\n{\"name\":\"c\", \"class\":\"sequence\", \"base\":{\"class\":\"os\"}}\n```\nThis example produces values with lengths that are exponentially distributed with mean length of 10.\n\n```json\n{\"name\":\"d\", \"class\":\"sequence\", \"base\":{\"class\":\"int\", \"min\":3, \"max\":9}, \"length\":10}\n```\nThis example produces results that always have three values, each of which has a different distribution.\n\n```json\n{\n \"name\": \"x\",\n \"class\": \"sequence\",\n \"array\": [\n   {\n     \"class\": \"int\",\n     \"min\": 3,\n     \"max\": 4\n   },\n   {\n     \"class\": \"int\",\n     \"min\": 6,\n     \"max\": 7\n   },\n   {\n     \"class\": \"int\",\n     \"min\": 8,\n     \"max\": 9\n   }\n ]\n}\n```\nNormally a `sequence` produces arrays of result whose length is randomly chosen from an exponential distribution.  \nIf you set the `lengthDistribution` parameter instead of the `length` parameter, then you can control how the length is \nchosen. Somewhat confusingly, if you set `lengthDistribution` to a constant you get lists with the same length every \ntime.  Here are some examples:\n```json\n    {\n      // generates lists with exactly 5 samples each\n      \"name\": \"fixed-length\",\n      \"class\":\"sequence\",\n      \"lengthDistribution\":5,\n      \"base\": {\n       // insert distribution here\n      }\n    }\n```\n```json\n    {\n      // generates lists with 5-10 samples in each\n      \"name\": \"fixed-length\",\n      \"class\":\"sequence\",\n      \"lengthDistribution\":{\"class\":\"integer\", \"min\":5, \"max\":10},\n      \"base\": {\n        // insert sampler definition here\n      }\n    }\n```\nA `sequence` sampler also normally returns all values as a list that is assigned to a field in the resulting records. You can\nopt to flatten the results by adding `\"flat\":true` as an option specification of the sequence. Thus, this schema:\n```json\n[\n  {\"name\":\"a\", \"class\":\"vin\"},\n  {\"name\":\"c\", \"class\":\"sequence\", \"base\":{\"class\":\"os\"}}\n]\n```\nwill generate records with a VIN number in field `\"a\"` and lists of variable length in field `\"c\"`. \n\nThis schema, in contrast:\n```json\n[\n  {\"name\":\"a\", \"class\":\"vin\"},\n  {\"name\":\"c\", \"class\":\"sequence\", \"base\":{\"class\":\"os\"}, \"flat\":true}\n]\n```\nwill produce records with a field `\"a\"` containing a VIN and a field `\"c\"` containing a string from the `os` sampler. Each\nlist that the `sequence` sample produces will result in the VIN being repeated once for each of the samples in the list.\n\nSometimes you want to not only flatten the results of a sequence so that multiple records are produced, but you may want to \npromote the fields of the samples in the sequence to be top-level fields in the resulting records. This is \nparticularly true if you want to return the data in CSV or other formats that don't handle nested data well. To do this,\nsimply omit the name of the sequence variable. For instance, this schema\n```json\n[\n    {\n\t\"name\":\"id\", \"class\":\"uuid\"\n    },\n    {\n\t\"class\":\"sequence\", \n\t\"length\": 3,\n\t\"base\":{\n\t    \"class\":\"map\",\n\t    \"value\": [\n\t\t{\n\t\t    \"name\": \"sample-time\",\n\t\t    \"class\": \"event\",\n\t\t    \"rate\": \"3/h\",\n\t\t    \"format\":\"yyyy-MM-dd HH:mm:ss\",\n\t\t    \"start\": \"2019-04-01 17:43:00\"\n\t\t},\n\t\t{\n\t\t    \"name\": \"temperature\",\n\t\t    \"class\": \"random-walk\",\n\t\t    \"mean\":0,\n\t\t    \"sd\":1,\n\t\t    \"start\":30\n\t\t}\n\t    ]\n\t}, \n\t\"flat\":true\n    }\n]\n```\nwill produce multiple records for each device that look like this:\n```json\n{\"id\":\"dace817b-f0c6-47a8-90a6-a21bb6d7a1d2\",\"sample-time\":\"2019-04-01 17:43:00\",\"temperature\":30.146672000677313}\n{\"id\":\"21925927-55c7-40e9-bec6-3aa14ca4ba80\",\"sample-time\":\"2019-04-01 17:43:00\",\"temperature\":31.337189250191063}\n{\"id\":\"21925927-55c7-40e9-bec6-3aa14ca4ba80\",\"sample-time\":\"2019-04-01 18:15:20\",\"temperature\":31.903948039663167}\n{\"id\":\"21925927-55c7-40e9-bec6-3aa14ca4ba80\",\"sample-time\":\"2019-04-01 18:17:51\",\"temperature\":32.00336665938387}\n{\"id\":\"08c90cac-d841-4f34-9fb3-359a299d027b\",\"sample-time\":\"2019-04-01 17:43:00\",\"temperature\":31.145485476447}\n{\"id\":\"08c90cac-d841-4f34-9fb3-359a299d027b\",\"sample-time\":\"2019-04-01 18:13:39\",\"temperature\":34.18391712370743}\n```\n\n#### `ssn`\nSamples somewhat realistic SSN's\n\nA social security number (SSN) has fields `ssn`, `state`, `description` and `type`.  The `ssn` field is what you might expect.  The\n`state` field is the two letter abbreviation of the state that the SSN was issued in (assuming that the SSN was issued before the 2011 conversion to\nrandom assignment).  The `description` is the longer form of the `state`.  The `type` field can have the value `normal` or `extra`.  The `extra` type applies to locations\nthat don't correspond to the 52 values that most people might expect (50 states + `DC` + `PR`).\n\nYou can limit which fields you get back with default fields of `ssn`, `state``description`.  You can also limit the types of values you get back.\n\nFor example:\n\n```json\n{\n \"name\": \"z\",\n \"class\": \"ssn\"\n}\n```\nor\n\n```json\n{\n \"name\": \"zLimited\",\n \"class\": \"ssn\",\n \"fields\": \"ssn,state,description,type\",\n \"types\": \"normal,extra\",\n \"seed\": 123\n}\n```\nAs is common with many samplers, you can set the seed if you like.\n\nIf you only want a string with the SSN in it, you can set the `verbose` flag to false:\n\n```json\n{\n \"name\": \"z\",\n \"class\": \"ssn\",\n \"verbose\": false\n}\n```\n\n#### `state`\nSamples from any of the 58 USPS state abbreviations.  Yes, there are 58 possible values.\n\n```json\n{\"name\":\"st\", \"class\":\"state\"}\n```\n#### `street-name`\nThis distribution generates fanciful three word street names.\n\n#### `string`\nThis distribution generates a specified distribution of strings.  One parameter called `dist` is \nrequired.  This parameter should be a structure with string keys and numerical values.  The probability for each \nkey is proportional to the value.\n\n```json\n{\"name\":\"foo\", \"class\":\"string\", \"dist\":{\"YES\":0.95, \"NO\":0.05, \"NA\":1}}\n```\n#### `uuid`\nGenerates random UUIDs.\n\n#### `os`\nSamples from operating system codes.  My own bias will show here.\n\n```json\n{\"name\":\"os\", \"class\":\"os\"}\n```    \n#### `random-walk` and `gamma`\nAllows sampling from a random walk.\n\nThe `random-walk` sample by default samples steps from a normal distribution and accumulates those steps into a current position.  \nThe returned value is the sum of those steps. Optionally, the actual distribution of the steps can be specified explicitly.  \n\nThe defaults for the `random-walk` sampler are sensible so that this\n\n```json\n{\n   \"name\": \"v1\",\n   \"class\": \"random-walk\"\n}\n```    \nsamples steps from a unit normal distribution.  The scale of the steps can be changed by setting the `s` (standard deviation),\n`variance` (squared standard deviation) or `precision` (inverse of variance) parameters.  Here is an example of setting the \nscale of the step distribution:\n\n```json\n{\n   \"name\": \"v2\",\n   \"class\": \"random-walk\",\n   \"s\": 2\n}\n```    \nIf you are setting the scale to\na constant, the `s` parameter would normally be used.  You can also set these parameters to have values that are themselves\nrandom variables that are sampled each step.  For example, this sets the precision to be sampled from a gamma distribution.\nThe result of this second-order sampling will be a t-distribution with `dof` = 2.  Using very small values of `dof` gives\na very heavy-tailed distribution that occasionally takes enormous steps.\n\n```json\n{\n   \"name\": \"v3\",\n   \"class\": \"random-walk\",\n   \"precision\": {\n       \"class\": \"gamma\",\n       \"dof\": 2\n   }\n}\n```\nThe `verbose` flag can be set to true.  If `verbose` is not set, or is explicitly set to false, then the value of the\ncurrent state will be returned.  If `verbose` is set to true, then the current value and the latest step will both be\nreturned in a structure.  \n\nSetting the scale of the steps to a random variable is usually done by setting the `precision` parameter to be\nsampled from a gamma distribution since the gamma is the conjugate distribution to the normal.  The `gamma` sampler\ncan be adjusted using `alpha` (shape), `beta` (scale), `rate` (rate or 1/beta) or `dof` and `scale` parameters.  \nWhen used to set the step size distribution for a `random-walk` sampler, it is common to use the `dof` and `scale` \nparameters.  \n\n```json\n{\n   \"name\": \"g1\",\n   \"class\": \"gamma\",\n   \"alpha\": 0.2,\n   \"beta\": 0.2\n}\n```\nWhen setting `dof` and `scale`, these are translated as `alpha` = `dof` / 2, `beta` = `scale` * `dof` / 2.\n\nIf desired, the mean step size can also be set either to a constant or a distribution.  This helps model walks that\nhave a consistent drift.  As an example, we could model the sampling times for a data acquisition that makes a\nmeasurement every 100 microseconds with 2 microseconds of jitter this way:\n\n```json\n{\n   \"name\": \"t\",\n   \"class\": \"random-walk\",\n   \"mean\":100,\n   \"sd\":5\n}\n```\n\nIn special cases, none of the methods above are satisfactory and it is desirable to set the step distribution \nexplicitly. For instance, it might be desirable to have steps that have a trimmed normal distribution or even \na gamma distribution if you want particularly long-tailed walks. This can be done by setting the ```step-distribution```\nparameter as in the following\n```json\n{\n   \"name\": \"v3\",\n   \"class\": \"random-walk\",\n   \"step-distribution\": {\n       \"class\": \"normal\",\n       \"mean\": 2,\n       \"sd\": 2,\n       \"min\": 0\n   }\n}\n\n```\n    \n#### `vin`\nSamples from sort of realistic VIN numbers.\n\nHere is are three different ways to use the VIN sampler.  The first one `v1`, uses a seed to force the generated sequence to be identical every time. \n\n```json\n{\n   \"name\": \"v1\",\n   \"class\": \"vin\",\n   \"seed\": 12,\n   \"country\": \"north_america\",\n   \"make\": \"ford\",\n   \"years\": \"2007-2011\"\n}\n```\nThe second example uses the verbose setting to generate a JSON structure instead of just a single value containing a VIN.  This structure includes additional clear text information about where the vehicle was supposedly made, what kind of engine and so on.         \n\n```json\n{\n   \"name\": \"v2\",\n   \"class\": \"vin\",\n   \"country\": \"north_america\",\n   \"make\": \"ford\",\n   \"years\": \"2007-2011\",\n   \"verbose\": \"true\"\n}\n```\nWith `verbose` set to true, the output of the sampler looks like this\n\n```json\n{\n   \"VIN\":\"3FAFW33407M000098\",\n\t    \"manufacturer\":\"Ford\",\n\t    \"model\":\"Ford F-Series, F-350, Crew Cab, 4WD, Dual Rear Wheels\",\n\t    \"engine\":\"V6,Essex,3.8 L,EFI,Gasoline,190hp\",\n\t    \"year\":2007\n}\n```\nNote that there can be implausible combinations of engine, year and model such as a 2007 DeLorean.  Also, the sampler currently only has information about Ford and BMW models.  For other makes, the model engine and plant information is just gibberish.\n\nThe third example shows how the country and year fields can have more complex constraints.\n         \n```json\n{\n   \"name\": \"v3\",\n   \"class\": \"vin\",\n   \"countries\": \"ca, mx\",\n   \"make\": \"ford\",\n   \"years\": \"2002,2007-2011\"\n}\n```\nCurrently all sampling for constructing a VIN is done by uniformly sampling all of the possible options.  This could easily be changed if desired.\n    \n\n#### `word`\nSamples words at random.  A seed file is given, but if more words are needed than seeded, they will be invented.\n\n#### `common-point-of-compromise`\nProduces a user history that emulates a common point of compromise fraud scenario.  Contact tdunning@maprtech.com for more info.\n\n```json\n[\n   {\n       \"name\": \"id\",\n       \"class\": \"id\"\n   },\n   {\n       \"name\": \"history\",\n       \"class\": \"common-point-of-compromise\",\n       \"seed\": 12,\n       \"exploitStart\": \"2014-01-20 00:00:00\",\n       \"exploitEnd\": \"2014-02-20 00:00:00\",\n       \"end\": \"2014-03-31 00:00:00\",\n       \"compromisedFraudRate\": 0.02,\n       \"uncompromisedFraudRate\": 0.001\n   }\n]\n```    \n#### `zip`\nSamples from a table of US Zip Codes.  This gives you latitude, longitude and other common parameters for zip codes.  This can be used to generate random ish locations for various purposes that have nothing to do with the postal system.\n\nAll parameters for this sampler are optional.\n\n```json\n[\n   {\n       \"name\": \"z\",\n       \"class\": \"zip\",\n       \"seed\": 12,\n       \"latitudeFuzz\": 1,\n       \"longitudeFuzz\": 1,\n       \"onlyContinental\": \"true\"\n   }\n]\n```\nZip samplers can also limit the points returned by using a latitude/longitude bounding box or by specifying a single point and a distance radius (in miles).\n\nThis gives all zips with centers within 200 miles of a point in Los Angeles\n\n```json\n{\n   \"name\": \"zLosAngeles\",\n   \"class\": \"zip\",\n   \"near\": \"33.97,-118.24\",\n   \"milesFrom\": 200\n}\n```\nNote that having a small radius here will make the sampler very slow because it will have to reject many samples.  A radius of 200 miles makes the sampler about 10 times slower.\n\nLikewise, this gives zips that have latitude from 20 to 30 degrees\n\n```json\n{\n   \"name\": \"zLosAngeles\",\n   \"class\": \"zip\",\n   \"latitude\": \"20,30\"\n}\n```\nIf you only want the zip code as a string without all the supporting information, set the `verbose` flag to false. For example:\n\n```json\n{\n   \"name\": \"zLosAngeles\",\n   \"class\": \"zip\",\n   \"near\": \"33.97,-118.24\",\n   \"milesFrom\": 200,\n   \"verbose\": false\n}\n```\n\n## Longer Examples\n### Star schema\nThe following schema generates a typical fact table from a simulated star schema:\n\n```json\n[\n   {\"name\":\"id\", \"class\":\"id\"},\n   {\"name\":\"user_id\", \"class\": \"foreign-key\", \"size\": 10000},\n   {\"name\":\"item_id\", \"class\": \"foreign-key\", \"size\": 2000}\n]\n```\nHere we have an id and two foreign key references to dimension tables for user information and item information.  This definition assumes that we will generate 10,000 users and 2000 item records.\n\nThe users can be generated using this schema.\n\n```json\n[\n   {\"name\":\"id\", \"class\":\"id\"},\n   {\"name\":\"name\", \"class\":\"name\", \"type\":\"first_last\"},\n   {\"name\":\"gender\", \"class\":\"string\", \"dist\":{\"MALE\":0.5, \"FEMALE\":0.5, \"OTHER\":0.02}},\n   {\"name\":\"address\", \"class\":\"address\"},\n   {\"name\":\"first_visit\", \"class\":\"date\", \"format\":\"MM/dd/yyyy\"}\n]\n```\nFor each user we generate an id, a name, an address and a date the user first visited the site.\n\nItems are simpler and are generated using this schema\n\n```json\n[\n   {\"name\":\"id\", \"class\":\"id\"},\n   {\"name\":\"size\", \"class\":\"int\", \"min\":10, \"max\":99}\n]\n```\nEach item has an id and a size which is just a random integer from 10 to 99.\n\n### Users and queries (`examples/users-and-queries.json`)\nYou can use the sequence type to generate variable or fixed-length arrays of values which can themselves be complex.  If you use the JSON output format, this structure will be preserved.  If you want to flatten an array produced by sequence, you can use the flatten sampler.\n\nFor example, this produces users with names and variable length query strings\n\n```json\n[\n    {\"name\":\"user_name\", \"class\":\"name\", \"type\": \"last_first\"},\n    {\"name\":\"query\", \"class\":\"array-flatten\", \"value\": {\n\t\"class\": \"sequence\", \"length\":4, \"base\": {\n            \"class\": \"word\"\n\t}\n    }}\n]\n```\nIf you use the TSV format with this schema, the queries will be comma delimited unquoted strings.  If you omit the `array-flatten` step, you will get a list of strings surrounded by square brackets and each string will be quoted (i.e. an array in JSON format).\n\n### IoT data (`examples/iot-data.json`)\nThis example shows how you can generate measurement data that appears to come from multiple devices, each with a fixed location.\n\nHere is the schema, followed by some explanations\n```json\n[\n    {\n        \"name\": \"device-id\",\n        \"class\": \"uuid\",\n        \"seed\": 1\n    },\n    {\n        \"class\": \"flatten\",\n        \"prefix\": \"\",\n        \"value\": {\n            \"class\":\"zip\"  ,\n            \"onlyContinental\": true,\n            \"fields\":\"latitude, longitude\"\n        }\n    },\n    {\n        \"class\": \"sequence\",\n        \"lengthDistribution\": 100,\n        \"base\": {\n            \"class\": \"map\",\n            \"value\": [\n                {\n                    \"name\": \"time\",\n                    \"class\": \"event\",\n                    \"format\":\"yyyy-MM-dd HH:mm:ss\",\n                    \"rate\": \"1/m\"\n                },\n                {\n                    \"name\": \"temp\",\n                    \"class\": \"random-walk\",\n                    \"start\": 30,\n                    \"mean\": 0,\n                    \"sd\":1\n                },\n                {\n                    \"name\": \"case-open\",\n                    \"class\": \"int\",\n                    \"dist\": [0, 999, 1, 1]\n                }\n            ]\n        },\n        \"flat\": true\n    }\n]       \n``` \nThis schema consists of three clauses. The first generates device IDs and is pretty simple. Internally, the UUID sampler generates some integers and formats them in very techy looking hexadecimal with dashes. The result looks like real UUIDs.\n\nThe second clause generates locations. In fact, however, it really generates zip codes. Lots of other fields are generated in addition to the actual zip code (city name, state and such) but we only retain the longitude and latitude. In order to avoid generating any Zip codes that don't have an actual location (such as military overseas codes), we use the `onlyContinental` flag. Furthermore, we use a `flattener` to unnest the location fields so that they appear at the top level of resulting records.\n\nThe last clause generates many values for each sample, each of which is a structure with three fields. These fields include `time`, `temp` and `case-open`. The `time` field in these structures is generated using the `event` sampler with an average rate of one event per minute. The `case-open` field is generated by sampling either a 1 or a 0 but with a much higher probability of picking a 0.\n\nThe interesting field among these three is the `temp` field. This is generated by a process that keeps the running total of normally distributed values. This gives reasonably varying real values that look different from device to device. \n\nThis last clause generates a variable number of values for each device with 100 samples on average. Overall, this means that each *sample* from this schema consists of one UUID, one location and one sequence of 100, more or less, time/temp/case-op flags. This single sample is flattened into multiple records because of the `flat:true` flag on the `sequence` clause in the schema. Moreover, since there is no name for the `sequence` clause, the structures in the sequence of samples are each unnested so that the records that are produced have five fields, none of which contains nested data. This makes it possible to generate normally looking CSV formatted data using this schema. \n### Nested data (`examples/nested-data.json`)\nYou can also generate arbitrarily nested data by using the map sampler.  For example, this schema will produce records with an id and a map named stuff that has two integers (\"a\" and \"b\") in it.\n\n```json\n[\n    {\"name\": \"id\", \"class\": \"id\"},\n    {\n\t\"name\": \"stuff\", \"class\": \"map\",\n\t\"value\": [\n            {\"name\": \"a\", \"class\": \"int\", \"min\": 3, \"max\": 4},\n            {\"name\": \"b\", \"class\": \"int\", \"min\": 4, \"max\": 5}\n\t]\n    }\n]\n```\n### Complex names (`examples/complex-names.json`)\nSuppose that we want to generate this kind of data:\n```json\n{\"name\":{\"first\":\"Marcelene\",\"last\":\"Gillette\"},\n\t\"student\":{\n\t    \"details\":{\"school.id\":7608,\"school.name\":\"Lloyd Burton Junior High School\"}}}\n{\"name\":{\"first\":\"Samuel\",\"last\":\"Colston\"},\n\t\"student\":{\n\t    \"details\":{\"school.id\":3555,\"school.name\":\"Kenneth Smith Junior High School\"}}}\n{\"name\":{\"first\":\"Jan\",\"last\":\"Acton\"},\n\t\"student\":{\n\t    \"details\":{\"school.id\":4295,\"school.name\":\"Jeanette Harden Elementary School\"}}}\n```\nThis is harder than it looks because there are some surprises:\n- The name is a structure, not a string\n- The school name is all fancy with a name and a kind of a school\n\nThe schema to generate this is a little long, but not really all that complicated.\nThere are a few tricks that I used:\n\n1) the [map type](https://github.com/tdunning/log-synth/blob/master/README.md#map) allows me to generate a structure for the name and the details structure. \n\n2) the [join combined with the sequence](https://github.com/tdunning/log-synth/blob/master/README.md#sequence)\ntype allows me to build the school name by combining a name and a\nrandomly selected string.\n\nHere is the schema (it looks fancier than it is):\n```json\n[{\n    \"name\": \"name\",\n    \"class\": \"map\",\n    \"value\": [{\n\t\"name\": \"first\",\n\t\"class\": \"name\",\n\t\"type\": \"first\"\n    }, {\n\t\"name\": \"last\",\n\t\"class\": \"name\",\n\t\"type\": \"last\"\n    }]\n}, {\n    \"name\": \"student\",\n    \"class\": \"map\",\n    \"value\": [{\n\t\"name\": \"details\",\n\t\"class\": \"map\",\n\t\"value\": [{\n\t    \"name\": \"school.id\",\n\t    \"class\": \"int\",\n\t    \"min\": 1000,\n\t    \"max\": 9999\n\t}, {\n\t    \"name\": \"school.name\",\n\t    \"class\": \"join\",\n\t    \"separator\": \" \",\n\t    \"value\": {\n\t\t\"class\": \"sequence\",\n\t\t\"length\": 2,\n\t\t\"array\": [{\n\t\t    \"class\": \"name\",\n\t\t    \"type\": \"first_last\"\n\t\t}, {\n\t\t    \"class\": \"string\",\n\t\t    \"dist\": {\n\t\t\t\"High School\": 1,\n\t\t\t\"Junior High School\": 2,\n\t\t\t\"Elementary School\": 5\n\t\t    }\n\t\t}]\n\t    }\n\t}]\n    }]\n}]\n```\n## Quoting of Strings\n\nBy default all strings in CSV or TSV formats are fully quoted.  This can confuse some software (sadly) so there are additional options to control how quoting is done.  \n\nThere are three basic strategies supported:\n\n* *DOUBLE_QUOTE* This is the previous behavior where all strings are safely quoted according to JSON conventions.\n\n* *BACK_SLASH* With this convention, all internal spaces, tabs, back slashes and commas are quoted by preceding them with backslash character. This convention is useful with Hive.\n\n* *OPTIMISTIC* With this convention, no quoting of strings is done.  This should not normally be used since it is very easy to get unparseable data.\n\nThe default convention is DOUBLE_QUOTE.\n\nTemplate based Data Generation\n=====================\n\nThis approach uses Freemarker template engine to render custom templates. The data variables in the template are fed from a specified schema.\n\n## Command-line options:\n\n`-template file` link to a Freemarker template\n\n`-schema file` to specify the schema (see above)\n\n## Template notation\n\nTo print the value of a variable in the template, use ${name.asText()} placeholder.\n\nTo generate a sample vCard simply create a file template.txt:\n```\nBEGIN:VCARD\nVERSION:3.0\nN:${last_name.asText()};${first_name.asText()};;${title.asText()}\nORG:Sample Org\nTITLE:${title.asText()}\nPHOTO;VALUE=URL;TYPE=GIF:http://thumbs.example.com/${filename.asText()}/${first_name.asText()?lower_case  }.gif\nTEL;TYPE=HOME,VOICE:${phone_number.asText()}\nADR;TYPE=WORK:;;${address.asText()?split(\" \")?join(\";\")}\nEMAIL;TYPE=PREF,INTERNET:${first_name.asText()[0]?lower_case}${last_name.asText()?lower_case}@example.com\nREV:${first_visit.asText()}\nEND:VCARD\n```\nThen a schema file, let’s call it schema.txt:\n```jsonjson\n[\n    {\"name\":\"title\", \"class\":\"string\", \"dist\":{\"Mr\":0.5, \"Mrs.\":0.14, \"Miss\":0.36}},\n    {\"name\":\"first_name\", \"class\":\"name\", \"type\":\"first\"},\n    {\"name\":\"last_name\", \"class\":\"name\", \"type\":\"last\"},\n\n    {\"name\": \"filename\", \"class\": \"join\", \"separator\": \"/\", \"value\": {\n          \"class\":\"sequence\",\n          \"length\":2,\n          \"array\":[\n              {\"class\":\"string\", \"dist\":{\"small\":10, \"medium\":5, \"large\":2}},\n              {\"class\":\"string\", \"dist\":{\"high\":10, \"low\":5, \"mobile\":15}}\n          ]\n     }},\n    {\"name\": \"phone_number\", \"class\": \"join\", \"separator\": \"-\", \"value\": {\n          \"class\":\"sequence\",\n          \"length\":3,\n          \"array\":[\n              { \"class\": \"int\", \"min\": 100, \"max\": 999},\n              { \"class\": \"int\", \"min\": 100, \"max\": 999},\n              { \"class\": \"int\", \"min\": 100, \"max\": 999}\n          ]\n    }},\n\n    {\"name\":\"address\", \"class\":\"address\"},\n    {\"name\":\"first_visit\", \"class\":\"date\", \"format\":\"yyyy-MM-dd HH:mm:ssZ\"}\n]\n```\nTo invoke the log-synth, just do:\n\n./target/log-synth -count 5000 -schema schema.txt -template template.txt -format TEMPLATE -output output/\n\nThe output documents will end up in the output/ folder as expected and they will look like:\n```\nBEGIN:VCARD\nVERSION:3.0\nN:Kittle;Gwendolyn;;Mr\nORG:Sample Org\nTITLE:Mr\nPHOTO;VALUE=URL;TYPE=GIF:http://thumbs.example.com/small/mobile/gwendolyn.gif\nTEL;TYPE=HOME,VOICE:774-383-580\nADR;TYPE=WORK:;;18033;Quaking;Brook;Avenue\nEMAIL;TYPE=PREF,INTERNET:gkittle@example.com\nREV:2013-07-14 01:37:08+0100\nEND:VCARDBEGIN:VCARD\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftdunning%2Flog-synth","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftdunning%2Flog-synth","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftdunning%2Flog-synth/lists"}