{"id":21163627,"url":"https://github.com/mortenson/ripoff","last_synced_at":"2025-07-03T09:06:31.341Z","repository":{"id":246422652,"uuid":"821082894","full_name":"mortenson/ripoff","owner":"mortenson","description":"generate fake data from templated yaml files","archived":false,"fork":false,"pushed_at":"2025-04-14T15:16:14.000Z","size":108,"stargazers_count":9,"open_issues_count":3,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-14T16:30:03.367Z","etag":null,"topics":["databases","fakedata","postgresql"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/mortenson/ripoff","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mortenson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-06-27T19:21:09.000Z","updated_at":"2025-04-14T15:16:18.000Z","dependencies_parsed_at":"2025-04-14T16:25:39.779Z","dependency_job_id":"4aae8e83-61e5-43dd-b1b6-1f1096f33cac","html_url":"https://github.com/mortenson/ripoff","commit_stats":null,"previous_names":["mortenson/ripoff"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/mortenson/ripoff","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mortenson%2Fripoff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mortenson%2Fripoff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mortenson%2Fripoff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mortenson%2Fripoff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mortenson","download_url":"https://codeload.github.com/mortenson/ripoff/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mortenson%2Fripoff/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263296443,"owners_count":23444489,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["databases","fakedata","postgresql"],"created_at":"2024-11-20T13:58:34.346Z","updated_at":"2025-07-03T09:06:31.323Z","avatar_url":"https://github.com/mortenson.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ripoff - generate fake data from templated yaml files\n\nripoff is a command line tool that generates fake data from yaml files (ripoffs) and inserts that data into PostgreSQL.\n\nSome features of ripoff are:\n\n- Model your fake data in one or more yaml files, god's favorite file format\n- Provide templated yaml files in cases where a row in one table requires many other rows, or you want loops\n- Due to deterministic random generation, re-running ripoff will perform upserts on all generated rows\n\n# Installation\n\n1. Run `go install github.com/mortenson/ripoff/cmd/ripoff@latest`\n2. Set the `DATABASE_URL` env variable to your local PostgreSQL database\n3. Run `ripoff \u003cdirectory to your yaml files\u003e`\n\nWhen writing your initial ripoffs, you may want to use the `-s` (\"soft\") flag which does not commit the generated transaction.\n\n# File format\n\nripoffs define rows to be inserted into your database. Any number of ripoffs can be included in a single directory.\n\n## Basic example\n\n```yaml\n# A map of rows to upsert, where keys are in the format \u003ctable_name\u003e:\u003cvalueFunc\u003e(\u003cseed\u003e),\n# and values are maps of column names to values.\nrows:\n  # A \"users\" table row identified with a UUID generated with the seed \"fooBar\"\n  users:uuid(fooBar):\n    email: foobar@example.com\n    # Note that ripoff will automatically set primary key columns, so you don't need to add:\n    # id: users:uuid(fooBar)\n  avatars:uuid(fooBarAvatar):\n    # ripoff will see this and insert the \"users:uuid(fooBar)\" row before this row\n    user_id: users:uuid(fooBar)\n  users:uuid(randomUser):\n    # Generate a random email with the seed \"randomUser\"\n    email: email(randomUser)\n```\n\nFor more (sometimes wildly complex) examples, see `./testdata`.\n\n## More on valueFuncs\n\nMost valueFuncs allow you to generate random data that's seeded with a static string. This ensures that repeat runs of ripoff are deterministic, which enables upserts (consistent primary keys).\n\nripoff provides:\n\n- `uuid(seedString)` - generates a v1 UUID\n- `int(seedString)` - generates an integer (note: might be awkward on auto incrementing tables)\n- `naturalDate(human readable text)` - generates a date using syntax defined by [go-naturaldate](https://github.com/tj/go-naturaldate), for example `naturalDate(one day ago)` (note: non-deterministic)\n\nand also all functions from [gofakeit](https://github.com/brianvoe/gofakeit?tab=readme-ov-file#functions) that have no arguments and return a string (called in camelcase, ex: `email(seedString)`). For the full list, see `./gofakeit.go`.\n\n## Using templates\n\nripoff files can be used as templates to create multiple rows at once.\n\nYaml files that start with `template_` will be treated as Go templates. Here's a template that creates a user and an avatar:\n\n```yaml\nrows:\n  # \"rowId\" is the id/key of the row that rendered this template.\n  # You could also pass an explicit seed and use it like `users:uuid({{ .seed }})`\n  {{ .rowId }}:\n    email: {{ .email }}\n    # It's convenient to use the rowId as a seed to other valueFuncs.\n    avatar_id: avatars:uuid({{ .rowId }})\n  avatars:uuid({{ .rowId }}):\n    url: {{ .avatarUrl }}\n```\n\nwhich you would use from a \"normal\" ripoff like:\n\n```yaml\nrows:\n  # The row id/key will be passed to the template in the \"rowId\" variable.\n  # This is useful if you'd like to reference \"users:uuid(fooBar)\" in a foreign key elsewhere.\n  users:uuid(fooBar):\n    # The template filename.\n    template: template_user.yml\n    # All other variables are arbitrary.\n    email: foobar@example.com\n    avatarUrl: image.png\n    avatarGrayscale: false\n```\n\n### Special template variables\n\n- `rowId` - The map key of the row using this template, ex `users:uuid(fooBar)`. Useful for allowing the \"caller\" to provide their own ID for the \"main\" row being created, if there is one. Optional to use if you find it awkward.\n- `enums` - A map of SQL enums names to an array of enum values. Useful for creating one row for each value of an enum (ex: each user role).\n\n# Export from your database to ripoff files\n\nAn experimental command has been added to generate ripoff files from your database. This may be useful to users just starting to use ripoff who don't have so much fake data that templating is required yet.\n\nCurrently, it attempts to export all data from all tables into a single ripoff file. You can use the `--exclude` flag to exclude specific tables from the export:\n\n```bash\n# Export all tables except 'users' and 'audit_logs'\nripoff-export --exclude users --exclude audit_logs /path/to/export\n```\n\nIn the future, additional flags may be added to allow you to include tables, add arbitrary `WHERE` conditions, modify the row id/key, export multiple files, or use existing templates.\n\n## Installation\n\n1. Run `go install github.com/mortenson/ripoff/cmd/ripoff-export@latest`\n2. Set the `DATABASE_URL` env variable to your local PostgreSQL database\n3. Run `ripoff-export \u003cdirectory to be deleted and exported to\u003e`\n\n# Security\n\nThis project explicitly allows SQL injection due to the way queries are constructed. Do not run `ripoff` on directories you do not trust.\n\n# Why this exists\n\nFake data generators generally come in two flavors:\n\n1. Model your fake data in the same language/DSL/ORM that your application uses\n2. Fuzz your database schema by spewing completely randomized data at it\n\nI find generating fake data to be a completely separate use case from \"normal\" ORM usage, and truly randomized fake data is awkward to use locally.\n\nSo ripoff is my approach to fake (but not excessively random) data generation. Since it's not aware of your application or schema, it's closer to writing templated SQL than learning some crazy high level DSL.\n\n# FAQ\n\n## Why use Go templates and valueFuncs?\n\nIt's kind of weird that `template_*` files use Go templates, but there's also valueFuncs like `uuid(someSeed)`.\n\nThis is done for two reasons - first, Go templates are ugly and result in invalid yaml, so no reason to force you to write them unless you need to. Second, ripoff builds its dependency graph based on the row ids/keys, not the actual generated random value. So you can think of the query building pipeline as:\n\n1. Load all ripoff files\n2. For each row in each file, check if the row uses a template\n3. If it does, process the template and append the templated rows into the total rows\n4. If not, just append that row\n5. Now we have a \"total ripoff\" (har har) file which contains all rows. I think it's cool at this point that the templating is \"done\"\n6. For each row, check if any column references another row and build a directed acyclic graph, then sort that graph\n7. Run queries for each row, in order of least to greatest dependencies\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmortenson%2Fripoff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmortenson%2Fripoff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmortenson%2Fripoff/lists"}