{"id":22703769,"url":"https://github.com/ul-mds/gecko-examples","last_synced_at":"2026-05-11T07:36:00.045Z","repository":{"id":229174738,"uuid":"775999250","full_name":"ul-mds/gecko-examples","owner":"ul-mds","description":"Example scripts for generating data with Gecko","archived":false,"fork":false,"pushed_at":"2024-05-22T06:21:04.000Z","size":37,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-29T20:11:32.481Z","etag":null,"topics":["data-science","numpy","pandas","python","record-linkage"],"latest_commit_sha":null,"homepage":"https://ul-mds.github.io/gecko/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ul-mds.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-22T13:29:04.000Z","updated_at":"2024-08-07T13:38:40.000Z","dependencies_parsed_at":"2024-03-28T14:42:00.530Z","dependency_job_id":"43a14ecf-dbd7-418e-bc1e-2f2afdd6dcf1","html_url":"https://github.com/ul-mds/gecko-examples","commit_stats":null,"previous_names":["ul-mds/gecko-examples"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ul-mds/gecko-examples","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ul-mds%2Fgecko-examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ul-mds%2Fgecko-examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ul-mds%2Fgecko-examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ul-mds%2Fgecko-examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ul-mds","download_url":"https://codeload.github.com/ul-mds/gecko-examples/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ul-mds%2Fgecko-examples/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266139901,"owners_count":23882563,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","numpy","pandas","python","record-linkage"],"created_at":"2024-12-10T08:12:33.733Z","updated_at":"2026-05-11T07:35:54.986Z","avatar_url":"https://github.com/ul-mds.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"This repository contains example scripts for generating data with [Gecko](https://github.com/ul-mds/gecko) — a Python\nlibrary for generation and mutation of realistic data at scale.\nIt uses the [Gecko data repository](https://github.com/ul-mds/gecko-data) to equip Gecko's built-in generators and\nmutators with the information they need to generate realistic datasets.\n\n# Installation\n\nYou need to have [Poetry](https://python-poetry.org/) installed.\n\nFirst, clone this repository.\n\n```\n$ git clone --recurse-submodules https://github.com/ul-mds/gecko-examples.git\n```\n\nChange into the repository's root directory.\nInstall the project and its dependencies, then use `poetry shell` to drop into the project's virtual environment.\n\n```\n$ cd gecko-examples\n$ poetry install\n$ poetry shell\n```\n\n# Usage\n\nThis project installs a script that you can invoke with the `gecko` command.\n\n```\n$ gecko --help\nUsage: gecko [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n  --help  Show this message and exit.\n\nCommands:\n  benchmark  Test Gecko's performance by timing data generation.\n  generate   Generate realistic-looking data with Gecko.\n```\n\nThe command exposes two subcommands: `benchmark` for testing Gecko's performance and `generate` to generate and\nmutate a realistic dataset.\nCurrently, there are two options for datasets that you can generate.\n\n- American population dataset with given name, gender and ethnicity\n- German population dataset with given name, last name, gender, street name, municipality and postcode\n\n[You can check out and modify the scripts that generate these datasets in this repository.](gecko_examples/dataset)\n\n## Generating data\n\nTo generate datasets, use the `generate` subcommand.\n\n```\n$ gecko generate --help\nUsage: gecko generate [OPTIONS] {german|american}\n\n  Generate realistic-looking data with Gecko.\n\nOptions:\n  -n INTEGER                  Number of records to generate.\n  -s, --seed INTEGER          Seed for random number generator.\n  -o, --output-dir DIRECTORY  Directory to save generated datasets to.\n  --help                      Show this message and exit.\n```\n\nBy default, the command will generate and mutate a million records, use the same fixed seed and store results in\na directory called \"output\" within the project repository.\n\n```\n$ gecko generate german\nGenerating 1000000 records for `german` dataset...\nMutating generated dataset...\nSaving generated dataset to /home/mjugl/PycharmProjects/pprl-gecko-examples/output/german-generated.csv...\nSaving mutated dataset to /home/mjugl/PycharmProjects/pprl-gecko-examples/output/german-mutated.csv...\nDone!\n$ ls output\ngerman-generated.csv  german-mutated.csv\n```\n\n## Benchmarking Gecko\n\nTo measure the time Gecko needs to generate data, use the `benchmark` subcommand.\n\n```\n$ gecko benchmark --help\nUsage: gecko benchmark [OPTIONS] {german|american}\n\n  Test Gecko's performance by timing data generation.\n\nOptions:\n  -n INTEGER                      Number of records to generate.\n  -t, --time-unit [s|ms|us|ns]    Units to display execution times in.\n  -w, --warmup-iterations INTEGER\n                                  Number of times to run dataset generation\n                                  before measuring execution times.\n  -i, --iterations INTEGER        Number of times to run dataset generation.\n  -o, --output-file FILE          File to write recorded times (in\n                                  nanoseconds) to.\n  --help                          Show this message and exit.\n```\n\nBy default, Gecko will generate and mutate a million records.\nAt first, five new datasets will be generated before measuring execution time 100 times.\nThe script will then print out a summary.\n\n```\n$ gecko benchmark german\nWarming up  [####################################]  100%          \nMeasuring  [####################################]  100%          \nMin: 4666.92ms / Max: 4940.43ms\nQ5: 4674.10ms / Q10: 4680.11ms / Q25: 4690.75ms / Q50: 4769.14ms / Q75: 4845.33ms / Q90: 4891.32ms / Q95: 4904.24ms\n```\n\nBe aware that this command only gives a rough estimate of the performance you can expect from Gecko.\nSince it uses Python's timeit module to measure execution time, there are a lot of other factors that can affect your\nresults.\n\n# License\n\nGecko is released under the MIT License.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ful-mds%2Fgecko-examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ful-mds%2Fgecko-examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ful-mds%2Fgecko-examples/lists"}