{"id":16307221,"url":"https://github.com/karupanerura/dutil","last_synced_at":"2025-04-05T22:27:15.044Z","repository":{"id":224336549,"uuid":"762975514","full_name":"karupanerura/dutil","owner":"karupanerura","description":"Google Cloud Firestore datastore mode unofficial CLI client and utilities. (dutil is named from Datastore UTILity.)","archived":false,"fork":false,"pushed_at":"2025-02-27T05:14:52.000Z","size":152,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-01T22:17:24.322Z","etag":null,"topics":["google-cloud"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/karupanerura.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-25T08:07:32.000Z","updated_at":"2025-02-27T05:14:50.000Z","dependencies_parsed_at":"2024-06-08T11:24:47.094Z","dependency_job_id":"1f31a4fa-93b9-409b-8f69-91e2b7949cc8","html_url":"https://github.com/karupanerura/dutil","commit_stats":null,"previous_names":["karupanerura/datastore-cli","karupanerura/dutil"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karupanerura%2Fdutil","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karupanerura%2Fdutil/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karupanerura%2Fdutil/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karupanerura%2Fdutil/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/karupanerura","download_url":"https://codeload.github.com/karupanerura/dutil/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247410726,"owners_count":20934648,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["google-cloud"],"created_at":"2024-10-10T21:13:50.773Z","updated_at":"2025-04-05T22:27:15.037Z","avatar_url":"https://github.com/karupanerura.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dutil\n\nGoogle Cloud Firestore datastore mode unofficial CLI client and utilities. (dutil is named from `Datastore UTILity`.)\n\n## Limitation\n\nSupport for datastore mode only. (Patches welcome)\n\n## Examples\n\n```prompt\n$ dutil io lookup -p my-project1 'KEY(MyKind, \"foo\")' | dutil convert table\n$ dutil io gql -p my-project1  'SELECT * FROM MyKind WHERE prop \u003e 2'\n$ dutil io query MyKind -p my-project1 --ancestor 'KEY(MyParentKind, \"foo\")' \u003e dump.jsonl\n$ dutil io upsert -p my-project2 \u003c dump.jsonl\n$ dutil io query MyKind -p my-project1 --where 'prop \u003e 2' --keys-only --format=encoded | xargs dutil io delete -p my-project1\n```\n\n## Install\n\nPre-built binaries are available on: https://github.com/karupanerura/dutil/releases/tag/v0.1.4\n\n```prompt\n$ VERSION=0.1.4\n$ curl -sfLO https://github.com/karupanerura/dutil/releases/download/v${VERSION}/dutil_${VERSION}_$(go env GOOS)_$(go env GOARCH).tar.gz\n$ tar zxf dutil_${VERSION}_$(go env GOOS)_$(go env GOARCH).tar.gz\n$ install -m 0755 dutil $PREFIX\n$ rm dutil dutil_${VERSION}_$(go env GOOS)_$(go env GOARCH).tar.gz\n```\n\n## Usage\n\n### dutil io\n\nI/O utilities.\n\n```\nUsage: dutil io \u003ccommand\u003e\n\nFlags:\n  -h, --help       Show context-sensitive help.\n      --version    Show version\n\nCommands:\n  io lookup --projectId=STRING \u003ckeys\u003e ...\n\n  io query --projectId=STRING \u003ckind\u003e\n\n  io insert --projectId=STRING\n\n  io update --projectId=STRING\n\n  io upsert --projectId=STRING\n\n  io delete --projectId=STRING \u003ckeys\u003e ...\n\n  io gql --projectId=STRING \u003cquery\u003e\n```\n\n#### dutil io lookup\n\n```\nUsage: dutil io lookup --projectId=STRING \u003ckeys\u003e ...\n\nArguments:\n  \u003ckeys\u003e ...    Keys to lookup (format:\n                https://support.google.com/cloud/answer/6361641)\n\nFlags:\n  -h, --help                    Show context-sensitive help.\n      --version                 Show version\n\n  -p, --projectId=STRING        Google Cloud Project ID ($DATASTORE_PROJECT_ID)\n  -d, --databaseId=STRING       Cloud Datastore database ID\n  -n, --namespace=STRING        Cloud Datastore namespace\n      --emulator-host=STRING    Cloud Datastore emulator host\n                                ($DATASTORE_EMULATOR_HOST)\n      --with-metadata           Lookup with internal metadata in datastore\n                                (EXPERIMENTAL)\n```\n\nNOTE: `--with-metadata` is an experimental feature to lookup with datastore internal metadata.\nTo simplify implementation, it separates API calls for each key.\n\n#### dutil io query\n\n```\nUsage: dutil io query --projectId=STRING \u003ckind\u003e\n\nArguments:\n  \u003ckind\u003e    Entity kind\n\nFlags:\n  -h, --help                    Show context-sensitive help.\n      --version                 Show version\n\n  -p, --projectId=STRING        Google Cloud Project ID ($DATASTORE_PROJECT_ID)\n  -d, --databaseId=STRING       Cloud Datastore database ID\n  -n, --namespace=STRING        Cloud Datastore namespace\n      --emulator-host=STRING    Cloud Datastore emulator host\n                                ($DATASTORE_EMULATOR_HOST)\n      --key-format=\"json\"       Key format to output for keys only query\n\nQuery\n  --keys-only                    Return only keys of entities\n  --ancestor=STRING              Ancestor key to query (format:\n                                 https://support.google.com/cloud/answer/6361641)\n  --distinct\n  --distinctOn=DISTINCTON,...\n  --project=PROJECT,...\n  --filter=STRING                Entity filter query (format:\n                                 GQL compound-condition\n                                 https://cloud.google.com/datastore/docs/reference/gql_reference)\n  --order=ORDER,...              Comma separated property names with optional\n                                 '-' prefix for descending order\n  --limit=INT                    Limit number of entities to query\n  --offset=INT                   Offset number of entities to query\n  --explain                      Explain query execution plan\n\nAggregation\n  --count=COUNT            Count entities using aggregation query, the value\n                           is alias name of the count result. (e.g. --count= or\n                           --count=myAlias)\n  --sum=FIELD-AND-ALIAS    Sum entities field using aggregation query, the value\n                           is a target field name and optional alias name. (e.g.\n                           --sum=myField or --sum=myField=myAlias)\n  --avg=FIELD-AND-ALIAS    Average entities field using aggregation query, the\n                           value is a target field name and optional alias name.\n                           (e.g. --sum=myField or --sum=myField=myAlias)\n```\n\n#### dutil io gql\n\n```\nUsage: dutil io gql --projectId=STRING \u003cquery\u003e\n\nArguments:\n  \u003cquery\u003e    GQL Query\n\nFlags:\n  -h, --help                    Show context-sensitive help.\n  -p, --projectId=STRING        Google Cloud Project ID ($DATASTORE_PROJECT_ID)\n  -d, --databaseId=STRING       Cloud Datastore database ID\n  -n, --namespace=STRING        Cloud Datastore namespace\n      --emulator-host=STRING    Cloud Datastore emulator host ($DATASTORE_EMULATOR_HOST)\n```\n\n#### dutil io insert\n\n```\nUsage: dutil io insert --projectId=STRING\n\nFlags:\n  -h, --help                    Show context-sensitive help.\n      --version                 Show version\n\n  -p, --projectId=STRING        Google Cloud Project ID ($DATASTORE_PROJECT_ID)\n  -d, --databaseId=STRING       Cloud Datastore database ID\n  -n, --namespace=STRING        Cloud Datastore namespace\n      --emulator-host=STRING    Cloud Datastore emulator host\n                                ($DATASTORE_EMULATOR_HOST)\n  -f, --force                   Force insert without confirmation\n                                ($DATASTORE_CLI_FORCE_INSERT)\n  -c, --commit                  Commit transaction without confirmation\n  -s, --silent                  Silent mode\n```\n\n#### dutil io update\n\n```\nUsage: dutil io update --projectId=STRING\n\nFlags:\n  -h, --help                    Show context-sensitive help.\n      --version                 Show version\n\n  -p, --projectId=STRING        Google Cloud Project ID ($DATASTORE_PROJECT_ID)\n  -d, --databaseId=STRING       Cloud Datastore database ID\n  -n, --namespace=STRING        Cloud Datastore namespace\n      --emulator-host=STRING    Cloud Datastore emulator host\n                                ($DATASTORE_EMULATOR_HOST)\n  -f, --force                   Force update without confirmation\n                                ($DATASTORE_CLI_FORCE_UPDATE)\n  -c, --commit                  Commit transaction without confirmation\n  -s, --silent                  Silent mode\n```\n\n#### dutil io upsert\n\n```\nUsage: dutil io upsert --projectId=STRING\n\nFlags:\n  -h, --help                    Show context-sensitive help.\n      --version                 Show version\n\n  -p, --projectId=STRING        Google Cloud Project ID ($DATASTORE_PROJECT_ID)\n  -d, --databaseId=STRING       Cloud Datastore database ID\n  -n, --namespace=STRING        Cloud Datastore namespace\n      --emulator-host=STRING    Cloud Datastore emulator host\n                                ($DATASTORE_EMULATOR_HOST)\n  -f, --force                   Force upsert without confirmation\n                                ($DATASTORE_CLI_FORCE_UPSERT)\n  -c, --commit                  Commit transaction without confirmation\n  -s, --silent                  Silent mode\n```\n\n#### dutil io delete\n\n```\nUsage: dutil io delete --projectId=STRING \u003ckeys\u003e ...\n\nArguments:\n  \u003ckeys\u003e ...    Keys to delete (format:\n                https://support.google.com/cloud/answer/6361641)\n\nFlags:\n  -h, --help                    Show context-sensitive help.\n      --version                 Show version\n\n  -p, --projectId=STRING        Google Cloud Project ID ($DATASTORE_PROJECT_ID)\n  -d, --databaseId=STRING       Cloud Datastore database ID\n  -n, --namespace=STRING        Cloud Datastore namespace\n      --emulator-host=STRING    Cloud Datastore emulator host\n                                ($DATASTORE_EMULATOR_HOST)\n  -f, --force                   Force delete without confirmation\n                                ($DATASTORE_CLI_FORCE_DELETE)\n  -c, --commit                  Commit transaction without confirmation\n  -s, --silent                  Silent mode\n```\n\n### dutil convert\n\nData format converters.\n\n```\nUsage: dutil convert \u003ccommand\u003e\n\nFlags:\n  -h, --help       Show context-sensitive help.\n      --version    Show version\n\nCommands:\n  convert table\n```\n\n#### dutil convert table\n\n```\nUsage: dutil convert table\n\nFlags:\n  -h, --help             Show context-sensitive help.\n      --version          Show version\n\n  -f, --from=\"entity\"    Type of JSON structure to convert to table\n```\n\n## Format\n\nThis command dumps and upsert (insert or update) with [JSON Lines](https://jsonlines.org/) format.\nThe JSONs format written in TypeScript is following:\n\n### Key\n\nKey is a datastore key for entity.\n\n```typescript\ntype Key = {\n    namespace?: string\n    kind: string\n    name?: string\n    id?: number // int64\n    parent?: Key\n}\n```\n\n### Entity\n\nEntity is a datastore entity.\n\n```typescript\ntype Entity = {\n    key: Key\n    properties: Property[]\n}\n```\n\n### Property\n\nProperty is a property of datastore entity.\n\n```typescript\ntype Property = {\n    name: string\n    noIndex?: true\n} \u0026 Value\n```\n\n### Value\n\nValue is a value of datastore property.\nThis is object has the properties `type` and `value`.\nThe data type of `value` is determined by the `type` value.\n\n```typescript\ntype Value = {\n    type: \"array\"\n    value: Value[]\n} | {\n    type: \"blob\"\n    value: string // standard base64-encoded string\n} | {\n    type: \"bool\"\n    value: boolean\n} | {\n    type: \"timestamp\"\n    value: string // RFC3339 format\n} | {\n    type: \"entity\"\n    value: []Property\n} | {\n    type: \"float\"\n    value: number // float64\n} | {\n    type: \"array\"\n    value: []Value\n} | {\n    type: \"geo\"\n    value: GeoPoint // Explained in subsequent sections\n} | {\n    type: \"int\"\n    value: number // int64\n} | {\n    type: \"key\"\n    value: Key\n} | {\n    type: \"null\"\n    value: null\n} | {\n    type: \"string\"\n    value: string\n}\n```\n\n### GeoPoint\n\nGeoPoint represents a geographical point with latitude and longitude coordinates.\n\n```typescript\ntype GeoPoint = {\n    lat: number // float64\n    lng: number // float64\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarupanerura%2Fdutil","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkarupanerura%2Fdutil","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarupanerura%2Fdutil/lists"}