{"id":19651268,"url":"https://github.com/embulk/embulk-output-command","last_synced_at":"2025-04-28T16:31:20.687Z","repository":{"id":27512329,"uuid":"30993032","full_name":"embulk/embulk-output-command","owner":"embulk","description":"Command file output plugin for Embulk","archived":false,"fork":false,"pushed_at":"2024-05-15T13:48:43.000Z","size":137,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-04-16T04:47:11.018Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/embulk.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog","contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-02-18T23:39:41.000Z","updated_at":"2024-05-14T02:41:24.000Z","dependencies_parsed_at":"2024-05-13T01:42:00.641Z","dependency_job_id":"842bef90-7462-459a-9f2f-2b3e267cfa63","html_url":"https://github.com/embulk/embulk-output-command","commit_stats":{"total_commits":17,"total_committers":4,"mean_commits":4.25,"dds":0.4117647058823529,"last_synced_commit":"568a8fce470b13eb39fd71e159fb70bceea263e1"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-output-command","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-output-command/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-output-command/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/embulk%2Fembulk-output-command/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/embulk","download_url":"https://codeload.github.com/embulk/embulk-output-command/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251345893,"owners_count":21574804,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T15:05:49.649Z","updated_at":"2025-04-28T16:31:19.078Z","avatar_url":"https://github.com/embulk.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# embulk-output-command\n\nCommand file output plugin for Embulk: runs a command and writes formatted data to its stdin.\n\n## Overview\n\n* **Plugin type**: file output\n* **Load all or nothing**: depends on the command\n* **Resume supported**: depends on the command\n\n## Configuration\n\n- **command**: command line (string, required)\n\nThe **command** is exected using a shell (`sh -c` on UNIX/Linux, `PowerShell.exe -Command` on Windows). Therefore, it can include pipe (`|`), environment variables (`$VAR`), redirects, and so on.\n\nThe command runs `total-task-count * total-seqid-count` times. For example, if there is 3 local files and formatter produces 2 files for each input file, the command is executed for 6 times.\n\n### Environment variables\n\nThe command can use following environment variables:\n\n- **INDEX**: task index (0, 1, 2, ...). This depends on input. For example, the input is local files, incremental numbers for each file.\n- **SEQID**: file sequence id in a task. This depends on formatter. For example, if the formatter produces 2 files, the SEQID is 0 and 1.\n\nYou can use the combination of (INDEX, SEQID) as an unique identifier of a task.\n\n## Example\n\n### For UNIX/Linux\n\n```yaml\nout:\n  type: command\n  command: \"cat - \u003e task.$INDEX.$SEQID.csv\"\n  formatter:\n    type: csv\n```\n\n### For Windows\n\nTo refer Environment variables, you should use `${Env:ENVVAR}`.\nFor example, in powershell, you can refer `INDEX` and `SEQID` environment variables, which are defined by `embulk-output-command`, like this:\n\n```powershell\n${Env:INDEX} # refer INDEX environment variable\n${Env:SEQID} # refer SEQID environment variable\n```\n\nNote that `${input}` equals to `cat -` in PowerShell.\n\n```yaml\nout:\n  type: command\n  command: ${input} \u003e task.${Env:INDEX}.${Env:SEQID}.csv\n  formatter:\n    type: csv\n```\n\nFor Maintainers\n----------------\n\n### Release\n\nModify `version` in `build.gradle` at a detached commit, and then tag the commit with an annotation.\n\n```\ngit checkout --detach main\n\n(Edit: Remove \"-SNAPSHOT\" in \"version\" in build.gradle.)\n\ngit add build.gradle\n\ngit commit -m \"Release vX.Y.Z\"\n\ngit tag -a vX.Y.Z\n\n(Edit: Write a tag annotation in the changelog format.)\n```\n\nSee [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) for the changelog format. We adopt a part of it for Git's tag annotation like below.\n\n```\n## [X.Y.Z] - YYYY-MM-DD\n\n### Added\n- Added a feature.\n\n### Changed\n- Changed something.\n\n### Fixed\n- Fixed a bug.\n```\n\nPush the annotated tag, then. It triggers a release operation on GitHub Actions after approval.\n\n```\ngit push -u origin vX.Y.Z\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembulk%2Fembulk-output-command","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fembulk%2Fembulk-output-command","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fembulk%2Fembulk-output-command/lists"}