{"id":13484757,"url":"https://github.com/greymd/teip","last_synced_at":"2025-05-15T23:05:04.016Z","repository":{"id":38320876,"uuid":"268647751","full_name":"greymd/teip","owner":"greymd","description":"Masking tape to help commands \"do one thing well\"","archived":false,"fork":false,"pushed_at":"2024-02-15T00:44:17.000Z","size":393,"stargazers_count":586,"open_issues_count":9,"forks_count":19,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-05-08T15:16:23.504Z","etag":null,"topics":["awk","cli","command-line","grep","rust","sed","shell","terminal","tool"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/greymd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null},"funding":{"github":"greymd"}},"created_at":"2020-06-01T22:38:25.000Z","updated_at":"2025-04-25T03:30:57.000Z","dependencies_parsed_at":"2023-02-16T10:15:45.139Z","dependency_job_id":"e2b9dc47-cd7e-411f-b235-7fc4118e8bef","html_url":"https://github.com/greymd/teip","commit_stats":{"total_commits":220,"total_committers":11,"mean_commits":20.0,"dds":0.4363636363636364,"last_synced_commit":"e5914aa0fe3ae2049f9d5725ea35859e756181ed"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greymd%2Fteip","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greymd%2Fteip/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greymd%2Fteip/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/greymd%2Fteip/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/greymd","download_url":"https://codeload.github.com/greymd/teip/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254302045,"owners_count":22048013,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["awk","cli","command-line","grep","rust","sed","shell","terminal","tool"],"created_at":"2024-07-31T17:01:32.687Z","updated_at":"2025-05-15T23:05:03.854Z","avatar_url":"https://github.com/greymd.png","language":"Rust","readme":"\u003ch1 align=\"center\"\u003e\n  \u003cbr /\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/wiki/greymd/teip/img/logo.png\" width=\"208\" /\u003e\n  \u003ch4 align=\"center\"\u003eMasking tape to help commands \"do one thing well\"\u003c/h4\u003e\n\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/greymd/teip/releases/latest\"\u003e\u003cimg src=\"https://img.shields.io/github/release/greymd/teip.svg\" alt=\"Latest version\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://crates.io/crates/teip\" alt=\"crate.io\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/teip.svg\"/\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/greymd/teip/actions?query=workflow%3ATest\"\u003e\u003cimg src=\"https://github.com/greymd/teip/actions/workflows/test.yml/badge.svg\" alt=\"Test Status\" /\u003e\u003c/a\u003e\n  \u003ca href=\"LICENSE\" alt=\"MIT License\"\u003e\u003cimg src=\"http://img.shields.io/badge/license-MIT-blue.svg?style=flat\" /\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n## Taping\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/wiki/greymd/teip/img/teip_intro2.png\" alt=\"Git Animation for Introduction\" width=\"80%\" /\u003e\n\u003c/p\u003e\n\n* Convert timestamps in /var/log/secure to UNIX time\n\n```bash\n$ cat /var/log/secure | teip -c 1-15 -- date -f- +%s\n```\n\n* Replace 'WORLD' with 'EARTH' on lines containing 'HELLO'\n\n```bash\n$ cat file | teip -g HELLO -- sed 's/WORLD/EARTH/'\n```\n\n* Make characters upper case in the 2nd field of a CSV (RFC4180)\n\n```bash\n$ cat file.csv | teip --csv -f 2 -- tr a-z A-Z\n```\n\n* Edit the 2nd, 3rd and 4th fields of a TSV file\n\n```bash\n$ cat file.tsv | teip -D '\\t' -f 2-4 -- tr a-z A-Z\n```\n\n* Edit lines containing 'hello' and the three lines before and after\n\n```bash\n$ cat access.log | teip -e 'grep -n -C 3 hello' -- sed 's/./@/g'\n```\n\n## Performance enhancement\n\n`teip` allows a command to focus on its own task.\n\nHere is a comparison of the processing time to replace approx 761,000 IP addresses with dummy ones in a 100 MiB text file.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/wiki/greymd/teip/benchmark/secure_bench.svg\" width=\"80%\" alt=\"benchmark bar chart\" /\u003e\n\u003c/p\u003e\n\nSee detail at \u003ca href=\"https://github.com/greymd/teip/wiki/Benchmark\"\u003ewiki \u003e Benchmark\u003c/a\u003e.\n\n## Features\n\n* Taping: Help the command \"do one thing well\"\n  - Passing a partial range of the standard input to any command — whatever you want\n  - The targeted command just actions the passed parts of the standard input\n  - Flexible methods for selecting a range (Select like `awk`, `cut` or `grep`)\n\n* High performance\n  - The targeted command's standard input/output are written to and read from by multiple `teip` threads asynchronously.\n  - If general UNIX commands in your environment can process a few-hundred MB file in a few seconds, then `teip` can do the same or better performance.\n\n## Installation\n\n### macOS (x86_64, ARM64) / Linux (x86_64)\n\nInstall [Homebrew](https://brew.sh/), and\n\n```bash\nbrew install teip\n```\n\n### Linux (x86_64, ARM64)\n\n#### `dpkg`\n\n\u003c!-- deb_url_start --\u003e\n```bash\nwget https://github.com/greymd/teip/releases/download/v2.3.2/teip-2.3.2.$(uname -m)-unknown-linux-musl.deb\nsudo dpkg -i ./teip*.deb\n```\n\u003c!-- deb_url_end --\u003e\n\n#### `apt`\n\n\u003c!-- deb_url_start --\u003e\n```bash\nwget https://github.com/greymd/teip/releases/download/v2.3.2/teip-2.3.2.$(uname -m)-unknown-linux-musl.deb\nsudo apt install ./teip*.deb\n```\n\u003c!-- deb_url_end --\u003e\n\n#### `dnf`\n\n\u003c!-- rpm_url_start --\u003e\n```bash\nsudo dnf install https://github.com/greymd/teip/releases/download/v2.3.2/teip-2.3.2.$(uname -m)-unknown-linux-musl.rpm\n```\n\u003c!-- rpm_url_end --\u003e\n\n#### `yum`\n\n\u003c!-- rpm_url_start --\u003e\n```bash\nsudo yum install https://github.com/greymd/teip/releases/download/v2.3.2/teip-2.3.2.$(uname -m)-unknown-linux-musl.rpm\n```\n\u003c!-- rpm_url_end --\u003e\n\n\u003c!-- release_url_start --\u003e\nIf necessary, check the hash value from the [latest release page](https://github.com/greymd/teip/releases/tag/v2.3.2).\nFiles whose filenames end with `sha256` have hash values listed.\n\u003c!-- release_url_end --\u003e\n\n\n### Windows (x86_64)\n\n\u003c!-- ins_url_start --\u003e\nDownload installer from [here](https://github.com/greymd/teip/releases/download/v2.3.2/teip_installer-2.3.2-x86_64-pc-windows-msvc.exe).\n\u003c!-- ins_url_end --\u003e\n\nSee [Wiki \u003e Use on Windows](https://github.com/greymd/teip/wiki/Use-on-Windows) for detail.\n\n### Other architectures\n\n\u003c!-- release_url_start --\u003e\nCheck the [latest release page](https://github.com/greymd/teip/releases/tag/v2.3.2) for executables for the platform you are using.\n\u003c!-- release_url_end --\u003e\n\nIf not present, please build teip from source.\n\n### Build from source\n\nWith Rust's package manager cargo\n\n```\ncargo install teip\n```\n\nTo enable Oniguruma regular expression (`-G` option), build with `--features oniguruma` option.\nPlease make sure the `libclang` shared library is available in your environment.\n\n```bash\n### Ubuntu\n$ sudo apt install cargo clang\n$ cargo install teip --features oniguruma\n```\n\n```bash\n### Red Hat base OS\n$ sudo dnf install cargo clang\n$ cargo install teip --features oniguruma\n```\n\n```powershell\n### Windows (PowerShell) and choco (chocolatey.org)\nPS C:\\\u003e choco install llvm\nPS C:\\\u003e cargo install teip --features oniguruma\n```\n\n## Usage\n\n```\nUSAGE:\n  teip -g \u003cpattern\u003e [-Gosvz] [--] [\u003ccommand\u003e...]\n  teip -c \u003clist\u003e [-svz] [--] [\u003ccommand\u003e...]\n  teip -l \u003clist\u003e [-svz] [--] [\u003ccommand\u003e...]\n  teip -f \u003clist\u003e [-d \u003cdelimiter\u003e | -D \u003cpattern\u003e | --csv] [-svz] [--] [\u003ccommand\u003e...]\n  teip -e \u003cstring\u003e [-svz] [--] [\u003ccommand\u003e...]\n\nOPTIONS:\n    -g \u003cpattern\u003e        Act on lines that match the regular expression \u003cpattern\u003e.\n        -o              -g acts on only matched ranges.\n        -G              -g interprets Oniguruma regular expressions.\n    -c \u003clist\u003e           Act on these characters.\n    -l \u003clist\u003e           Act on these lines.\n    -f \u003clist\u003e           Act on these white-space separated fields.\n        -d \u003cdelimiter\u003e  Use \u003cdelimiter\u003e for the field delimiter of -f.\n        -D \u003cpattern\u003e    Use regular expression \u003cpattern\u003e for the field delimiter of -f\n        --csv           -f interprets \u003clist\u003e as field numbers of a CSV according to\n                        RFC 4180, instead of whitespace separated fields.\n    -e \u003cstring\u003e         Execute \u003cstring\u003e in another process that will receive identical\n                        standard input as the main teip command, emitting numbers to be\n                        used as line numbers for actioning.\n\nFLAGS:\n    -h, --help          Prints help information.\n    -V, --version       Prints version information.\n    -s                  Execute a new command for each actioned chunk.\n        --chomp         The command spawned by -s receives the standard input without\n                        trailing newlines.\n    -I  \u003creplace-str\u003e   Replace the \u003creplace-str\u003e with the actioned chunk in \u003ccommand\u003e,\n                        implying -s.\n    -v                  Invert the range of actioning.\n    -z                  Line delimiter is NUL instead of a newline.\n\nALIASES:\n    -g \u003cpattern\u003e\n        -A \u003cnumber\u003e     Alias of -e 'grep -n -A \u003cnumber\u003e \u003cpattern\u003e'\n        -B \u003cnumber\u003e     Alias of -e 'grep -n -B \u003cnumber\u003e \u003cpattern\u003e'\n        -C \u003cnumber\u003e     Alias of -e 'grep -n -C \u003cnumber\u003e \u003cpattern\u003e'\n    --sed \u003cpattern\u003e     Alias of -e 'sed -n \"\u003cpattern\u003e=\"'\n    --awk \u003cpattern\u003e     Alias of -e 'awk \"\u003cpattern\u003e{print NR}\"'\n```\n\n## Getting Started\n\nTry this at first:\n\n```bash\n$ echo \"100 200 300 400\" | teip -f 3\n```\n\nThe result is almost the same as the input, but \"300\" is highlighted and surrounded by `[...]`,\nbecause `-f 3` specifies the 3rd field of space-separated input.\n\n```bash\n100 200 [300] 400\n```\n\nUnderstand that the area enclosed in `[...]` is a **hole** in the masking tape.\n\n\u003cimg src=\"https://raw.githubusercontent.com/wiki/greymd/teip/img/teip_hole.png\" width=\"300\" /\u003e\n\nNext, put the `sed` and its arguments at the end.\n\n```bash\n$ echo \"100 200 300 400\" | teip -f 3 sed 's/./@/g'\n```\n\nThe result is as below.\nThe highlight and `[...]` will not be present when a command is added.\n\n```\n100 200 @@@ 400\n```\n\nAs you can see, the `sed` command only acted on the input defined by the \"hole\" and ignored the masked\nparts.  Technically, `teip` passes only the highlighted part to the `sed` process, and replaces the\nhighlighted part with the result of the `sed` command.\n\nOf course, any command you like can be specified.\nIt is called the **targeted command** in this article.\n\nLet's try `cut` as the targeted command, to extract the first character only.\n\n```bash\n$ echo \"100 200 300 400\" | teip -f 3 cut -c 1\nteip: Invalid arguments.\n```\n\nOops! Why did this fail?\n\nThis is because the `cut`command  uses the `-c` option.\nAn option of the same name is also provided by `teip`, which is confusing.\n\nWhen specifying a targeted command to `teip`, it is better to give it after `--`.\nThen, `teip` interprets any arguments after `--` as the targeted command and its arguments.\n\n```bash\n$ echo \"100 200 300 400\" | teip -f 3 -- cut -c 1\n100 200 3 400\n```\n\nGreat — the first character `3` is extracted from `300`!\n\nAlthough `--` is not always necessary, it is always better to use it.\nSo, `--` is used in all the examples from here on.\n\nNow let's double these number with `awk`.\nThe command looks like the following (Note that the variable to be doubled is not `$3`).\n\n```bash\n$ echo \"100 200 300 400\" | teip -f 3 -- awk '{print $1*2}'\n100 200 600 400\n```\n\nOK, the selection in the \"hole\" went from 300 to 600.\n\nNow, let's change `-f 3` to `-f 3,4` and run teip.\n\n```bash\n$ echo \"100 200 300 400\" | teip -f 3,4 -- awk '{print $1*2}'\n100 200 600 800\n```\n\nThe numbers in the 3rd and 4th fields were doubled!\n\nAs you may have noticed, the argument to `-f` is compatible with the __LIST__ of `cut`.\nYou can refer to `cut --help` to see how it works.\n\nExamples:\n\n```bash\n$ echo \"100 200 300 400\" | teip -f -3 -- sed 's/./@/g'\n@@@ @@@ @@@ 400\n\n$ echo \"100 200 300 400\" | teip -f 2-4 -- sed 's/./@/g'\n100 @@@ @@@ @@@\n\n$ echo \"100 200 300 400\" | teip -f 1- -- sed 's/./@/g'\n@@@ @@@ @@@ @@@\n```\n\n## Select range by character\n\nThe `-c` option allows you to specify a range by character.\nThe below example is specifying the 1st, 3rd, 5th, 7th characters and applying the `sed` command to them.\n\n```bash\n$ echo ABCDEFG | teip -c 1,3,5,7\n[A]B[C]D[E]F[G]\n\n$ echo ABCDEFG | teip -c 1,3,5,7 -- sed 's/./@/'\n@B@D@F@\n```\n\nLike `-f`, the argument to `-c` is compatible with `cut`'s __LIST__.\n\n## Processing delimited text like CSV and TSV\n\nThe `-f` option recognizes delimited fields [like `awk`](https://www.gnu.org/software/gawk/manual/html_node/Regexp-Field-Splitting.html) by default.\n\nAny continuous whitespace (all forms of whitespace categorized by [Unicode](https://www.unicode.org/Public/UCD/latest/ucd/PropList.txt)) is interpreted as a single delimiter.\n\n```bash\n$ printf \"A       B \\t\\t\\t\\   C \\t D\" | teip -f 3 -- sed s/./@@@@/\nA       B                       @@@@   C         D\n```\n\nThis behavior might be inconvenient for the processing of CSV and TSV.\n\nHowever, the `-d` option in conjunction with `-f` can be used to specify a delimiter.\nYou can process a simple CSV file like this:\n\n```bash\n$ echo \"100,200,300,400\" | teip -f 3 -d , -- sed 's/./@/g'\n100,200,@@@,400\n```\n\nIn order to process TSV, the TAB character must be given at the command line.\nIf you are using Bash, type `$'\\t'` which is in the form of [ANSI-C Quoting](https://www.gnu.org/software/bash/manual/html_node/ANSI_002dC-Quoting.html).\n\n```bash\n$ printf \"100\\t200\\t300\\t400\\n\" | teip -f 3 -d $'\\t' -- sed 's/./@/g'\n100     200     @@@     400\n```\n\n`teip` also provides `-D` option to specify an extended regular expression as the delimiter.\nThis is useful when you want to ignore consecutive delimiters, or when there are multiple types of delimiter.\n\n```bash\n$ echo 'A,,,,,B,,,,C' | teip -f 2 -D ',+'\nA,,,,,[B],,,,C\n```\n\n```bash\n$ echo \"1970-01-02 03:04:05\" | teip -f 2-5 -D '[-: ]'\n1970-[01]-[02] [03]:[04]:05\n```\n\nThe TAB character regular expression (`\\t`) can also be specified with the `-D` option.\n\n```\n$ printf \"100\\t200\\t300\\t400\\n\" | teip -f 3 -D '\\t' -- sed 's/./@/g'\n100     200     @@@     400\n```\n\nFor the available regular expression notations, refer to [regular expression of Rust](https://docs.rs/regex/1.3.7/regex/).\n\n## Complex CSV processing\n\nIf you want to process a complex CSV file, such as the one below, which has columns surrounded by double quotes, use the `-f` option together with the `--csv` option.\n\n```csv\nName,Address,zipcode\nSola Harewatar,\"Doreami Road 123\nSorashido city\",12877\nYui Nagomi,\"Nagomi Street 456, Nagomitei, Oishina town\",26930-0312\n\"Conectol Motimotit Hooklala Glycogen Comex II a.k.a \"\"Kome kome\"\"\",\"Cooking dam\",513123\n```\n\nWith `--csv`, teip will parse the input as a CSV file according to [RFC4180](https://www.rfc-editor.org/rfc/rfc4180). Thus, you can use `-f` to specify column numbers for CSV files with complex structures.\n\nFor example, the CSV above will have a \"hole\" as shown below.\n\n```\n$ cat tests/sample.csv | teip --csv -f2 \nName,[Address],zipcode\nSola Harewatar,[\"Doreami Road 123]\n[Sorashido city\"],12877\nYui Nagomi,[\"Nagomi Street 456, Nagomitei, Oishina town\"],26930-0312\n\"Conectol Motimotit Hooklala Glycogen Comex II a.k.a \"\"Kome kome\"\"\",[\"Cooking dam\"],513123\n```\n\nBecause `-f2` was specified, there is a hole in the second column of each row.\nThe following command is an example of rewriting all characters in the second column to \"@\".\n\n```\n$ cat tests/sample.csv  | teip --csv -f2 -- sed 's/[^\"]/@/g'\nName,@@@@@@@,zipcode\nSola Harewatar,\"@@@@@@@@@@@@@@@@\n@@@@@@@@@@@@@@\",12877\nYui Nagomi,\"@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\",26930-0312\n\"Conectol Motimotit Hooklala Glycogen Comex II a.k.a \"\"Kome kome\"\"\",\"@@@@@@@@@@@\",513123\n```\n\nNotes for the `--csv` option:\n\n* Double quotes `\"` surrounding fields are also included in the holes.\n* Escaped double quotes `\"\"` are treated as-is; two double quotes `\"\"` are given as input to the targeted command.\n* Fields containing newlines will have multiple holes, separated by newlines, instead of a single hole.\n  * However, if the `-s` or `-z` option is used, such a field is treated as a single hole, and line breaks are included.\n\n## Matching with Regular Expressions\n\nYou can also use `-g` to select a specific line matching a regular expression as the hole location.\n\n```bash\n$ echo -e \"ABC1\\nEFG2\\nHIJ3\" | teip -g '[GJ]\\d'\nABC1\n[EFG2]\n[HIJ3]\n```\n\nBy default, the entire line containing the pattern is the range of holes.\nWith the -o option, the range of the holes will ony cover the matched range.\n\n```bash\n$ echo -e \"ABC1\\nEFG2\\nHIJ3\" | teip -og '[GJ]\\d'\nABC1\nEF[G2]\nHI[J3]\n```\n\nNote that `-og` is one of the most useful idioms and is frequently used in this manual.\n\nHere is an example using `\\d`, which matches numbers.\n\n```bash\n$ echo ABC100EFG200 | teip -og '\\d+'\nABC[100]EFG[200]\n\n$ echo ABC100EFG200 | teip -og '\\d+' -- sed 's/.*/@@@/g'\nABC@@@EFG@@@\n```\n\nThis feature is quite versatile and can be useful for handling files that have no fixed form such as logs, markdown, etc.\n\n## What commands are appropriate?\n\n`teip` passes the strings from the hole line-by-line, so that each hole is one line of input.\nTherefore, a targeted command must follow the below rule.\n\n* **A targeted command must print a single line of result for each line of input.**\n\nIn the simplest example, the `cat` command always succeeds,\nbecause the `cat` prints the same number of lines as it is given in input.\n\n```bash\n$ echo ABCDEF | teip -og . -- cat\nABCDEF\n```\n\nIf the above rule is not satisfied, the result will be inconsistent.\nFor example, `grep` may fail.\nHere is an example.\n\n```bash\n$ echo ABCDEF | teip -og .\n[A][B][C][D][E][F]\n\n$ echo ABCDEF | teip -og . -- grep '[ABC]'\nABC\nteip: Output of given command is exhausted\n\n$ echo $?\n1\n```\n\n`teip` did not receive results corresponding to the holes of D, E, and F.\nThat is why the above example fails.\n\nIf an inconsistency occurs, `teip` will exit with the error message.\nAlso, the exit status will be 1.\n\nTo learn more about `teip`'s behavior, see [Wiki \u003e Chunking](https://github.com/greymd/teip/wiki/Chunking).\n\n## Advanced usage\n\n### Solid mode (`-s`)\n\nIf you want to use a command that does not satisfy the condition, **\"A targeted command must print a single line of result for each line of input\"**, enable \"Solid mode\" with the `-s` option.\n\nSolid mode spawns the targeted command multiple times: once for each hole in the input.\n\n```bash\n$ echo ABCDEF | teip -s -og . -- grep '[ABC]'\n```\n\nIn the above example, understand that the following commands are executed by `teip` internally:\n\n```bash\n$ echo A | grep '[ABC]' # =\u003e A\n$ echo B | grep '[ABC]' # =\u003e B\n$ echo C | grep '[ABC]' # =\u003e C\n$ echo D | grep '[ABC]' # =\u003e Empty\n$ echo E | grep '[ABC]' # =\u003e Empty\n$ echo F | grep '[ABC]' # =\u003e Empty\n```\n\nThe empty result is replaced with an empty string.\nTherefore, D, E, and F are replaced with the empty string.\n\n```bash\n$ echo ABCDEF | teip -s -og . -- grep '[ABC]'\nABC\n\n$ echo $?\n0\n```\n\nHowever, this option is not suitable for processing large files because of its high overhead, which can significantly degrade performance.\n\n#### Solid mode with placeholder (`-I \u003creplace-str\u003e`)\n\nIf you want to use the contents of the hole as an argument to the targeted command, use the `-I` option.\n\n```bash\n$ echo AAA BBB CCC | teip -f 2 -I @ -- echo '[@]'\nAAA [BBB] CCC\n```\n\n`\u003creplace-str\u003e` can be any string, and multiple characters are allowed.\n\n```bash\n$ seq 5 | teip -f 1 -I NUMBER -- awk 'BEGIN{print NUMBER * 3}'\n3\n6\n9\n12\n15\n```\n\nPlease note that `-s` is automatically enabled with `-I`.\nTherefore, it is not suitable for processing huge files.\nIn addition, the targeted command does not get any input from stdin.\nThe targeted command is expected to work without stdin.\n\n#### Solid mode with `--chomp`\n\nIf the `-s` option does not work as expected, `--chomp` may be helpful.\n\nA targeted command in solid mode always accepts input with a newline (`\\x0A`) at the end.\nThis is because `teip` assumes the use of commands which return a single line of output in response to a single line of input.\nTherefore, even if there is no line break in the hole, a line break is added, to ensure it is treated as a single line of input.\n\nHowever, there are situations where this behavior is inconvenient.\nFor example, when using commands whose behavior changes depending on the presence or absence of a newline.\n\n```bash\n$ echo AAABBBCCC | teip -og BBB -s\nAAA[BBB]CCC\n$ echo AAABBBCCC | teip -og BBB -s -- tr '\\n' '@'\nAAABBB@CCC\n```\n\nThe above is an example where the targeted command is: \"`tr` command which converts newlines (`\\x0A`) to @\".\n\"BBB\" does not contain a newline, but the output is \"BBB@\", because implicitly-added line breaks have been processed.\nTo prevent this behavior, use the `--chomp` option.\nThis option gives the targeted command pure input with no newlines added.\n\n```bash\n$ echo AAABBBCCC | teip -og BBB -s --chomp -- tr '\\n' '@'\nAAABBBCCC\n```\n\n`--chomp` is useful whenever using commands which interpret and process input as binary such as `tr`.\nBelow is an example of \"removing newlines from the second column of a CSV which contains newlines.\n\n```bash\n$ cat tests/sample.csv\nName,Address,zipcode\nSola Harewatar,\"Doreami Road 123\nSorashido city\",12877\n```\n\nThe result is:\n\n```bash\n$ cat tests/sample.csv | teip --csv -f 2 -s --chomp -- tr '\\n' '@'\nName,Address,zipcode\nSola Harewatar,\"Doreami Road 123@Sorashido city\",12877\n```\n\n### Line number (`-l`)\n\nYou can specify a line number and drill holes only in that line.\n\n```bash\n$ echo -e \"ABC\\nDEF\\nGHI\" | teip -l 2\nABC\n[DEF]\nGHI\n```\n\n```bash\n$ echo -e \"ABC\\nDEF\\nGHI\" | teip -l 1,3\n[ABC]\nDEF\n[GHI]\n```\n\n### Overlay `teip`s\n\nAny command can be used with `teip`, surprisingly, even if it is **`teip` itself**.\n\n```bash\n$ echo \"AAA@@@@@AAA@@@@@AAA\" | teip -og '@.*@'\nAAA[@@@@@AAA@@@@@]AAA\n\n$ echo \"AAA@@@@@AAA@@@@@AAA\" | teip -og '@.*@' -- teip -og 'A+'\nAAA@@@@@[AAA]@@@@@AAA\n\n$ echo \"AAA@@@@@AAA@@@@@AAA\" | teip -og '@.*@' -- teip -og 'A+' -- tr A _\nAAA@@@@@___@@@@@AAA\n```\n\nIn other words, by composing multiple functions of `teip` with AND conditions, it is possible to drill holes in a more complex range.\nFurthermore, this works asynchronously and in multi-processes, similar to the shell pipeline.\nPerformance will hardly degrade unless the machine reaches the limits of parallelism.\n\n### Oniguruma regular expression (`-G`)\n\nIf `-G` option is given together with `-g`, the regular expressin is interpreted as an [Oniguruma regular expression](https://github.com/kkos/oniguruma/blob/master/doc/RE).\nFor example, \"keep\" and \"look-ahead\" syntax can be used.\n\n```bash\n$ echo 'ABC123DEF456' | teip -G -og 'DEF\\K\\d+'\nABC123DEF[456]\n\n$ echo 'ABC123DEF456' | teip -G -og '\\d+(?=D)'\nABC[123]DEF456\n```\n\n### Empty holes\n\nIf a blank field exists when the `-f` option is used, the blank is not ignored and is treated as an empty hole.\n\n```bash\n$ echo ',,,' | teip -d , -f 1-\n[],[],[],[]\n```\n\nTherefore, the following command can work (Note that `.*` matches empty values as well).\n\n```bash\n$ echo ',,,' | teip -f 1- -d, sed 's/.*/@@@/'\n@@@,@@@,@@@,@@@\n```\n\nIn the above example, the `sed` command reads four newline characters and prints `@@@` four times.\n\n### Invert match (`-v`)\n\nThe `-v` option allows you to invert the range of holes.\nWhen the `-f` or `-c` option is used with `-v`, holes are made in the complement of the specified field.\n\n```bash\n$ echo 1 2 3 4 5 | teip -v -f 1,3,5 -- sed 's/./_/'\n1 _ 3 _ 5\n```\n\nOf course, `-v` can also be used with `-og`.\n\n```bash\n$ printf 'AAA\\n123\\nBBB\\n' | teip -og '\\d+' -- sed 's/./@/g'\n@@@\n123\n@@@\n```\n\n### Zero-terminated mode (`-z`)\n\nIf you want to process the data in a more flexible way, the `-z` option may be useful.\nThis option allows you to use the NUL character (the ASCII NUL character) as a line delimiter, instead of the newline character.\nIt behaves like `-z` provided by GNU sed or GNU grep, or the `-0` option provided by xargs.\n\n```bash\n$ printf '111,\\n222,33\\n3\\0\\n444,55\\n5,666\\n' | teip -z -f3 -d,\n111,\n222,[33\n3]\n444,55\n5,[666]\n```\n\nWith this option, the standard input is interpreted per each NUL character rather than per each newline character.\nYou should also pay attention to the fact that strings in the hole have the NUL character appended instead of a newline character.\n\nIf you use a targeted command that cannot handle NUL characters (and cannot print NUL-separated results), the final result can be unintended.\n\n```bash\n$ printf '111,\\n222,33\\n3\\0\\n444,55\\n5,666\\n' | teip -z -f3 -d, -- sed -z 's/.*/@@@/g'\n111,\n222,@@@\n444,55\n5,@@@\n\n$ printf '111,\\n222,33\\n3\\0\\n444,55\\n5,666\\n' | teip -z -f3 -d, -- sed 's/.*/@@@/g'\n111,\n222,@@@\n@@@\n444,55\n5,teip: Output of given command is exhausted\n```\n\nThis option is useful for treating multiple lines as a single combined input.\n\n```bash\n$ cat test.html | teip -z -og '\u003cbody\u003e.*\u003c/body\u003e'\n\u003chtml\u003e\n\u003chead\u003e\n  \u003ctitle\u003eAAA\u003c/title\u003e\n\u003c/head\u003e\n[\u003cbody\u003e\n  \u003cdiv\u003eAAA\u003c/div\u003e\n  \u003cdiv\u003eBBB\u003c/div\u003e\n  \u003cdiv\u003eCCC\u003c/div\u003e\n\u003c/body\u003e]\n\u003c/html\u003e\n\n$ cat test.html | teip -z -og '\u003cbody\u003e.*\u003c/body\u003e' -- grep -a BBB\n\u003chtml\u003e\n\u003chead\u003e\n  \u003ctitle\u003eAAA\u003c/title\u003e\n\u003c/head\u003e\n  \u003cdiv\u003eBBB\u003c/div\u003e\n\u003c/html\u003e\n```\n\n### External execution for match offloading (`-e`)\n\n`-e` is the option to use external commands to define pattern matching.\nWithout `-e`, you must use `teip`'s own functions, such as `-c` or `-g`, to control the position of the holes on the masking tape.\nWith `-e`, however, you can use the external commands you are familiar with to specify the range of holes.\n\n`-e` allows you to specify a shell pipeline as a string.\nOn a UNIX-like OS, this pipeline is executed via `/bin/sh`; on Windows via `cmd.exe`.\n\nFor example, given a simple pipeline `echo 3`, which outputs `3`, only the third line will be actioned by teip.\n\n```bash\n$ echo -e 'AAA\\nBBB\\nCCC' | teip -e 'echo 3'\nAAA\nBBB\n[CCC]\n```\n\nThis works even if the `-e` output is somewhat \"dirty\".\nFor example, if spaces or tab characters are included at the beginning of the `-e` output, they are ignored.\nAlso, once a number is seen, all non-numerical characters to the right of the number are ignored.\n\n```bash\n$ echo -e 'AAA\\nBBB\\nCCC' | teip -e 'echo \" 3\"'\nAAA\nBBB\n[CCC]\n$ echo -e 'AAA\\nBBB\\nCCC' | teip -e 'echo \" 3:testtest\"'\nAAA\nBBB\n[CCC]\n```\n\nTechnically, the first captured group in the regular expression `^\\s*([0-9]+)` is interpreted as a line number.\n\n`-e` will also recognize multiple numbers if the pipeline provides multiple lines of numbers.\nFor example, the `seq` command to display only odd numbers up to 10 is\n\n```bash\n$ seq 1 2 10\n1\n3\n5\n7\n9\n```\n\nThis means that only odd-numbered rows can be actioned by specifying the following:\n\n```bash\n$ echo -e 'AAA\\nBBB\\nCCC\\nDDD\\nEEE\\nFFF' | teip -e 'seq 1 2 10' -- sed 's/. /@/g'\n@@@\nBBB\n@@@\nDDD\n@@@\nFFF\n```\n\nNote that the order of the numbers must be ascending.\nNow, on its own, this looks like a feature that is just a slight improvement of the `-l` option.\n\nHowever, the breakthrough feature of `-e` is that **the pipeline obtains identical standard input as the main `teip` command**.\nThus, it generates output using not only `seq` and `echo`, but also commands such as `grep`, `sed`, and `awk`, which process the standard input.\n\nLet's look at a more concrete example.\nThe following is a `grep` command that prints **the line numbers of the line containing the string \"CCC\" and the two lines after it**.\n\n```bash\n$ echo -e 'AAA\\nBBB\\nCCC\\nDDD\\nEEE\\nFFF' | grep -n -A 2 CCC\n3:CCC\n4-DDD\n5-EEE\n```\n\nIf you give this command to `-e`, you can punch holes in **the line containing the string \"CCC\" and the two lines after it**!\n\n```bash\n$ echo -e 'AAA\\nBBB\\nCCC\\nDDD\\nEEE\\nFFF' | teip -e 'grep -n -A 2 CCC'\nAAA\nBBB\n[CCC]\n[DDD]\n[EEE]\nFFF\n```\n\n`grep` is not the only useful command for `-e`.\nGNU `sed` has `=`, which prints the line number being processed.\nBelow is an example of how to use `=` to drill from the line containing \"BBB\" to the line containing \"EEE\".\n\n```bash\n$ echo -e 'AAA\\nBBB\\nCCC\\nDDD\\nEEE\\nFFF' | teip -e 'sed -n \"/BBB/,/EEE/=\"'\nAAA\n[BBB]\n[CCC]\n[DDD]\n[EEE]\nFFF\n```\n\nOf course, similar operations can also be done with `awk`.\n\n```bash\n$ echo -e 'AAA\\nBBB\\nCCC\\nDDD\\nEEE\\nFFF' | teip -e 'awk \"/BBB/,/EEE/{print NR}\"'\n```\n\nThe following is an example of combining the commands `nl` and `tail`.\nYou can make holes in only the last three lines of input!\n\n```bash\n$ echo -e 'AAA\\nBBB\\nCCC\\nDDD\\nEEE\\nFFF' | teip -e 'nl -ba | tail -n 3'\nAAA\nBBB\nCCC\n[DDD]\n[EEE]\n[FFF]\n```\n\nThe argument to `-e` is a single string.\nThe pipe (`|`) and other symbols can be used within it.\n\n### Alias options (`-A`, `-B`, `-C`, `--awk`, `--sed`)\n\nThere are several **experimental options** which are aliases of `-e` and specific directives.\nThese options may be discontinued in the future since they are only experimental.\nDo not use them in a script or something that is not a one-off.\n\n#### `-A \u003cnumber\u003e`\n\nThis is an alias of `-e 'grep -n -A \u003cnumber\u003e \u003cpattern\u003e'`.\nIf it is used together with `-g \u003cpattern\u003e`, it makes holes in rows matching `\u003cpattern\u003e`, and `\u003cnumber\u003e` rows after the match.\n\n```bash\n$ cat AtoG.txt | teip -g B -A 2\nA\n[B]\n[C]\n[D]\nE\nF\nG\n```\n\n\n#### `-B \u003cnumber\u003e`\n\nThis is an alias of `-e 'grep -n -B \u003cnumber\u003e \u003cpattern\u003e'`\nIf it is used together with `-g \u003cpattern\u003e`, it makes holes in rows matching `\u003cpattern\u003e`, and `\u003cnumber\u003e` rows before the match.\n\n```\n$ cat AtoG.txt | teip -g E -B 2\nA\nB\n[C]\n[D]\n[E]\nF\nG\n```\n\n\n\n#### `-C \u003cnumber\u003e`\n\nThis is an alias of `-e 'grep -n -C \u003cnumber\u003e \u003cpattern\u003e'`.\nIf it is used together with `-g \u003cpattern\u003e`, it makes holes in rows matching `\u003cpattern\u003e`, and `\u003cnumber\u003e` rows before and after the match.\n\n```bash\n$ cat AtoG.txt | teip -g E -C 2\nA\nB\n[C]\n[D]\n[E]\n[F]\n[G]\n```\n\n#### `--sed \u003cpattern\u003e`\n\nThis is an alias of `-e 'sed -n \"\u003cpattern\u003e=\"`.\n\n```bash\n$ cat AtoG.txt | teip --sed '/B/,/E/'\nA\n[B]\n[C]\n[D]\n[E]\nF\nG\n```\n\n```bash\n$ cat AtoG.txt | teip --sed '1~3'\n[A]\nB\nC\n[D]\nE\nF\n[G]\n```\n\n#### `--awk \u003cpattern\u003e`\n\nThis is an alias of `-e 'awk \"\u003cpattern\u003e{print NR}\"`.\n\n```bash\n$ cat AtoG.txt | teip --awk '/B/,/E/'\nA\n[B]\n[C]\n[D]\n[E]\nF\nG\n```\n\n```bash\n$ cat AtoG.txt | teip --awk 'NR%3==0'\nA\nB\n[C]\nD\nE\n[F]\nG\n```\n\n\n## Environment variables\n\n`teip` refers to the following environment variables.\nAdd a statement to your default shell's startup file (i.e `.bashrc`, `.zshrc`) to change them as you like.\n\n### `TEIP_HIGHLIGHT`\n\n**DEFAULT VALUE:** `\\x1b[36m[\\x1b[0m\\x1b[01;31m{}\\x1b[0m\\x1b[36m]\\x1b[0m`\n\nThe default format for highlighting holes.\nIt must include at least one `{}` as a placeholder.\n\nExample:\n\n```bash\n$ export TEIP_HIGHLIGHT=\"\u003c\u003c\u003c{}\u003e\u003e\u003e\"\n$ echo ABAB | teip -og A\n\u003c\u003c\u003cA\u003e\u003e\u003eB\u003c\u003c\u003cA\u003e\u003e\u003eB\n\n$ export TEIP_HIGHLIGHT=$'\\x1b[01;31m{}\\x1b[0m'\n$ echo ABAB | teip -og A\nABAB  ### Same color as grep\n```\n\n[ANSI Escape Sequences](https://gist.github.com/fnky/458719343aabd01cfb17a3a4f7296797) and [ANSI-C Quoting](https://www.gnu.org/software/bash/manual/html_node/ANSI_002dC-Quoting.html) are helpful for customizing this value.\n\n### `TEIP_GREP_PATH`\n\n**DEFAULT VALUE:** `grep`\n\nThe path to the `grep` command used by the `-A`, `-B`, and `-C` options.\nFor example, if you want to use `ggrep` instead of `grep`, set this variable to `ggrep`.\n\n```bash\n$ export TEIP_GREP_PATH=/opt/homebrew/bin/ggrep\n$ echo -e 'AAA\\nBBB\\nCCC\\nDDD\\nEEE\\nFFF' | teip -g CCC -A 2\nAAA\nBBB\n[CCC]\n[DDD]\n[EEE]\nFFF\n```\n\n### `TEIP_SED_PATH`\n\n**DEFAULT VALUE:** `sed`\n\nThe path to the `sed` command used by the `--sed` option.\nFor example, if you want to use `gsed` instead of `sed`, set this variable to `gsed`.\n\n### `TEIP_AWK_PATH`\n\n**DEFAULT VALUE:** `awk`\n\nThe path to the `awk` command used by the `--awk` option.\nFor example, if you want to use `gawk` instead of `awk`, set this variable to `gawk`.\n\n## Background\n\n### Why make this?\n\nSee this [post](https://dev.to/greymd/teip-masking-tape-for-shell-is-what-we-needed-5e05).\n\n### Why \"teip\"?\n\n* [tee](https://en.wikipedia.org/wiki/Tee_%28command%29) + in-place.\n* And it sounds similar to Masking-\"tape\".\n\n## License\n\n### Modules imported/referenced from other repositories\n\nThank you so much for these helpful modules!\n\n* ./src/list/ranges.rs\n  - One of the modules used in the `cut` command is from [uutils/coreutils](https://github.com/uutils/coreutils)\n  - The original source code is distributed under the MIT license\n  - The license file is in the same directory\n\n* ./src/csv/parser.rs\n  - Many parts of the source code are referenced from [BurntSushi/rust-csv](https://github.com/BurntSushi/rust-csv).\n  - The original source code is dual-licensed under the MIT and Unlicense\n\n### Source code\n\nTeip is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).\n\n### Logo\n\n\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\"\u003e\u003cimg alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc/4.0/88x31.png\" /\u003e\u003c/a\u003e\u003cbr /\u003eThe logo of teip is licensed under a \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\"\u003eCreative Commons Attribution-NonCommercial 4.0 International License\u003c/a\u003e.\n","funding_links":["https://github.com/sponsors/greymd"],"categories":["Rust","Xargs-like Tools","cli","Other","\u003ca name=\"text-search-replace\"\u003e\u003c/a\u003eText search and replace (alternatives to sed)"],"sub_categories":["Open USP Tsukubai"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgreymd%2Fteip","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgreymd%2Fteip","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgreymd%2Fteip/lists"}