{"id":13567577,"url":"https://github.com/codenameyau/sed-awk-cheatsheet","last_synced_at":"2025-04-13T13:21:52.828Z","repository":{"id":41546292,"uuid":"55846601","full_name":"codenameyau/sed-awk-cheatsheet","owner":"codenameyau","description":"Things you can do with sed and awk","archived":false,"fork":false,"pushed_at":"2019-03-12T03:07:44.000Z","size":33,"stargazers_count":150,"open_issues_count":1,"forks_count":42,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-27T04:22:38.820Z","etag":null,"topics":["awk","bash","linux","script","sed","shell"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/codenameyau.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-04-09T13:46:44.000Z","updated_at":"2025-01-11T18:29:22.000Z","dependencies_parsed_at":"2022-07-18T22:19:05.514Z","dependency_job_id":null,"html_url":"https://github.com/codenameyau/sed-awk-cheatsheet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenameyau%2Fsed-awk-cheatsheet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenameyau%2Fsed-awk-cheatsheet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenameyau%2Fsed-awk-cheatsheet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codenameyau%2Fsed-awk-cheatsheet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/codenameyau","download_url":"https://codeload.github.com/codenameyau/sed-awk-cheatsheet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248718104,"owners_count":21150507,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["awk","bash","linux","script","sed","shell"],"created_at":"2024-08-01T13:02:35.238Z","updated_at":"2025-04-13T13:21:52.804Z","avatar_url":"https://github.com/codenameyau.png","language":"Shell","funding_links":[],"categories":["Shell"],"sub_categories":[],"readme":"# sed-awk\n\n## TLDR\n\n```bash\n$ tldr sed\n  sed\n  Run replacements based on regular expressions.\n\n  - Replace the first occurrence of a string in a file, and print the result:\n    sed 's/find/replace/' filename\n\n  - Replace only on lines matching the line pattern:\n    sed '/line_pattern/s/find/replace/'\n\n  - Replace all occurrences of a string in a file, overwriting the file (i.e. in-place):\n    sed -i 's/find/replace/g' filename\n\n  - Replace all occurrences of an extended regular expression in a file:\n    sed -r 's/regex/replace/g' filename\n\n  - Apply multiple find-replace expressions to a file:\n    sed -e 's/find/replace/' -e 's/find/replace/' filename\n```\n\n```bash\n$ tldr awk\n\n  awk\n  A versatile programming language for working on files.\n\n  - Print the fifth column in a space separated file:\n    awk '{print $5}' filename\n\n  - Print the second column of the lines containing \"something\" in a space separated file:\n    awk '/something/ {print $2}' filename\n\n  - Print the third column in a comma separated file:\n    awk -F ',' '{print $3}' filename\n\n  - Sum the values in the first column and print the total:\n    awk '{s+=$1} END {print s}' filename\n\n  - Sum the values in the first column and pretty-print the values and then the total:\n    awk '{s+=$1; print $1} END {print \"--------\"; print s}' filename\n```\n\n## Useful Commands\n\n```bash\n# List out the second column in the table.\ncat text/table.txt | sed 1d | awk '{ print $2 }'\n\n# Sum the columns in the table.\ncat text/table.txt | sed 1d | awk '{ sum += $2 } END { print sum }'\n\n# Kills all processes by name.\nps aux | grep chrome | awk '{ print $2 }' | kill\npkill chrome\n\n# Deletes trailing whitespace.\nsed 's/\\s\\+$//g' filename\n\n# Deletes all blank lines from file.\nsed '/^$/d' filename\n\n# Insert 'use strict' to the top of every js file.\nsed \"1i 'use strict';\" *.js\n\n# Append a new line at the end of every file.\nsed '1a \\n' *\n\n# Generate random numbers and then sort.\nfor i in {1..20}; do echo $(($RANDOM * 777 * $i)); done | sort -n\n\n# Commatize numbers.\nsed -r ':loop; s/(.*[0-9])([0-9]{3})/\\1,\\2/; t loop' text/numbers.txt\n```\n\n## Tutorial\nFollow the tutorials here:\n- http://www.thegeekstuff.com/tag/sed-tips-and-tricks/\n- http://www.grymoire.com/Unix/Sed.html\n- http://www.grymoire.com/Unix/Awk.html\n\n```bash\n# Unzip data.\nunzip data.zip\n\n# Zip data.\nzip -r data.zip data/\n\n# Preview the files.\nhead data/names.csv \u0026\u0026 tail data/names.csv\n\n# Preview csv columns.\nsed -n 1p data/colleges.csv | tr ',' '\\n'\n\n# Count the number of lines.\nwc -l data/*\n```\n\n### Sed Print\n\n```sh\n# Print contents of a file.\nsed -n '/fox/p' text/*\nsed -n '/Sysadmin/p' text/geek.txt\n\n# Print lines starting with `3` and skipping by `2`.\nsed -n '3~2p' text/geek.txt\n\n# Print the last line.\nsed -n '$p' text/geek.txt\n\n# Prints the lines matching the between the two patterns.\nsed -n '/Hardware/,/Website/p' text/geek.txt\n```\n\n### Sed Print Line Number\n\n```sh\n# Prints the line number for all lines in the file.\nsed -n '=' filename\n\n# Prints the line number that matches the pattern.\nsed -n '/Linux/=' filename\n\n# Prints the line number in range of two patterns (inclusive).\nsed -n '/Linux/,/Hardware/=' filename\n\n# Prints the total number of lines.\nsed -n '$=' filename\n```\n\n### Sed Delete\nThe `d` command performs a deletion.\n\n```sh\n# Deletes the 3rd line from beginning of file.\nsed '3d' text/geek.txt\n\n# Delete every lines starting from 3 and skipping by 2.\nsed '3~2d' text/geek.txt\n\n# Delete lines from 3 to 5.\nsed '3,5d' text/geek.txt\n\n# Delete the last line.\nsed '$d' text/geek.txt\n\n# Delete lines matching the pattern.\nsed '/Sysadmin/d' text/geek.txt\n```\n\n### Sed Substitute\nThe `s` command performs a substitution.\n\n```sh\n# Simple substituion for the first result.\nsed 's/Linux/Unix/' text/geek.txt\n\n# Simple substituion for global instances.\nsed 's/Linux/Unix/g' text/geek.txt\n\n# Replace nth instance.\nsed 's/Linux/Unix/2' text/geek.txt\n\n# Write matched lines to output.\nsed -n 's/Linux/Unix/gp' text/geek.txt \u003e text/geek-sub.txt\n\n# Use regex group for capturing additional patterns (up to 9).\nsed 's/\\(Linux\\).\\+/\\1/g' text/geek.txt\nsed -r 's/(Linux).+/\\1/g' text/geek.txt\n\n# Remove the last word.\nsed -r 's/\\d$//g' text/geek.txt\n\n# Remove all letters.\nsed -r 's/[a-zA-Z]//g' text/geek.txt\n\n# Remove html tags (WIP).\nsed -r 's|(\u003c/?[a-z]+\u003e)||g' text/html.txt\n\n# Commatize any number.\nsed ':a;s/\\B[0-9]\\{3\\}\\\u003e/,\u0026/;ta' text/numbers.txt\nsed -r ':loop; s/\\B[0-9]{3}\\\u003e/,\u0026/; t loop' text/numbers.txt\n```\n\n### Sed Transform\nThe `y` command performs a transformation.\n\n```sh\n# Converts all lowercase chars to uppercase.\nsed 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/' text/geek.txt\n\n# Converts all uppercase chars to lowercase.\nsed 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/' text/geek.txt\n\n# Perform a two character shift.\nsed 'y/abcdefghijklmnopqrstuvwxyz/cdefghijklmnopqrstuvwxyzab/' text/geek.txt\n```\n\n### Sed Multiple Commands\nThe `-e` flag allows for multiple commands.\n\n```sh\nsed -r -e 's/etc\\.*//g' -e 's/(\\s+)(\\))/\\2/g' text/geek.txt\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodenameyau%2Fsed-awk-cheatsheet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodenameyau%2Fsed-awk-cheatsheet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodenameyau%2Fsed-awk-cheatsheet/lists"}