{"id":13471940,"url":"https://github.com/benhoyt/goawk","last_synced_at":"2025-05-12T15:24:03.875Z","repository":{"id":38360159,"uuid":"136558084","full_name":"benhoyt/goawk","owner":"benhoyt","description":"A POSIX-compliant AWK interpreter written in Go, with CSV support","archived":false,"fork":false,"pushed_at":"2024-11-24T19:29:52.000Z","size":3267,"stargazers_count":1976,"open_issues_count":10,"forks_count":84,"subscribers_count":40,"default_branch":"master","last_synced_at":"2025-05-12T15:23:50.345Z","etag":null,"topics":["awk","csv","go","interpreter","parser"],"latest_commit_sha":null,"homepage":"https://benhoyt.com/writings/goawk/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/benhoyt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"benhoyt"}},"created_at":"2018-06-08T02:53:23.000Z","updated_at":"2025-05-03T13:29:29.000Z","dependencies_parsed_at":"2024-02-28T03:28:24.487Z","dependency_job_id":"bf02e439-0ba2-4cc8-b4da-b7ceb7073f75","html_url":"https://github.com/benhoyt/goawk","commit_stats":{"total_commits":610,"total_committers":16,"mean_commits":38.125,"dds":0.05245901639344264,"last_synced_commit":"4b6c79a6da06fc8e53eeb25c96f3b0e89d53c818"},"previous_names":[],"tags_count":48,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benhoyt%2Fgoawk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benhoyt%2Fgoawk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benhoyt%2Fgoawk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benhoyt%2Fgoawk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/benhoyt","download_url":"https://codeload.github.com/benhoyt/goawk/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253764115,"owners_count":21960516,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["awk","csv","go","interpreter","parser"],"created_at":"2024-07-31T16:00:50.537Z","updated_at":"2025-05-12T15:24:03.849Z","avatar_url":"https://github.com/benhoyt.png","language":"Go","readme":"\n# GoAWK: an AWK interpreter with CSV support\n\n[![Documentation](https://pkg.go.dev/badge/github.com/benhoyt/goawk)](https://pkg.go.dev/github.com/benhoyt/goawk)\n[![GitHub Actions Build](https://github.com/benhoyt/goawk/actions/workflows/tests.yml/badge.svg)](https://github.com/benhoyt/goawk/actions/workflows/tests.yml)\n\nAWK is a fascinating text-processing language, and somehow after reading the delightfully-terse [*The AWK Programming Language*](https://ia802309.us.archive.org/25/items/pdfy-MgN0H1joIoDVoIC7/The_AWK_Programming_Language.pdf) I was inspired to write an interpreter for it in Go. So here it is, feature-complete and tested against \"the one true AWK\" and GNU AWK test suites.\n\nGoAWK is a POSIX-compatible version of AWK, and additionally has a CSV mode for reading and writing CSV and TSV files. This feature was sponsored by the [library of the University of Antwerp](https://www.uantwerpen.be/en/library/). Read the [CSV documentation](https://github.com/benhoyt/goawk/blob/master/docs/csv.md).\n\nYou can also read one of the articles I've written about GoAWK:\n\n* The original article about [how GoAWK works and performs](https://benhoyt.com/writings/goawk/)\n* How I converted the tree-walking interpreter to a [bytecode compiler and virtual machine](https://benhoyt.com/writings/goawk-compiler-vm/)\n* A description of why and how I added [CSV support](https://benhoyt.com/writings/goawk-csv/)\n\n\n## Basic usage\n\nTo use the command-line version, simply use `go install` to install it, and then run it using `goawk` (assuming `~/go/bin` is in your `PATH`):\n\n```shell\n$ go install github.com/benhoyt/goawk@latest\n\n$ goawk 'BEGIN { print \"foo\", 42 }'\nfoo 42\n\n$ echo 1 2 3 | goawk '{ print $1 + $3 }'\n4\n\n# Or use GoAWK's CSV and @\"named-field\" support:\n$ echo -e 'name,amount\\nBob,17.50\\nJill,20\\n\"Boba Fett\",100.00' | \\\n  goawk -i csv -H '{ total += @\"amount\" } END { print total }'\n137.5\n```\n\nTo use it in your Go programs, you can call `interp.Exec()` directly for simple needs:\n\n```go\ninput := strings.NewReader(\"foo bar\\n\\nbaz buz\")\nerr := interp.Exec(\"$0 { print $1 }\", \" \", input, nil)\nif err != nil {\n    fmt.Println(err)\n    return\n}\n// Output:\n// foo\n// baz\n```\n\nOr you can use the `parser` module and then `interp.ExecProgram()` to control execution, set variables, and so on:\n\n```go\nsrc := \"{ print NR, tolower($0) }\"\ninput := \"A\\naB\\nAbC\"\n\nprog, err := parser.ParseProgram([]byte(src), nil)\nif err != nil {\n    fmt.Println(err)\n    return\n}\nconfig := \u0026interp.Config{\n    Stdin: strings.NewReader(input),\n    Vars:  []string{\"OFS\", \":\"},\n}\n_, err = interp.ExecProgram(prog, config)\nif err != nil {\n    fmt.Println(err)\n    return\n}\n// Output:\n// 1:a\n// 2:ab\n// 3:abc\n```\n\nIf you need to repeat execution of the same program on different inputs, you can call [`interp.New`](https://pkg.go.dev/github.com/benhoyt/goawk/interp#New) once, and then call the returned object's `Execute` method as many times as you need.\n\nRead the [package documentation](https://pkg.go.dev/github.com/benhoyt/goawk) for more details.\n\n\n## Differences from AWK\n\nThe intention is for GoAWK to conform to `awk`'s behavior and to the [POSIX AWK spec](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html), but this section describes some areas where it's different.\n\nAdditional features GoAWK has over AWK:\n\n* It has proper support for CSV and TSV files ([read the documentation](https://github.com/benhoyt/goawk/blob/master/docs/csv.md)).\n* It's the only AWK implementation we know with a code coverage feature ([read the documentation](https://github.com/benhoyt/goawk/blob/master/docs/cover.md)).\n* It supports negative field indexes to access fields from the right, for example, `$-1` refers to the last field.\n* It's embeddable in your Go programs! You can even call custom Go functions from your AWK scripts.\n* Most AWK scripts are faster than `awk` and on a par with `gawk`, though usually slower than `mawk`. (See [recent benchmarks](https://benhoyt.com/writings/goawk-compiler-vm/#virtual-machine-results).)\n* The parser supports `'single-quoted strings'` in addition to `\"double-quoted strings\"`, primarily to make Windows one-liners easier when using the `cmd.exe` shell (which uses `\"` as the quote character).\n\nThings AWK has over GoAWK:\n\n* Scripts that use regular expressions are slower than other implementations (unfortunately Go's `regexp` package is relatively slow).\n* AWK is written by Alfred Aho, Peter Weinberger, and Brian Kernighan.\n\n\n## Stability\n\nThis project has a good suite of tests, which include my own intepreter tests, the original AWK test suite, and the relevant tests from the Gawk test suite. I've used it a bunch personally, and it's used in the [Benthos](https://github.com/benthosdev/benthos) stream processor as well as by the software team at the library of the University of Antwerp. However, to `err == human`, so please use GoAWK at your own risk. I intend not to change the Go API in a breaking way in any v1.x.y version.\n\n\n## AWKGo\n\nThe GoAWK repository also includes the creatively-named AWKGo, an AWK-to-Go compiler. This is experimental and is not subject to the stability requirements of GoAWK itself. You can [read more about AWKGo](https://benhoyt.com/writings/awkgo/) or browse the code on the [`awkgo` branch](https://github.com/benhoyt/goawk/tree/awkgo/awkgo).\n\n\n## License\n\nGoAWK is licensed under an open source [MIT license](https://github.com/benhoyt/goawk/blob/master/LICENSE.txt).\n\n\n## The end\n\nHave fun, and please [contact me](https://benhoyt.com/) if you're using GoAWK or have any feedback!\n","funding_links":["https://github.com/sponsors/benhoyt"],"categories":["Go","Members","Shell","Repositories","Books"],"sub_categories":["PCAP"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenhoyt%2Fgoawk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbenhoyt%2Fgoawk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenhoyt%2Fgoawk/lists"}