{"id":13575925,"url":"https://github.com/soveran/map","last_synced_at":"2025-12-30T00:15:48.566Z","repository":{"id":66254685,"uuid":"124780680","full_name":"soveran/map","owner":"soveran","description":"Map lines from stdin to commands","archived":false,"fork":false,"pushed_at":"2020-06-30T17:15:53.000Z","size":9,"stargazers_count":224,"open_issues_count":2,"forks_count":5,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-22T15:50:01.160Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/soveran.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-03-11T17:24:52.000Z","updated_at":"2025-03-10T19:04:09.000Z","dependencies_parsed_at":null,"dependency_job_id":"218c510a-ac0f-414f-bf00-e79e1cb44f58","html_url":"https://github.com/soveran/map","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soveran%2Fmap","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soveran%2Fmap/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soveran%2Fmap/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/soveran%2Fmap/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/soveran","download_url":"https://codeload.github.com/soveran/map/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247293989,"owners_count":20915329,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:01:05.475Z","updated_at":"2025-12-30T00:15:48.536Z","avatar_url":"https://github.com/soveran.png","language":"C","readme":"# map\n\nMap lines from stdin to commands.\n\nDescription\n-----------\n\nMap lets you process each line from stdin with a command of your\nchoice. For example:\n\n```console\n$ ls\nLICENSE   README.md makefile  map.1   map.c\n```\n\n```console\n$ ls | map f 'echo $f $f'\nLICENSE LICENSE\nREADME.md README.md\nmakefile makefile\nmap.1 map.1\nmap.c map.c\n```\n\nNote that the command must be wrapped in single quotes to prevent\nthe variable from being expanded by the shell.\n\nInstallation\n------------\n\nInstall map into `/usr/local/bin` with the following command:\n\n```console\n$ make install\n```\n\nYou can use `make PREFIX=/some/other/directory install` if you wish\nto use a different destination. If you want to remove map from\nyour system, use `make uninstall`.\n\nMotivation\n----------\n\nThere are many ways to accomplish what you can do with `map`,\nincluding `find`, `xargs`, `awk`, and shell for-loops. The approach\ntaken by `map` is extremely pragmatic and allows me to express\nconcisely what I want. Given the fact that it's designed as a filter,\nit can operate on any kind of list, not only lists of files.\n\nThe problem that prompted me to think about `map` was the following:\ngiven a list of files, I wanted to execute two commands on each.\nHere's how you can do it with different tools:\n\nWith `map`:\n\n```sh\nls *.c | map f 'foo $f; bar $f'\n```\n\nWith `xargs`:\n\n```sh\nls *.c | xargs -I % sh -c 'foo %; bar %;'\n```\n\nWith `awk`:\n\n```sh\nls *.c | awk '{ system(\"foo \"$0\"; bar \"$0) }'\n```\n\nWith `find`:\n\n```sh\nfind . -name \\*.c -maxdepth 1 -exec foo {} \\; -exec bar {} \\;\n```\n\nWith a `bash` for-loop:\n\n```bash\nfor f in $(ls *.c)\ndo\n  foo $f\n  bar $f\ndone\n```\n\nWith a `csh` for-loop:\n\n```sh\nforeach f (*.c)\n  foo $f\n  bar $f\nend\n```\n\nMap's modest claim is that it improves on the ergonomics of existing\ntools. It's not only that sometimes it allows you to type less, but\nalso the conceptual model needed to operate it is simpler.\n\nLet's consider these tasks:\n\n1. Execute a command `foo` on each C file:\n\n```sh\nls *.c | map f 'foo $f'\n```\n\n```sh\nls *.c | xargs foo\n```\n\n```sh\nfind . -name *.c -maxdepth 1 -exec foo {} \\;\n```\n\n2. Execute commands `foo` and `bar` on each C file:\n\n```sh\nls *.c | map f 'foo $f; bar $f'\n```\n\n```sh\nls *.c | xargs -I % sh -c 'foo %; bar %;'\n```\n\n```sh\nfind . -name *.c -maxdepth 1 -exec foo {} \\; -exec bar {} \\;\n```\n\n3. Download files from a list of URLs in a file:\n\n```sh\ncat urls | map u 'curl -O $u'\n```\n\n```sh\ncat urls | xargs -n 1 curl -O\n```\n\n4. Sleep three times for one second and say \"done\" after each elapsed\n   second:\n\n```sh\nprintf \"1\\n1\\n1\\n\" | map t 'sleep $t \u0026\u0026 say done'\n```\n\n```sh\nprintf \"1\\n1\\n1\\n\" | xargs -n 1 -I % sh -c 'sleep % \u0026\u0026 say done'\n```\n\n5. Same as #4, but run the commands in parallel:\n\n```sh\nprintf \"1\\n1\\n1\\n\" | map t 'sleep $t \u0026\u0026 say done \u0026'\n```\n\n```sh\nprintf \"1\\n1\\n1\\n\" | xargs -n 1 -P 3 -I % sh -c 'sleep % \u0026\u0026 say done'\n```\n\nThe last three examples are not possible with `find`, because it\nonly operates on file hierarchies.\n\nWhen using `map`, the commands don't vary much because the second\nargument is a template for a well known syntax. On the other hand,\nthere's more variation in the invocations to `xargs` and `find`,\nwhich means you may need to remember those command line options if\nyou want an idiomatic solution.\n\nAs with anything in life, familiarity helps and if you use a tool\nin a certain way over and over it will seem simple to operate, but\nwe can still analyze the conceptual models and determine how much\ninformation is needed in each case. I would say the advantage of\n`map` is that it requires less knowledge.\n \nOf course `xargs` and `find` are much larger tools, with a bigger\nfeature set. Map has around 20 lines of code. For comparison, here's\nthe [source code of GNU xargs][xargs]. No doubt `xargs` will offer\na lot more features, but so far with `map` I've completely stopped\nusing `xargs` and for-loops. Another way to think about `map` vs\n`xargs`: if `map` had been specified in POSIX and `xargs` was just\nreleased, I'm not sure I would install it unless `map` proved to\nbe unfit for a given use case.\n\n[xargs]: https://fossies.org/dox/findutils-4.7.0/xargs_8c_source.html\n\nSomething important to keep in mind is that `map` works on lines,\nnot on files. My reasoning was that in the context of a POSIX\nenvironment, a map function can be expected to filter lines from\nstdin. In that regard, it is very generic because a line can represent\nanything.\n\nKnown issues\n------------\n\nIn the examples above I frequently use the output of `ls`. In my\nprojects, filenames don't contain spaces, newlines, or other special\ncharacters because I find them inconvenient, even though they are\nvalid. As `map` works on lines, filenames that contain newlines\nshould be handled separately. For files that contain whitespace,\nthe common solution is to wrap the variable in double quotes, so\ninstead of `$f` you would use `\"$f\"`.\n\nContributing\n------------\n\nIf you find a bug, please create an issue detailing the ways to\nreproduce it. If you have a suggestion, create an issue detailing\nthe use case.\n","funding_links":[],"categories":["C","Other"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoveran%2Fmap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsoveran%2Fmap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoveran%2Fmap/lists"}