{"id":18940227,"url":"https://github.com/juliaio/argtools.jl","last_synced_at":"2025-10-29T16:39:52.905Z","repository":{"id":46354102,"uuid":"268535378","full_name":"JuliaIO/ArgTools.jl","owner":"JuliaIO","description":"Tools for writing functions that handle many kinds of IO arguments","archived":false,"fork":false,"pushed_at":"2024-04-11T19:55:55.000Z","size":75,"stargazers_count":14,"open_issues_count":1,"forks_count":9,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-10-09T06:58:15.698Z","etag":null,"topics":["julia"],"latest_commit_sha":null,"homepage":"","language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JuliaIO.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-06-01T13:49:58.000Z","updated_at":"2024-09-10T20:39:17.000Z","dependencies_parsed_at":"2024-02-05T20:34:04.439Z","dependency_job_id":"7def287a-8e1a-417f-a92b-883a887c1c51","html_url":"https://github.com/JuliaIO/ArgTools.jl","commit_stats":{"total_commits":40,"total_committers":4,"mean_commits":10.0,"dds":"0.17500000000000004","last_synced_commit":"08b11b2707593d4d7f92e5f1b9dba7668285ff82"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaIO%2FArgTools.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaIO%2FArgTools.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaIO%2FArgTools.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaIO%2FArgTools.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JuliaIO","download_url":"https://codeload.github.com/JuliaIO/ArgTools.jl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223682529,"owners_count":17185239,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["julia"],"created_at":"2024-11-08T12:21:22.327Z","updated_at":"2025-10-29T16:39:47.880Z","avatar_url":"https://github.com/JuliaIO.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ArgTools\n\n[![Build Status](https://github.com/JuliaIO/ArgTools.jl/actions/workflows/ci.yml/badge.svg)](https://github.com/JuliaIO/ArgTools.jl/actions/workflows/ci.yml)\n[![Codecov](https://codecov.io/gh/JuliaIO/ArgTools.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/JuliaIO/ArgTools.jl)\n\n`ArgTools` provides tools for creating consistent, flexible APIs that work with\nvarious kinds of function arguments. In the current version, it helps deal with\narguments that are, at their core, IO handles, but which you'd like to allow the\nuser to specify directly as file names, commands, pipelines, or, of course, as\nraw IO handles. For write arguments, it's also possible to use `nothing` and\nwrite to a temporary file whose path is returned.\n\n## API\n\nThere are two parts to the `ArgTools` API:\n\n1. Functions and types for helping define flexible function APIs.\n2. Functions for helping to test APIs defined with above.\n\nWhile it's great to be able to define a flexible API, if you're not sure that\nit works the way it's supposed to, what's the benefit. Since it's can be quite\nverbose to test such a combinatorial explosion of methods, `ArgTools` also\nprovides tools to help testing all the ways your tools can be called to make\nsure everything is working as intended.\n\n### Argument Handling\n\nThe API for helping defining flexible function signatures consists of two types\nand four helper functions: `ArgRead` and `ArgWrite`; `arg_read`, `arg_write`,\n`arg_isdir` and `arg_mkdir`.\n\n\u003c!-- BEGIN: copied from inline doc strings --\u003e\n\n#### ArgRead\n\n```jl\nArgRead = Union{AbstractString, AbstractCmd, IO}\n```\nThe `ArgRead` types is a union of the types that the `arg_read` function knows\nhow to convert into readable IO handles. See [`arg_read`](@ref) for details.\n\n#### ArgWrite\n\n```jl\nArgWrite = Union{AbstractString, AbstractCmd, IO}\n```\nThe `ArgWrite` types is a union of the types that the `arg_write` function knows\nhow to convert into writeable IO handles, except for `Nothing` which `arg_write`\nhandles by generating a temporary file. See [`arg_write`](@ref) for details.\n\n#### arg_read\n\n```jl\narg_read(f::Function, arg::ArgRead) -\u003e f(arg_io)\n```\nThe `arg_read` function accepts an argument `arg` that can be any of these:\n\n- `AbstractString`: a file path to be opened for reading\n- `AbstractCmd`: a command to be run, reading from its standard output\n- `IO`: an open IO handle to be read from\n\nWhether the body returns normally or throws an error, a path which is opened\nwill be closed before returning from `arg_read` and an `IO` handle will be\nflushed but not closed before returning from `arg_read`.\n\n#### arg_write\n\n```jl\narg_write(f::Function, arg::ArgWrite) -\u003e arg\narg_write(f::Function, arg::Nothing) -\u003e tempname()\n```\nThe `arg_write` function accepts an argument `arg` that can be any of these:\n\n- `AbstractString`: a file path to be opened for writing\n- `AbstractCmd`: a command to be run, writing to its standard input\n- `IO`: an open IO handle to be written to\n- `Nothing`: a temporary path should be written to\n\nIf the body returns normally, a path that is opened will be closed upon\ncompletion; an IO handle argument is left open but flushed before return. If the\nargument is `nothing` then a temporary path is opened for writing and closed\nopen completion and the path is returned from `arg_write`. In all other cases,\n`arg` itself is returned. This is a useful pattern since you can consistently\nreturn whatever was written, whether an argument was passed or not.\n\nIf there is an error during the evaluation of the body, a path that is opened by\n`arg_write` for writing will be deleted, whether it's passed in as a string or a\ntemporary path generated when `arg` is `nothing`.\n\n#### arg_isdir\n\n```jl\narg_isdir(f::Function, arg::AbstractString) -\u003e f(arg)\n```\nThe `arg_isdir` function takes `arg` which must be the path to an existing\ndirectory (an error is raised otherwise) and passes that path to `f` finally\nreturning the result of `f(arg)`. This is definitely the least useful tool\noffered by `ArgTools` and mostly exists for symmetry with `arg_mkdir` and to\ngive consistent error messages.\n\n#### arg_mkdir\n\n```jl\narg_mkdir(f::Function, arg::AbstractString) -\u003e arg\narg_mkdir(f::Function, arg::Nothing) -\u003e mktempdir()\n```\nThe `arg_mkdir` function takes `arg` which must either be one of:\n\n- a path to an already existing empty directory,\n- a non-existent path which can be created as a directory, or\n- `nothing` in which case a temporary directory is created.\n\nIn all cases the path to the directory is returned. If an error occurs during\n`f(arg)`, the directory is returned to its original state: if it already existed\nbut was empty, it will be emptied; if it did not exist it will be deleted.\n\n\u003c!-- END: copied from inline doc strings --\u003e\n\n### Function Testing\n\nUsing `ArgTools` is easy; thoroughly testing flexible functions defined using\n`ArgTools` is a bit trickier, but the package includes testing tools that help.\nThe API for testing functions defined with the argument handling API consists\nof two functions and a macro: `arg_readers`, `arg_writers` and `@arg_test`.\n\n\u003c!-- BEGIN: copied from inline doc strings --\u003e\n\n#### arg_readers\n\n```jl\narg_readers(arg :: AbstractString, [ type = ArgRead ]) do arg::Function\n    ## pre-test setup ##\n    @arg_test arg begin\n        arg :: ArgRead\n        ## test using `arg` ##\n    end\n    ## post-test cleanup ##\nend\n```\n\nThe `arg_readers` function takes a path to be read and a single-argument do\nblock, which is invoked once for each test reader type that `arg_read` can\nhandle. If the optional `type` argument is given then the do block is only\ninvoked for readers that produce arguments of that type.\n\nThe `arg` passed to the do block is not the argument value itself, because some\nof test argument types need to be initialized and finalized for each test case.\nConsider an open file handle argument: once you've used it for one test, you\ncan't use it again; you need to close it and open the file again for the next\ntest. This function `arg` can be converted into an `ArgRead` instance using\n`@arg_test arg begin ... end`.\n\n#### arg_writers\n\n```jl\narg_writers([ type = ArgWrite ]) do path::String, arg::Function\n    ## pre-test setup ##\n    @arg_test arg begin\n        arg :: ArgWrite\n        ## test using `arg` ##\n    end\n    ## post-test cleanup ##\nend\n```\n\nThe `arg_writers` function takes a do block, which is invoked once for each test\nwriter type that `arg_write` can handle with a temporary (non-existent) `path`\nand `arg` which can be converted into various writable argument types which\nwrite to `path`. If the optional `type` argument is given then the do block is\nonly invoked for writers that produce arguments of that type.\n\nThe `arg` passed to the do block is not the argument value itself, because some\nof test argument types need to be initialized and finalized for each test case.\nConsider an open file handle argument: once you've used it for one test, you\ncan't use it again; you need to close it and open the file again for the next\ntest. This function `arg` can be converted into an `ArgWrite` instance using\n`@arg_test arg begin ... end`.\n\nThere is also an `arg_writers` method that takes a path name like `arg_readers`:\n\n```jl\narg_writers(path::AbstractString, [ type = ArgWrite ]) do arg::Function\n    ## pre-test setup ##\n    @arg_test arg begin\n        arg :: ArgWrite\n        ## test using `arg` ##\n    end\n    ## post-test cleanup ##\nend\n```\n\nThis method is useful if you need to specify `path` instead of using path name\ngenerated by `tempname()`. Since `path` is passed from outside of `arg_writers`,\nthe path is not an argument to the do block in this form.\n\n#### @arg_test\n\n```jl\n@arg_test arg1 arg2 ... body\n```\n\nThe `@arg_test` macro is used to convert `arg` functions provided by\n`arg_readers` and `arg_writers` into actual argument values. When you write\n`@arg_test arg body` it is equivalent to `arg(arg -\u003e body)`.\n\n\u003c!-- END: copied from inline doc strings --\u003e\n\n## Examples\n\nThe examples, like the API, are split into two parts:\n\n1. An example of defining a function with a flexible API using the main API;\n2. Examples of how to thoroughly test that function using the test utilities.\n\n### Usage Example\n\nThe best explanation may be an example, which is also used for testing:\n\n```jl\nusing ArgTools\n\nfunction send_data(src::ArgRead, dst::Union{ArgWrite, Nothing} = nothing)\n    arg_read(src) do src_io\n        arg_write(dst) do dst_io\n            buffer = Vector{UInt8}(undef, 2*1024*1024)\n            while !eof(src_io)\n                n = readbytes!(src_io, buffer)\n                write(dst_io, view(buffer, 1:n))\n            end\n        end\n    end\nend\n```\n\nThis defines the `send_data` function which reads data from a source and writes\nit to a destination, specified by the `src` and `dst` arguments, respectively.\nThanks to `ArgTools`, this relatively simple definition acts as a swiss-army\nknife for sending data from a source to a destination. Here are some examples:\n\n```jl\njulia\u003e cd(mktempdir())\n\njulia\u003e write(\"hello.txt\", \"Hello, world.\\n\")\n14\n\njulia\u003e run(`cat hello.txt`);\nHello, world.\n\njulia\u003e send_data(\"hello.txt\", \"hello_copy.txt\")\n\"hello_copy.txt\"\n\njulia\u003e run(`cat $ans`);\nHello, world.\n\njulia\u003e rm(\"hello_copy.txt\")\n\njulia\u003e send_data(\"hello.txt\", stdout);\nHello, world.\n\njulia\u003e send_data(\"hello.txt\", pipeline(`gzip -9`, \"hello.gz\"));\n\njulia\u003e run(`gzcat hello.gz`);\nHello, world.\n\njulia\u003e hello_copy = send_data(`gzcat hello.gz`)\n\"/var/folders/4g/b8p546px3nd550b3k288mhp80000gp/T/jl_cguepi\"\n\njulia\u003e run(`cat $hello_copy`);\nHello, world.\n```\n\nTo understand the definition of `send_data`, let's work from the inside out:\n\n* The main body of the function operates on the `src_io` and `dst_io` IO\n  handles, using a buffer to read data from the former to the latter in 2MiB\n  blocks.\n\n* The calls to `arg_read` and `arg_write` transform the `src` and `dst`\n  arguments from various types to `src_io` and `dst_io` IO handles. This allows\n  the inner body to handle the core case of dealing with IO handles, without\n  having to worry about the various possible incoming argument types. See the API\n  section below for more details about how `arg_read` and `arg_write` work on\n  different types.\n\n* The arguments to `send_data` are `src::ArgRead` and `dst::ArgWrite` where\n  `dst` is optional and defaults to `nothing` if not given. The `ArgRead` type is\n  a union including all the types that `arg_read` knows how to handle. Similarly,\n  the `ArgWrite` type is a union including the types that `arg_write` knows how to\n  handle, except for `nothing` which must be explicitly opted into, for which\n  `arg_write` creates a temporary file and returns its path.\n\nTaken altogether, this allows the `send_data` function to work with a combinatorial\nexplosion of type signatures:\n\n* `send_data(src::AbstractString)`\n* `send_data(src::AbstractCmd)`\n* `send_data(src::IO)`\n* `send_data(src::AbstractString, dst::AbstractString)`\n* `send_data(src::AbstractCmd,    dst::AbstractString)`\n* `send_data(src::IO,             dst::AbstractString)`\n* `send_data(src::AbstractString, dst::AbstractCmd)`\n* `send_data(src::AbstractCmd,    dst::AbstractCmd)`\n* `send_data(src::IO,             dst::AbstractCmd)`\n* `send_data(src::AbstractString, dst::IO)`\n* `send_data(src::AbstractCmd,    dst::IO)`\n* `send_data(src::IO,             dst::IO)`\n\nEach combination guarantees the proper initialization and cleanup of its\narguments whether it is opening a file and closing it upon completion or error,\nor creating a temporary output file and returning it upon completion or deleting\nit on error. If the arguments are commands or pipelines, those are correctly\nopened with the necessary read/write options.\n\n### Testing Example\n\nNow that we've defined the `send_data` function, we must test it. But it has so\nmany different kinds of arguments that it can accept, how do we produce tests\nfor all of these combinations? `ArgTools` also offers tools to help with testing\nAPIs that it lets you define. The example tests assume that the above definition\nof `send_data` has already been evaluated in the same Julia session.\n\n```jl\nusing Test\n\n# create a source file\nsrc_file = tempname()\ndata = rand(UInt8, 666)\nwrite(src_file, data)\n\nprint_sig(args...) =\n    println(\"send_data(\", join(map(typeof, args), \", \"), \")\")\n\narg_readers(src_file) do src\n    # test 1-arg methods\n    @arg_test src begin\n        print_sig(src)\n        dst_file = send_data(src)\n        @test data == read(dst_file)\n        rm(dst_file)\n    end\n\n    # test 2-arg methods\n    arg_writers() do dst_file, dst\n        @arg_test src dst begin\n            print_sig(src, dst)\n            @test dst == send_data(src, dst)\n        end\n        @test data == read(dst_file)\n    end\nend\n\n# cleanup\nrm(src_file)\n```\n\nEvaluating this testing code prints the following output:\n```jl\nsend_data(String)\nsend_data(String, String)\nsend_data(String, Cmd)\nsend_data(String, Base.CmdRedirect)\nsend_data(String, IOStream)\nsend_data(String, Base.Process)\nsend_data(Cmd)\nsend_data(Cmd, String)\nsend_data(Cmd, Cmd)\nsend_data(Cmd, Base.CmdRedirect)\nsend_data(Cmd, IOStream)\nsend_data(Cmd, Base.Process)\nsend_data(Base.CmdRedirect)\nsend_data(Base.CmdRedirect, String)\nsend_data(Base.CmdRedirect, Cmd)\nsend_data(Base.CmdRedirect, Base.CmdRedirect)\nsend_data(Base.CmdRedirect, IOStream)\nsend_data(Base.CmdRedirect, Base.Process)\nsend_data(IOStream)\nsend_data(IOStream, String)\nsend_data(IOStream, Cmd)\nsend_data(IOStream, Base.CmdRedirect)\nsend_data(IOStream, IOStream)\nsend_data(IOStream, Base.Process)\nsend_data(Base.Process)\nsend_data(Base.Process, String)\nsend_data(Base.Process, Cmd)\nsend_data(Base.Process, Base.CmdRedirect)\nsend_data(Base.Process, IOStream)\nsend_data(Base.Process, Base.Process)\n```\n\nTest code doesn't isn't normally this verbose, but for this example it may be\nhelpful to understand what's happening. What this output shows is the various\nways in which this short bit of code tests invoking the `send_data` function.\nHere are some details about what's happening:\n\n* The call to `arg_readers(src_file)` evaluates the attached do block with five\n  different `arg` values, which can be converted to readable arguments of the\n  types: `String`, `Cmd`, `CmdRedirect`, `IOStream` and `Process`.\n\n* The call to `@arg_test src begin ... end` converts `src` into a readable\n  arguments of those same types and closes or finalizes each at the end.\n\n* The call to `arg_writers()` evaluates the attached do block with five\n  different `arg` values, which can be converted to writable arguments of the\n  types: `String`, `Cmd`, `CmdRedirect`, `IOStream` and `Process`.\n\n* The call to `@arg_test src dst begin ... end` converts `src` into a readable\n  arguments and `dst` into writeable arguments of the same set of types, and\n  closes or otherwise finalizes each one at the end of the block.\n\nThis example test code illustrates some of the reasoning features of the testing\nAPI which might initially seem puzzling. For example, it shows why `arg_readers`\nand `arg_writers` don't simply produce argument values that can be passed to the\nfunction being tested, instead requiring conversion by the `@arg_test` macro.\nThere are two reasons:\n\n1. The same value returned from `arg_readers` or `arg_writers` may need to be\n   used in multiple tests and some argument types, such as IO handles, need to\n   be initialized before each test and finalized after. The `@arg_test` block\n   delimits where initialization and finalization occur.\n\n2. Sometimes operations need to be done after the `@arg_test` block but before\n   the end of the enclosing `arg_readers` or `arg_writers` block. Testing that\n   `dst_file` has the expected contents, i.e. `@test data == read(dst_file)`,\n   will not work reliably inside of the `@arg_test` block: data is not guaranteed\n   to have been fully written to `dst_file` until `dst` is finalized. This is an\n   issue when `dst` is an already-opened process, for example: `arg_write` leaves\n   the process open since it received it that way (you might want to write more\n   data to it), and while it does flush the handle, there is no guarantee that\n   the process will get data to its final destination until the process has\n   exited. Putting the test after the `@arg_test` block ensures that the process\n   has terminated, so we can reliably test the contents of `dst_file`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuliaio%2Fargtools.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjuliaio%2Fargtools.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuliaio%2Fargtools.jl/lists"}