{"id":13738063,"url":"https://github.com/pwwang/cmdy","last_synced_at":"2025-04-29T09:56:28.122Z","repository":{"id":62563068,"uuid":"181923603","full_name":"pwwang/cmdy","owner":"pwwang","description":"\"Shell language\" to run command in python","archived":false,"fork":false,"pushed_at":"2023-06-27T03:18:47.000Z","size":217,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-29T09:56:22.859Z","etag":null,"topics":["command","shell","subprocess"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pwwang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-04-17T15:54:18.000Z","updated_at":"2024-12-30T22:24:11.000Z","dependencies_parsed_at":"2024-04-20T04:33:34.030Z","dependency_job_id":"e7d0dbee-99b7-4b25-8809-03845a8955a5","html_url":"https://github.com/pwwang/cmdy","commit_stats":{"total_commits":98,"total_committers":2,"mean_commits":49.0,"dds":"0.030612244897959218","last_synced_commit":"447b365ff1246a50793024603ebdd84431a1df64"},"previous_names":[],"tags_count":33,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pwwang%2Fcmdy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pwwang%2Fcmdy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pwwang%2Fcmdy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pwwang%2Fcmdy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pwwang","download_url":"https://codeload.github.com/pwwang/cmdy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251480069,"owners_count":21596016,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["command","shell","subprocess"],"created_at":"2024-08-03T03:02:10.223Z","updated_at":"2025-04-29T09:56:28.105Z","avatar_url":"https://github.com/pwwang.png","language":"Python","readme":"# cmdy\n\"Shell language\" to run command in python\n\n[![pypi][1]][2] [![tag][3]][4] [![travis][5]][6] [![codacy quality][7]][8] [![codacy quality][9]][8] ![pyver][10]\n\n## Installation\n```shell\npip install cmdy\n```\n\n## Usage\n\nSee [Demo](./demo.ipynb)\n\n### Basic usage\n```python\nfrom cmdy import ls\nprint(ls())\n```\n\n```python\nfor line in ls().iter():\n    print('Got:', line, end='')\n```\n\n#### With non-keyword arguments\n```python\nfrom cmdy import tar\nprint(tar(\"cvf\", \"/tmp/test.tar\", \"./cmdy\"))\n```\n\n#### With keyword arguments\n```python\nfrom cmdy import curl\ncurl(\"http://duckduckgo.com/\", o=\"/tmp/page.html\", silent=True)\n# curl http://duckduckgo.com/ -o /tmp/page.html --silent\n```\n\n#### Order keyword arguments\n```python\ncurl(\"http://duckduckgo.com/\", \"-o\", \"/tmp/page.html\", \"--silent\")\n# or\n\nfrom diot import OrderedDiot\nkwargs = OrderedDiot()\nkwargs.silent = True\nkwargs.o = '/tmp/page.html'\ncurl(\"http://duckduckgo.com/\", kwargs)\n# You can also use collections.OrderedDict\n```\n\n#### Prefix and separator for keyword arguments\n```python\nfrom cmdy import bedtools, bcftools\nbedtools.intersect(wa=True, wb=True,\n                   a='query.bed', b=['d1.bed', 'd2.bed', 'd3.bed'],\n                   names=['d1', 'd2', 'd3'], sorted=True,\n                   _prefix='-').h().strcmd\n# 'bedtools intersect -wa -wb -a query.bed \\\n# -b d1.bed d2.bed d3.bed -names d1 d2 d3 -sorted'\n```\n\n```python\n# default prefix is auto\nbcftools.query(_=['a.vcf', 'b.vcf'], H=True,\n               format='%CHROM\\t%POS\\t%REF\\t%ALT\\n').h().strcmd\n\n# \"bcftools query -H --format '%CHROM\\t%POS\\t%REF\\t%ALT\\n' a.vcf b.vcf\"\n\nls(l=True, block_size='KB', _sep='auto').h().cmd\n['ls', '-l', '--block-size=KB']\n```\n\n#### Mixed combinations of prefices and separators in one command\n```python\nfrom cmdy import java\n# Note this is just an example for old verion picard.\n# Picard is changing it's style\n\npicard = java(jar='picard.jar', _prefix='', _sep='=', _sub=True)\nc = picard.SortSam(I='input.bam', O='sorted.bam',\n               SORTED_ORDER='coordinate',\n               _prefix='', _sep='=', _deform=None).h\nprint(c.cmd)\n\n# same as the above\njava({'jar': 'picard.jar', '_prefix': '-', '_sep': ' '},\n     'SortSam', I='input.bam', O='sorted.bam',\n     SORTED_ORDER='coordinate', _prefix='', _sep='=', _deform=None).h().cmd\n\n# _deform prevents SORTED_ORDER to be deformed to SORTED-ORDER\n\n# ['java', 'jar=picard.jar',\n#  'SortSam', 'I=input.bam', 'O=sorted.bam', 'SORTED_ORDER=coordinate']\n```\n\n#### Subcommands\n\n```python\nfrom cmdy import git\ngit.branch(v=True).fg\n# \u003cCmdyResult: ['git', 'branch', '-v']\u003e\n```\n\n```python\n# What if I have separate arguments for main and sub-command?\ngit(git_dir='.', _sub=True).branch(v=True).h\n# \u003cCmdyHolding: ['git', '--git-dir', '.', 'branch', '-v']\u003e\n```\n\n#### Duplicated keys for list arguments:\n```python\nfrom cmdy import sort\nprint(sort(k=['1,1', '2,2'], t='_', _='./.editorconfig', _dupkey=True))\n# sort -k 1,1 -k 2,2 ./.editorconfig\n```\n\n### Return code and exception\n```python\nfrom cmdy import x\nx()\n```\n\n```python console\nTraceback (most recent call last):\n  File \"\u003cipython-input-16-092cc5b72e61\u003e\", line 2, in \u003cmodule\u003e\n    x()\n/path/.../to/cmdy/__init__.py\", line 146, in __call__\n    ready_cfgargs, ready_popenargs, _will())\n/path/.../to/cmdy/__init__.py\", line 201, in __new__\n    result = holding.run()\n/path/.../to/cmdy/__init__.py\", line 854, in run\n    return orig_run(self, wait)\n/path/.../to/cmdy/__init__.py\", line 717, in run\n    return orig_run(self, wait)\n/path/.../to/cmdy/__init__.py\", line 327, in run\n    ret = CmdyResult(self._run(), self)\n/path/.../to/cmdy/__init__.py\", line 271, in _run\n    raise CmdyExecNotFoundError(str(fnfe)) from None\ncmdy.cmdy_util.CmdyExecNotFoundError: [Errno 2] No such file or directory: 'x': 'x'\n```\n\n```python\nfrom cmdy import ls\nls('non-existing-file')\n```\n\n```python console\n\nTraceback (most recent call last):\n  File \"\u003cipython-input-17-132683fc2227\u003e\", line 2, in \u003cmodule\u003e\n    ls('non-existing-file')\n/path/.../to/cmdy/__init__.py\", line 146, in __call__\n    ready_cfgargs, ready_popenargs, _will())\n/path/.../to/cmdy/__init__.py\", line 204, in __new__\n    return result.wait()\n/path/.../to/cmdy/__init__.py\", line 407, in wait\n    raise CmdyReturnCodeError(self)\ncmdy.cmdy_util.CmdyReturnCodeError: Unexpected RETURN CODE 2, expecting: [0]\n\n  [   PID] 167164\n\n  [   CMD] ['ls non-existing-file']\n\n  [STDOUT]\n\n  [STDERR] ls: cannot access non-existing-file: No such file or directory\n```\n\n#### Don't raise exception but store the return code\n```python\nfrom cmdy import ls\nresult = ls('non-existing-file', _raise=False)\nresult.rc # 2\n```\n\n#### Tolerance on return code\n```python\nfrom cmdy import ls\n# or _okcode=[0,2]\nls('non-existing-file', _okcode='0,2').rc # 2\n```\n\n### Timeouts\n```python\nfrom cmdy import sleep\nsleep(3, _timeout=1)\n```\n\n```python console\nTraceback (most recent call last):\n  File \"\u003cipython-input-20-47b0ec7af55f\u003e\", line 2, in \u003cmodule\u003e\n    sleep(3, _timeout=1)\n/path/.../to/cmdy/__init__.py\", line 146, in __call__\n    ready_cfgargs, ready_popenargs, _will())\n/path/.../to/cmdy/__init__.py\", line 204, in __new__\n    return result.wait()\n/path/.../to/cmdy/__init__.py\", line 404, in wait\n    ) from None\ncmdy.cmdy_util.CmdyTimeoutError: Timeout after 1 seconds.\n```\n\n### Redirections\n```python\nfrom cmdy import cat\ncat('./pytest.ini').redirect \u003e '/tmp/pytest.ini'\nprint(cat('/tmp/pytest.ini'))\n```\n\n#### Appending\n```python\n# r short for redirect\ncat('./pytest.ini').r \u003e\u003e '/tmp/pytest.ini'\nprint(cat('/tmp/pytest.ini')) # content doubled\n```\n\n#### Redirecting to a file handler\n```python\nwith open('/tmp/pytest.ini', 'w') as f\n    cat('./pytest.ini').r \u003e f\n\nprint(cat('/tmp/pytest.ini'))\n```\n\n#### STDIN, STDOUT and/or STDERR redirections\n```python\nfrom cmdy import STDIN, STDOUT, STDERR, DEVNULL\n\nc = cat().r(STDIN) \u003c '/tmp/pytest.ini'\nprint(c)\n```\n\n```python\n# Mixed\nc = cat().r(STDIN, STDOUT) ^ '/tmp/pytest.ini' \u003e DEVNULL\n# we can't fetch result from a redirected pipe\nprint(c.stdout)\n\n# Why not '\u003c' for STDIN?\n# Because the priority of the operator is not in sequential order.\n# We can use \u003c for STDIN, but we need to ensure it runs first\nc = (cat().r(STDIN, STDOUT) \u003c '/tmp/pytest.ini') \u003e DEVNULL\nprint(c.stdout)\n\n# A simple rule for multiple redirections to always use \"\u003e\" in the last place\n```\n\n```python\n# Redirect stderr to stdout\nfrom cmdy import bash\nc = bash(c=\"cat 1\u003e\u00262\").r(STDIN, STDERR) ^ '/tmp/pytest.ini' \u003e STDOUT\nprint(c.stdout)\n```\n\n```python\n# Redirect the world\nc = bash(c=\"cat 1\u003e\u00262\").r(STDIN, STDOUT, STDERR) ^ '/tmp/pytest.ini' ^ DEVNULL \u003e STDOUT\nprint(c.stdout) # None\nprint(c.stderr) # None\n```\n\n### Pipings\n```python\nfrom cmdy import grep\nc = ls().p | grep('README')\nprint(c)\n# README.md\n# README.rst\n```\n\n```python\n# p short for pipe\nc = ls().p | grep('README').p | grep('md')\nprint(c) # README.md\nprint(c.piped_strcmds) # ['ls', 'grep README', 'grep md']\n```\n\n```python\nfrom cmdy import _CMDY_EVENT\n# !!! Pipings should be consumed immediately!\n# !!! DO NOT do this\nls().p\nls() # \u003c- Will not run as expected\n# All commands will be locked as holding until pipings are consumed\n_CMDY_EVENT.clear()\nprint(ls()) # runs\n\n# See Advanced/Holdings if you want to hold a piping command for a while\n```\n\n### Running command in foreground\n```python\nls().fg\n```\n\n```python\nfrom cmdy import tail\ntail('/tmp/pytest.ini', f=True, _timeout=3).fg\n# This mimics the `tail -f` program\n# You will see the content comes out one after another\n# and then program hangs for 3s\n```\n\nYou can also write an `echo-like` program easily. See '[echo.py](./echo.py)'\n\n### Iterating on output\n```python\nfor line in ls().iter():\n    print(line, end='')\n```\n\n#### Iterating on stderr\n```python\nfor line in bash(c=\"cat /tmp/pytest.ini 1\u003e\u00262\").iter(STDERR):\n    print(line, end='')\n```\n\n#### Getting live output\n```python\n# Like we did for `tail -f` program\n# This time, we can do something with each output line\n\n# Let's use a thread to write content to a file\n# And we try to get the live contents using cmdy\nimport time\nfrom threading import Thread\ndef live_write(file, n):\n\n    with open(file, 'w', buffering=1) as f:\n        # Let's write something every half second\n        for i in range(n):\n            f.write(str(i) + '\\n')\n            time.sleep(.5)\n\ntest_file = '/tmp/tail-f.txt'\nThread(target=live_write, args=(test_file, 10)).start()\n\nfrom cmdy import tail\n\ntail_iter = tail(f=True, _=test_file).iter()\n\nfor line in tail_iter:\n    # Do whatever you want with the line\n    print('We got:', line, end='')\n    if line.strip() == '8':\n        break\n\n# make sure thread ends\ntime.sleep(2)\n```\n\n```python\n# What about timeout?\n\n# Of course you can use a timer to check inside the loop\n# You can also set a timeout for each fetch\n\n# Terminate after 10 queries\n\nThread(target=live_write, args=(test_file, 10)).start()\n\nfrom cmdy import tail\n\ntail_iter = tail(f=True, _=test_file).iter()\n\nfor i in range(10):\n    print('We got:', tail_iter.next(timeout=1), end='')\n```\n\n### Advanced\n#### Baking the `cmdy` object\n\nSometimes, you may want to run the same program a couple of times, with the same set of arguments or configurations, and you don't want to type those arguments every time, then you can bake the Cmdy object with that same arguments or configurations.\n\nFor example, if you want to run ls as ls -l all the time:\n\n```python\nfrom cmdy import ls\nll = ls._(l=True)\nprint(ll().h.cmd) # ['ls', '-l']\nprint(ll(a=True).h.cmd) # ['ls', '-l', '-a']\n# I don't want the l flag for some commands occasionally\nprint(ll(l=False).h.cmd) # ['ls']\n\n# Bake a baked command\nlla = ll._(a=True)\nprint(lla().h.cmd) # ['ls', '-l', '-a']\n```\n\n```python\n# I know git is always gonna run with subcommand\ngit = git._(_sub=True)\n# don't bother to pass _sub=True every time\nprint(git(git_dir='.').branch(v=True).h)\n# \u003cCmdyHolding: ['git', '--git-dir', '.', 'branch', '-v']\u003e\nprint(git().status().h)\n# \u003cCmdyHolding: ['git', 'status']\u003e\n```\n\n```python\n# What if I have a subcommand call bake?\nfrom cmdy import git, CmdyActionError\n\nprint(git.branch().h.cmd) # ['git', 'branch']\nprint(type(git._())) # \u003cclass 'cmdy.Cmdy'\u003e\n\n# run the git with _sub\nprint(git(_sub=True).bake().h.cmd) # ['git', 'bake']\n```\n\n#### Baking the whole module\n```python\nimport cmdy\n# run version of the whole world\nsh = cmdy(version=True)\n# anything under sh directly will be supposed to have subcommand\nfrom sh import git, gcc\nprint(git().h)\n# \u003cCmdyHolding: ['git', '--version']\u003e\nprint(gcc().h)\n# \u003cCmdyHolding: ['gcc', '--version']\u003e\n```\n\nNote that module baking is deep copying, except the exception classes and some utils. This means, you would expect following behavior:\n\n```python\nimport cmdy\nfrom cmdy import CmdyHolding, CmdyExecNotFoundError\n\nsh = cmdy()\n\nc = sh.echo().h\nprint(type(c)) # \u003cclass 'cmdy.CmdyHolding'\u003e\nprint(isinstance(c, CmdyHolding)) # False\nprint(isinstance(c, sh.CmdyHolding)) # True\n\ntry:\n    sh.notexisting()\nexcept CmdyExecNotFoundError:\n    # we can catch it, as CmdyExecNotFoundError is sh.CmdyExecNotFoundError\n    print('Catched!')\n```\n\n#### Holding objects\n\nYou may have noticed that we have a couple of examples above with a final call .h or .h(), which is holding the command from running.\n\nYou can do that, too, if you have multiple operations\n\n```python\nprint(ls().h) # \u003cCmdyHolding: ['ls']\u003e\n\n# however, you cannot hold after some actions\nls().r.h\n# CmdyActionError: Should be called in the first place: .h() or .hold()\n```\n\nOnce a command is on hold (by .h, .hold, .h() or .hold())\n\nYou have to explictly call run() to set the command running\n\n```python\nfrom time import time\ntic = time()\nc = sleep(2).h\nprint(f'Time elapsed: {time() - tic:.3f} s')\n# Time elapsed: 0.022 s\n\n# not running even with fg\nc.fg\nprint(f'Time elapsed: {time() - tic:.3f} s')\n# Time elapsed: 0.034 s\nc.run()\nprint(f'Time elapsed: {time() - tic:.3f} s')\n# Time elapsed: 2.043 s\n```\n\n#### Reuse of command\n```python\n# After you set a command running,\n# you can retrieve the holding object,\n# and reuse it\nfrom cmdy import ls\nc = ls().fg\n# nothing will be produced\nc.holding.reset().r \u003e DEVNULL\n```\n\n#### Async mode\n```python\nimport curio\nfrom cmdy import ls\na = ls().a # async command is never blocking!\n\nasync def main():\n    async for line in a:\n        print(line, end='')\n\ncurio.run(main())\n```\n\n#### Extending `cmdy`\n\nAll those actions for holding/result objects were implemented internally as plugins. You can right your own plugins, too.\n\nA plugin has to be defined as a class and then instantiated.\n\n**There are 5 APIs for developing a plugin for `cmdy`**\n\n- `cmdy._plugin_factory.register`: A decorator for the plugin class\n- `cmdy._plugin_factory.hold_then`: A decorator to decorate methods in the plugin class, which define actions after a holding object. Arguments:\n  - `alias`: The alias of this action (e.g. `r/redir` for `redirect`)\n  - `final`: Whether this is a final action, meaning no other actions should be followed\n  - `prop`: Whether this action can be called as a property\n  - `hold_right`: Should I put right following action on hold? This is useful when we have connectors which then can set the command running. (e.g `\u003e` for redirect and `|` for pipe)\n- `cmdy._plugin_factory.run_then`: A decorator to decorate methods in the plugin class, which define actions after a sync result object. Arguments are similar as `cmdy._plugin_factory.hold_then` except that `prop` and `hold_right` are not avaialbe.\n- `async_run_then.add_method`: A decorator to decorate methods in the plugin class, which add methods to the `CmdyHolding`, `CmdyResult` or `CmdyAsyncResult` class. `cls` is the only argument that specifies which class we are hacking.\n- `async_run_then.add_property`: Property version of `cmdy_plugin_add_method`\n\n**Notes on name conflicts:**\n\nIf we need to add the methods to multiple classes in the plugin with the same name, you can define a different name with extra underscore suffix(es).\n\n**Notes on module baking:**\n\n- Always use the baked module to get those classes\n    ```pytho\n    import cmdy\n    cmdy2 = cmdy()\n\n    @cmdy2._plugin_factory.register\n    class MyPlugin:\n        ...\n    ```\n- Plugin enable and disable only take effect within the same module. For example:\n\n    ```python\n    import cmdy\n    sh = cmdy()\n    # only affects cmdy not sh\n    sh._plugins.fg.disable()\n    # to disable this plugin for sh as well:\n    sh._plugins.fg.disable()\n    ```\n\n```python\n# An example to define a plugin\nimport cmdy\n\n@cmdy._plugin_factory.register\nclass MyPlugin:\n    @cmdy._plugin_factory.add_method(cmdy.CmdyHolding)\n    def say_hello(self):\n        return 'Hello world!'\n\n    @cmdy._plugin_factory.hold_then('hello')\n    def helloworld(self):\n        print(self.say_hello())\n        # keep chaining\n        return self\n\nmyplugin = MyPlugin()\n\n# command will never run,\n# because we didn't do self.run() in helloworld(self)\nls().helloworld()  # prints Hello world!\n# property calls enabled by default\nls().helloworld  # prints Hello world!\n# we have alias\nls().hello  # prints Hello world!\n\n```\n\n[1]: https://img.shields.io/pypi/v/cmdy?style=flat-square\n[2]: https://pypi.org/project/cmdy/\n[3]: https://img.shields.io/github/tag/pwwang/cmdy?style=flat-square\n[4]: https://github.com/pwwang/cmdy\n[5]: https://img.shields.io/travis/pwwang/cmdy?style=flat-square\n[6]: https://travis-ci.org/pwwang/cmdy\n[7]: https://img.shields.io/codacy/grade/fa12f06d39404f98b94c19e83865fd4e?style=flat-square\n[8]: https://app.codacy.com/project/pwwang/cmdy/dashboard\n[9]: https://img.shields.io/codacy/coverage/fa12f06d39404f98b94c19e83865fd4e?style=flat-square\n[10]: https://img.shields.io/pypi/pyversions/cmdy?style=flat-square\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpwwang%2Fcmdy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpwwang%2Fcmdy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpwwang%2Fcmdy/lists"}