{"id":18008597,"url":"https://github.com/mgree/smoosh","last_synced_at":"2025-08-22T06:06:37.715Z","repository":{"id":36251935,"uuid":"150447186","full_name":"mgree/smoosh","owner":"mgree","description":"The Symbolic, Mechanized, Observable, Operational SHell: an executable formalization of the POSIX shell standard.","archived":false,"fork":false,"pushed_at":"2023-02-16T21:51:09.000Z","size":3841,"stargazers_count":118,"open_issues_count":23,"forks_count":5,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-21T19:12:51.246Z","etag":null,"topics":["formalization","posix","posix-sh","posix-shell","semantics","sh","shell","shell-scripting","symbolic-execution"],"latest_commit_sha":null,"homepage":"http://shell.cs.pomona.edu","language":"OCaml","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mgree.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-26T15:17:56.000Z","updated_at":"2025-02-28T19:13:19.000Z","dependencies_parsed_at":"2024-10-30T01:47:01.793Z","dependency_job_id":null,"html_url":"https://github.com/mgree/smoosh","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mgree%2Fsmoosh","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mgree%2Fsmoosh/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mgree%2Fsmoosh/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mgree%2Fsmoosh/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mgree","download_url":"https://codeload.github.com/mgree/smoosh/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245654356,"owners_count":20650852,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["formalization","posix","posix-sh","posix-shell","semantics","sh","shell","shell-scripting","symbolic-execution"],"created_at":"2024-10-30T01:19:15.181Z","updated_at":"2025-03-26T12:32:10.469Z","avatar_url":"https://github.com/mgree.png","language":"OCaml","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Main workflow](https://github.com/mgree/smoosh/actions/workflows/build.yml/badge.svg)](https://github.com/mgree/smoosh/actions/workflows/build.yml)\n\nSmoosh (the Symbolic, Mechanized, Observable, Operational SHell) is a formalization of the [POSIX shell standard](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/contents.html); Smoosh is one part of [Michael Greenberg's](http://www.cs.pomona.edu/~michael/) broader [project on the POSIX shell](http://shell.cs.pomona.edu).\n\nSmoosh is written in a mix of [Lem](https://www.cl.cam.ac.uk/~pes20/lem/) and OCaml, using [libdash](https://github.com/mgree/libdash) to parse shell code.\n\n# Installation\n\nThere are two ways to work with Smoosh: virtually (in a Vagrant VM or\nin a Docker container) or natively. Because Smoosh depends on many\nparts and specific versions of some libraries, it may be easier to\ninstall via a VM or Docker.\n\n## Building Smoosh natively\n\nTo install Smoosh directly on your computer, you will need to manually\nconfigure your system with the dependencies listed in\n`.github/workflows/build.yml`. In particular:\n\n  - A C toolchain\n  - Autoconf, autotools, libtool, pkg-config, libffi, and libgmp (on macOS, this may be called `glibtoolize`, e.g., run `brew install libtool`)\n  - OPAM (well sourced, i.e., `eval $(opam env)`)\n  - Ruby and Ruby bundler (for the web server)\n\nOnce you have those dependencies, you should be able to crib from `.travis.yml`:\n\n```ShellSession\n$ git clone --recurse-submodules https://github.com/mgree/smoosh.git\nCloning into 'smoosh'...\n... [lots of fetching] ...\nSubmodule path 'libdash': checked out 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy'\nSubmodule path 'modernish': checked out 'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz'\n$ cd smoosh\n$ opam install -y ocamlfind ocamlbuild num zarith extunix lem\n$ (cd libdash; opam pin -y add .)\n$ make -C src all all.byte\n$ export PATH=\"$(pwd)/src:$PATH\"\n```\n\nThanks to [@idkjs](https://github.com/idkjs) for documenting a [macOS build](https://github.com/idkjs/smoosh-macOS).\n\n\n## Building Smoosh in a Vagrant VM\n\nIn a system with Vagrant, you should be able to download the Smoosh VM\nimage from the Vagrant Cloud service (~1.3GB):\n\n```ShellSession\n~$ mkdir smoosh; cd smoosh\n~/smoosh$ vagrant init mgree/smoosh --box-version 0.1.1\n~/smoosh$ vagrant up\nBringing machine 'default' up with 'virtualbox' provider...\n==\u003e default: Box 'mgree/smoosh' could not be found. Attempting to find and install...\n    default: Box Provider: virtualbox\n    default: Box Version: 0.1.1\n==\u003e default: Loading metadata for box 'mgree/smoosh'\n    default: URL: https://vagrantcloud.com/mgree/smoosh\n==\u003e default: Adding box 'mgree/smoosh' (v0.1.1) for provider: virtualbox\n    default: Downloading: https://vagrantcloud.com/mgree/boxes/smoosh/versions/0.1.1/providers/virtualbox.box\n...\n~/smoosh$ vagrant ssh\nvagrant@debian9:~$ cd smoosh\nvagrant@debian9:~/smoosh$\n```\n\nYou are now in a directory where you can run tests. The `smoosh`\nexecutable should be on your path in any case.\n\n## Building Smoosh in Docker\n\nTo build via Docker, you merely need to fetch the Smoosh repo and its submodules. The `build.sh` script in the base of the repo will invoke the appropriate Docker commands.\n\n```ShellSession\n$ git clone --recurse-submodules https://github.com/mgree/smoosh.git\nCloning into 'smoosh'...\n... [lots of fetching] ...\nSubmodule path 'libdash': checked out 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy'\nSubmodule path 'modernish': checked out 'zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz'\n$ cd smoosh\n$ ./build.sh\n... [long docker build] ...\nSuccessfully tagged smoosh:latest\n... [tests build] ...\nSuccessfully tagged smoosh-test:latest\n... [tests run] ...\n========================================================================\n\nsmoosh v0.1 (build YYYY-MM-DD_HH:MM)\n\nALL TESTS PASSED\n... [more docker builds] ...\nSuccessfully tagged smoosh-web:latest\n```\n\nIf the build process was successful, there are now two tagged Docker images, which can be run interactively via `docker run -it [image tag]`. The two images are:\n\n  - `smoosh`, a Docker environment with Smoosh installed as `/bin/smoosh`\n  - `smoosh-test`, an extension of the `smoosh` image with unit and system tests\n\n### Running tests using the Docker image\n\n- To run the test suite after building, run: `docker build -t smoosh-test -f Dockerfile.test . \u0026\u0026 docker run smoosh-test`\n- To explore the built image, run: `docker run -it smoosh`\n\nTo test by hand, there are three sets of relevant tests: the libdash tests (in `libdash/test`), the unit tests for symbolic smoosh (in `src`), and the shell test suite (in `tests`). All three directories have `Makefile`s with appropriate `test` targets, so you can test both by running the following:\n\n```ShellSession\n$ docker run -it smoosh-test\nopam@XXXXXXXXXXXX:~$ make -C libdash/test test\nopam@XXXXXXXXXXXX:~$ make -C src/ test\nopam@XXXXXXXXXXXX:~$ make -C tests test\n```\n\n### Using the Shtepper\n\nThe [Shtepper](http://shell.cs.pomona.edu/shtepper) is a web-based visualization tool for symbolically running POSIX shell scripts. While available online, you can also run a local version of the Shtepper using the `smoosh-web` Docker image. To start the local Shtepper, you must build the web interface first. Run, from the `smoosh` repo root:\n\n```ShellSession\n$ docker build -t smoosh-web -f Dockerfile.web .\n...\n$ docker run -p 80:2080 --name smoosh-web -t smoosh-web\nThin web server (v1.7.2 codename Bachmanity)\nMaximum connections set to 1024\nListening on 0.0.0.0:2080, CTRL+C to stop\n...\n```\n\nYou can then navigate to [http://localhost/](http://localhost/) in your web browser of choice. The Shtepper should work in any web browser, but has only undergone extensive testing in Firefox.\n\n# Running tests\n\nHowever you've installed Smoosh, you can run the tests by going to the\nappropriate Smoosh directory (the home directory in Docker; `~/smoosh`\nin a Vagrant VM). There are three sets of local Smoosh tests: libdash\nparser tests, unit tests, and system tests. You can run them in one go\nas follows:\n\n```ShellSession\nvagrant@debian9:~/smoosh$ make -C libdash/test test \u0026\u0026 make -C src test \u0026\u0026 make -C tests\nmake: Entering directory '/home/vagrant/smoosh/libdash/test'\nocamlfind ocamlopt -g -package dash,ctypes,ctypes.foreign -linkpkg test.ml -o test.native\nocamlfind ocamlcp -p a -package dash,ctypes,ctypes.foreign -linkpkg test.ml -o test.byte\nTESTING test.native\ntests/braces_amp.sh OK\ntests/builtin.trap.exitcode.test OK\ntests/diverge.sh OK\ntests/empty_case OK\ntests/escaping OK\ntests/for_spaces.sh OK\ntests/grab_submissions.sh OK\ntests/grade.sh OK\ntests/redir_indirect OK\ntests/run_grader.sh OK\ntests/run_lda.sh OK\ntests/send_emails.sh OK\ntests/syntax OK\ntests/test.sh OK\ntests/tilde_arith OK\ntests/timeout3 OK\nTESTING test.byte\ntests/braces_amp.sh OK\ntests/builtin.trap.exitcode.test OK\ntests/diverge.sh OK\ntests/empty_case OK\ntests/escaping OK\ntests/for_spaces.sh OK\ntests/grab_submissions.sh OK\ntests/grade.sh OK\ntests/redir_indirect OK\ntests/run_grader.sh OK\ntests/run_lda.sh OK\ntests/send_emails.sh OK\ntests/syntax OK\ntests/test.sh OK\ntests/tilde_arith OK\ntests/timeout3 OK\nmake: Leaving directory '/home/vagrant/smoosh/libdash/test'\nmake: Entering directory '/home/vagrant/smoosh/src'\n./runtest\n\n=== Initializing Dash parser...\n=== Running evaluation tests...\n=== ...ran 229 evaluation tests with 0 failures.\n\n\n=== Running word expansion tests...\n=== ...ran 64 word expansion tests with 0 failures.\n\n\n=== Running path/fs tests...\n=== ...ran 27 path/fs tests with 0 failures.\n\n\n=== Running arithmetic tests...\n=== ...ran 253 arithmetic tests with 0 failures.\n\nmake: Leaving directory '/home/vagrant/smoosh/src'\nmake: Entering directory '/home/vagrant/smoosh/tests'\n== Running shell tests ===============================================\n......................................................................\n......................................................................\n......................\nshell_tests.sh: 162/162 tests passed\nmake: Leaving directory '/home/vagrant/smoosh/tests'\n```\n\nThe unit tests are run directly via a binary, `smoosh/src/runtest`;\nthe system tests use the `Makefile` in the `smoosh/tests`\ndirectory. These system tests are shell scripts paired with expected\nSTDOUT, STDERR, and exit statuses. \n\n## Running tests on another shell\n\nBoth the Docker image and Vagrant VM will have other shells\ninstalled. The versions are nearly the same as those mentioned in the\npaper, but may have changed slightly due to rolling releases. (Numbers\nmay therefore differ slightly.)\n\n|Shell|Version          |\n|:----|:----------------|\n|dash |0.5.8-2.4        |\n|bash |4.4-5            |\n|yash |2.43-1           |\n|zsh  |5.3.1-4+b3       |\n|ksh  |93u+20120801-3.1 |\n|mksh |54-2+b4          |\n\nYou can run the system tests on any shell by setting `TEST_SHELL`. Some shells may not terminate on all tests; you may need to run `make clean` while changing tests.\n\n```ShellSession\nvagrant@debian9:~/smoosh$ TEST_SHELL=dash make -C tests\n...\n```\n\nTo get more detail on test failures, you should set the `TEST_DEBUG` variable, e.g.:\n\n```ShellSession\nvagrant@debian9:~/smoosh$ TEST_DEBUG=1 TEST_SHELL=dash make -C tests\n...\n```\n\n## Running Modernish's tests/shell diagnostic\n\nTo run the Modernish tests, you must go to `smoosh/modernish` and\nsimulate an install as follows:\n\n```ShellSession\nvagrant@debian9:~/smoosh/modernish$ yes n | ./install.sh -s smoosh\nRelaunching install.sh with /home/mgree/.local/bin/smoosh...\n* Modernish version 0.15.2-dev, now running on /home/mgree/.local/bin/smoosh.\n* This shell identifies itself as smoosh version 0.1.\n  Modernish detected the following bugs, quirks and/or extra features on it:\n... [weird noise on native Linux/VM; crash in Docker] ...\n   LOCALVARS TRAPPRSUBSH BUG_MULTIBYTE\n* Running modernish test suite on /home/mgree/.local/bin/smoosh ...\n* lib/modernish/tst/@sanitychecks.t \n  002: ASCII chars and control char constants   - FAIL\n* WARNING: modernish has some bug(s) in combination with this shell.\n           Run 'modernish --test' after installation for more details.\nAre you happy with /home/mgree/.local/bin/smoosh as the default shell? (y/n) install.sh: Aborting.\n```\n\nTo test another shell, run `yes n | smoosh/modernish/install.sh -s [shell name]`.\n\nNB that the HDOCMASK bug seems to appear only in Linux and not on\nmacOS.\n\n# POPL 2020 Artifact Evaluation\n\nWhat can be reproduced from the Smoosh paper?\n\n  - You should be able to build Smoosh on any computer that supports Docker.\n\n  - The built Smoosh should pass all of its unit and system tests.\n\n  - You should be able to run the Smoosh system tests on other shells. Due to rolling releases, your environment may have slightly different shell versions (which may then pass a different number of tests).\n\nWhat can not be reproduced from the Smoosh paper?\n\n  - The POSIX test suite cannot be distributed, so we cannot reproduce\n    those tests. We do, however, have permission to distribute the\n    resulting journals from running the test suite. Look in\n    `smoosh/posix-journals`.\n    \n  - As of 2019-10-21, Modernish under virtualization (whether in\n    Docker or in a Vagrant VM) exposes a bug in Smoosh's interaction\n    with the dash parser. This bug wasn't poked when running on macOS.\n    \n    The manifestations are different: in Docker, Smoosh crashes with a\n    'broken DEFPATH' error; in a VM, some backtraces appear but the\n    Modernish diagnostic completes with the correct output (just\n    `BUG_MULTIBYTE`).\n    \n    Modernish _should_ still complete without a problem on macOS, but\n    I'm unable to test this (as my Mac is not booting).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmgree%2Fsmoosh","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmgree%2Fsmoosh","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmgree%2Fsmoosh/lists"}