{"id":17454275,"url":"https://github.com/spamegg1/unix-sml","last_synced_at":"2026-04-16T05:33:27.662Z","repository":{"id":63225629,"uuid":"562153637","full_name":"spamegg1/unix-sml","owner":"spamegg1","description":"Working through \"Unix System Programming with Standard ML\" by Anthony L. Shipman, 2001","archived":false,"fork":false,"pushed_at":"2023-01-21T12:37:26.000Z","size":1355,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-16T09:00:32.716Z","etag":null,"topics":["alice-ml","archlinux","debian","fedora","linux","ml","mlkit","mlton","moscow-ml","mosml","polyml","sml","sml-nj","smlnj","smlsharp","standard-ml","system-programming","ubuntu","unix"],"latest_commit_sha":null,"homepage":"","language":"Standard ML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spamegg1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-11-05T13:27:29.000Z","updated_at":"2024-02-02T20:18:14.000Z","dependencies_parsed_at":"2023-01-22T15:45:41.415Z","dependency_job_id":null,"html_url":"https://github.com/spamegg1/unix-sml","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/spamegg1/unix-sml","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spamegg1%2Funix-sml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spamegg1%2Funix-sml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spamegg1%2Funix-sml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spamegg1%2Funix-sml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spamegg1","download_url":"https://codeload.github.com/spamegg1/unix-sml/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spamegg1%2Funix-sml/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31872651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"online","status_checked_at":"2026-04-16T02:00:06.042Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alice-ml","archlinux","debian","fedora","linux","ml","mlkit","mlton","moscow-ml","mosml","polyml","sml","sml-nj","smlnj","smlsharp","standard-ml","system-programming","ubuntu","unix"],"created_at":"2024-10-18T01:27:53.901Z","updated_at":"2026-04-16T05:33:27.644Z","avatar_url":"https://github.com/spamegg1.png","language":"Standard ML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# UNIX System Programming with Standard ML\n\n## Getting Standard ML\n\n### Basic installation on Linux, MacOS, Windows\n\n```bash\nsudo apt install smlnj # On Debian/Ubuntu and derivatives\nsudo pacman -S smlnj   # On Arch and derivatives \n```\n\nOn Debian/Ubuntu, this will install the language in `/usr/lib/smlnj`. \n\n[Mac OS installer download link](http://smlnj.cs.uchicago.edu/dist/working/2022.1/smlnj-amd64-2022.1.pkg)\n\n[Windows installer download link](http://smlnj.cs.uchicago.edu/dist/working/110.99.3/smlnj-110.99.3.msi)\n\n### Libraries\n\nYou can also get all kinds of extra `sml` libraries and tools:\n\n```bash\nsudo apt install ml-yacc ml-lex ml-lpt libckit-smlnj libcml-smlnj libcmlutil-smlnj libexene-smlnj libmlnlffi-smlnj libmlrisctools-smlnj libpgraphutil-smlnj libsml-dev smlnj-doc\n```\n\nAs a shortcut to install all available `smlnj` packages, you can use:\n\n```bash\nsudo apt install smlnj* *smlnj ml-*\n```\n\n### The REPL\n\nThe REPL is quite difficult to use because it does not support cycling through the history of commands with Up/Down arrow keys, or navigating left/right on the line of input with Left/Right arrow keys. To fix these issues, you should install `rlwrap`:\n\n```bash\nsudo apt install rlwrap # Debian/Ubuntu\nsudo dnf install rlwrap # Fedora\nsudo pacman -S rlwrap   # Arch\nbrew install rlwrap     # MacOS users can get it on Homebrew: https://brew.sh\n```\n\n(Windows users are screwed. [`rlwrap`](https://github.com/hanslub42/rlwrap) is not easily available as far as I know. I looked at [Chocolatey](https://chocolatey.org/), [Scoop](https://scoop.sh/), [Ninite](https://ninite.com/) and [Winget](https://winget.run/); none of them have it. You'd have to compile from source, which I don't expect to be easy on Windows.)\n\nNow you should make an alias in your `~/.bash_aliases` file:\n\n```bash\nalias sml='rlwrap sml'\n```\n\nThis applies to other ML implementations too (mentioned below):\n\n```bash\nalias poly='rlwrap poly'\nalias ocaml='rlwrap ocaml'\nalias alice='rlwrap alice'\nalias mosml='rlwrap mosml'\nalias smlsharp='rlwrap smlsharp'\n```\n\n### Other systems and ML's\n\n#### Fedora\n\nFedora repositories do not have `smlnj`. You will have to [install](https://www.smlnj.org/dist/working/2021.1/install.html) from [source](http://smlnj.cs.uchicago.edu/dist/working/2021.1/config.tgz). \n\n#### Adventures in ML Wonderland\n\nEven more adventurous options below!\n\n#### Poly/ML\n\nThere is [Poly/ML](https://www.polyml.org/index.html) on [Fedora](https://packages.fedoraproject.org/pkgs/polyml/polyml/), Debian and Arch: \n\n```bash\nsudo apt install polyml # Debian/Ubuntu\nsudo dnf install polyml # Fedora\nsudo pacman -S polyml   # Arch\n```\n\nbut it does not come with the SML/NJ libraries the book uses. You'll have to compile/install them from [source](https://github.com/eldesh/smlnjlib-polyml). Not easy. [Has](https://github.com/eldesh/mllex-polyml) [dependencies](https://github.com/eldesh/mlyacc-polyml) that have [build issues](https://github.com/eldesh/mlyacc-polyml/issues/3). Then you should be good.\n\nWith Poly/ML you can fire up a REPL just like `sml`, but with the `poly` command instead. Normal `sml` syntax works, like `use \"myfile.sml\";`. You can use [the `polyc` command](https://www.polyml.org/FAQ.html#standalone) to create executables, but you need to install the development libraries (`libpolyml-dev` package on Ubuntu). I was able to compile a hello world program to an executable successfully. Not sure how it works when the SML/NJ library is involved.\n\n#### MLton\n\n[MLton](http://mlton.org/) (available on [Fedora](https://packages.fedoraproject.org/pkgs/mlton/mlton/) and Arch, not on Debian/Ubuntu): \n\n```bash\nsudo dnf install mlton # Fedora\nsudo pacman -S mlton   # Arch\n```\n\nI was able to compile and install from [source](https://github.com/MLton/mlton) on Ubuntu, with\n\n```bash\nsudo make\nsudo make install\n```\n\nand it was installed under `/usr/local/lib/mlton` and it really does have ([a port of](http://mlton.org/SMLNJLibrary)) the [SML/NJ library](https://www.smlnj.org/doc/smlnj-lib/index.html) under `/usr/local/lib/mlton/sml/smlnj-lib`. I was able to compile a hello world program, but running it does not print anything. Don't know why.\n\nMLton is all about compile-optimizing the whole program, so it does not have a convenient REPL. This is a huge downside for going through the book. But you can make executables with it. There is also support for SML/NJ's Compilation Manager in [MLton](http://mlton.org/CompilationManager) (it has to port `.cm` files to its own format first). It has the `SMLofNJ` structure with the `exportFn` function we'll use below. So if you don't mind the absence of a REPL and you're OK with the \"edit/compile/edit/compile\" cycle way of doing things, then it might work for you.\n\n#### OCaml\n\nOf course there is also [OCaml](https://www.ocaml.org/) but it's an entirely different language at this point (take a look into `/usr/lib/ocaml` and compare it to `/usr/lib/smlnj/lib`). It's available on Debian, Fedora and Arch:\n\n```bash\nsudo apt install ocaml # Debian/Ubuntu\nsudo dnf install ocaml # Fedora\nsudo pacman -S ocaml   # Arch\n```\n\nIt would be interesting to do this book with OCaml, but it would be a lot of work, more of an \"adaptation\" of this book rather than simply going through it. However you might find that many things are much easier to do with OCaml, since it's an industrial language and probably has better support for a lot of things out of the box. There are WAY MORE library packages on Ubuntu for `ocaml` than `sml`.\n\n#### SML# (SMLsharp)\n\nAvailable only on Debian/Ubuntu (not on Fedora or Arch):\n\n```bash\nsudo apt install smlsharp\n```\n\nThis gets me the latest version 4.0.0.\n\nI couldn't get it to work, it wants LLVM version 13, even though I have 14. After manually installing LLVM 13 and also `libmassivethreads-dev` at least it works. \n\nI stand corrected: [it is available on other systems](https://github.com/smlsharp/repos): CentOS, Fedora, MacOS via [Homebrew](https://brew.sh), and Windows (via WSL2, like Ubuntu).\n\n```bash\nbrew tap smlsharp/smlsharp # for Mac OS, but should\nbrew install smlsharp      # also work on any Linux\n```\nOne very interesting aspect of SML# is [\"record polymorphism\"](https://smlsharp.github.io/en/about/features/) which was mentioned as a hypothetical type system feature in some of the exam questions in [Programming Languages Part A](https://www.coursera.org/learn/programming-languages) course on Coursera.\n\nDoes that look like a REPL? \n\n```sml\n ➜ smlsharp\nSML# unknown for x86_64-pc-linux-gnu with LLVM 13.0.1\n# \"hello world\";\nval it = \"hello world\" : string\n# 5+9;\nval it = 14 : int\n#\n```\nVery nice (needs the `rlwrap` treatment). But I could not compile a hello world program. It said `unbound variable: print`! Can you believe it? Looks like I have to do something similar to SML/NJ's `.cm` files. I need a `hello.smi` that says `_require \"basis.smi\"`.\n\nNope, still no:\n\n```bash\n ➜ smlsharp hello.sml -o hello\ncommand failed at status 256: gcc -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now /usr/lib/smlsharp/runtime/main.o /tmp/tmp.2q9kvI/000009.o /tmp/tmp.2q9kvI/000013.a /usr/lib/smlsharp/runtime/libsmlsharp.a -lrt -ldl -lm -lgmp -lmyth -lpthread  -o hello\nlto1: fatal error: bytecode stream in file ‘/usr/lib/smlsharp/runtime/main.o’ generated with LTO version 11.2 instead of the expected 11.3\ncompilation terminated.\nlto-wrapper: fatal error: gcc returned 1 exit status\ncompilation terminated.\n/usr/bin/ld: error: lto-wrapper failed\ncollect2: error: ld returned 1 exit status\n```\n\nThere must be a way to turn off \"link time optimization\" in `gcc` by default. Anyway, moving on...\n\n#### Moscow ML\n\nI was able to find and install a [`.deb` file](https://ppa.launchpadcontent.net/kflarsen/mosml/ubuntu/pool/main/m/mosml/mosml_2.10.2-0ubuntu0_amd64.deb) from January 2021, even though the [PPA repository](https://launchpad.net/~kflarsen/+archive/ubuntu/mosml) does not support Ubuntu 22.04. That was relatively easy. Not available on Fedora or Arch. You'd have to install from [source](https://github.com/kfl/mosml).\n\nIt has a REPL! (You should do the `rlwrap` trick with this one too.) Nice.\n\n```sml\n➜ mosml\nMoscow ML version 2.10\nEnter `quit();' to quit.\n- 2+2;\n\u003e val it = 4 : int\n- List.map (fn x =\u003e x*x) [1,2,3];        \n\u003e val it = [1, 4, 9] : int list\n- quit();\n```\n\n#### MLkit\n\nAnother compiler and toolkit (no REPL). Extremely easy to install on Linux and MacOS. [Get the binary release](https://github.com/melsman/mlkit/releases/tag/v4.7.2), unzip, do `sudo make install`. It just copies the binaries to your `/usr/local/bin/mlkit` directory.\n\n#### AliceML\n\nQuite difficult to install. Has to be compiled [from source](https://github.com/aliceml/aliceml). The instructions *almost* worked... compilation failed after 12 minutes. It seems to fail regarding [Gecode](https://www.gecode.org/). But it seems to have built most of the binaries already, just not all of them. *Edit:* the developer [responded within hours](https://github.com/aliceml/aliceml/issues/22) and fixed the issue.\n\nIs that a REPL? We use the `alice` command (once again, it needs the `rlwrap` treatment):\n\n```sml\n ➜ alice\nAlice 1.4 (\"Kraftwerk 'Equaliser' Album\") mastered 2022/11/15\n### loaded signature from x-alice:/lib/system/Print\n### loaded signature from x-alice:/lib/tools/Inspector\n### loaded signature from x-alice:/lib/distribution/Remote\n- 2+2;\nval it : int = 4\n- List.map (fn x =\u003e x*x) [1, 2, 3];\nval it : int list = [1, 4, 9]\n- \nThat's all, said Humpty Dumpty. Good-bye.\n```\n\nVery funny :laughing: Reference to Kraftwerk is also super cool. However we probably cannot use it for this book due to [its known limitations](https://www.ps.uni-saarland.de/alice/manual/limitations.html) which include the `OS.FileSys, OS.IO, Unix` and `StreamIO` structures and functors heavily used by the book.\n\n#### Standard ML Package Manager\n\nWhat? [That's awesome!](https://github.com/diku-dk/smlpkg) So I can just install libraries easy peasy? Extremely easy to install on Linux and MacOS: [Get the binary release](https://github.com/diku-dk/smlpkg/releases/tag/v0.1.5), unzip, then `sudo make install` and it just copies the binary to `/usr/local/bin`.\n\nOkay, there are 19 packages listed, I'm gonna install all of them! See what happens! Looking at one of them:\n\n\u003e This library is set up to work well with the SML package manager [smlpkg](https://github.com/diku-dk/smlpkg).  To use the package, in the root of your project directory, execute the command:\n\u003e\n\u003e ```bash\n\u003e $ smlpkg add github.com/diku-dk/sml-aplparse\n\u003e ```\n\nAh, OK, I get it now. It's not \"global\" installation. Project-based instead. That's fine and still useful. Maybe I'll turn this repository into a package and release it at some point!\n\n#### What I have\n\nI have successfully installed: `smlnj, poly, ocaml, mlton, mlkit, smlsharp, smlpkg, alice` on Ubuntu, but I'll stick to SML/NJ.\n\n#### Parts that might not work\n\nThe parts of the book that are SML/NJ specific might not work on other ML implementations. The Compilation Manager (and the `.cm` files) to create executables, for example. As I mentioned above, you can try to use alternate methods (such as `polyc` or `mlton` or `smlsharp`) but you'll have to figure out those on your own. I think Poly/ML is the most promising alternative but you have to get over the hurdles of installing the SML/NJ library. Poly/ML site says that it supports 100% of Standard ML as defined in the 1997 definition. Huh... turns out SML/NJ [deviates from the definition](http://mlton.org/SMLNJDeviations) but Poly/ML and MLton do not. Interesting! So much for \"Standard\" ML :laughing: But some of the deviations actually make sense and seem useful. \n\n## Changes since 2001\n\nSince the book is 22 years old (now in 2023), the Compilation Manager (`CM` from now on) of Standard ML (`sml` from now on) has changed quite a bit.\n\nThe reference you need is the revised [Compilation Manager book](https://www.smlnj.org/doc/CM/new.pdf). It says the new `CM` is in effect since version 110.20, and I'm using `sml` version 110.79 on Ubuntu 22.04.\n\nYou'll see some of the changes required below, as I go through the book.\n\n## Chapter 2: Hello World\n\n### Hello world program: `hw.sml` and `hw.cm`\n\nOn page 30, I had to change the `hw.cm` file from:\n\n```sml\ngroup is\n    hw.sml\n```\n\nto:\n\n```sml\ngroup is\n    hw.sml\n    $/basis.cm\n```\n\notherwise the library structures `SMLofNJ` and `OS.Process` would not be loaded.\n\nOn page 31 for the run script, I had to change the `smlbin` directory from:\n\n```bash\nsmlbin=/src/smlnj/current/bin\n```\n\nto:\n\n```bash\nsmlbin=/usr/lib/smlnj/bin\n```\n\nAlso, the command to compile is also different. The book says to use:\n\n```bash\n\u003e CM_ROOT=hw.cm sml\nStandard ML of New Jersey, Version 110.0.7, September 28, 2000\n- CM.make();\n```\n\nbut this doesn't work. Instead I used:\n\n```bash\n\u003e sml\nStandard ML of New Jersey v110.79 [built: Sat Oct 26 12:27:04 2019]\n- CM.make \"hw.cm\";\n```\n\nAlternatively, the `CM` book showed me an even simpler way to do the same thing, by using the binary executable command `ml-build` provided by the SML/NJ installation:\n\n```bash\nml-build hw.cm Main.main hw\n```\n\nTo run the run-script the book says:\n\n```bash\n\u003e hw\nhello world\n```\n\nbut I had to use:\n\n```bash\n\u003e ./hw\nHello world\n```\n\nTurns out it's also possible to use [`heap2exec`](https://www.smlnj.org/doc/heap2exec/index.html) to create a binary executable. I'll give this a try. Apparently it's not included in the default installation, unless I install from source and change `config/targets`.\n\n### The `echo` program\n\nSimilar changes as the `hw.sml` and `hw.cm` above.\n\n### Finite State Machine for counting words\n\nThe `C` code has a semicolon missing on page 44, at:\n\n```c\nif (!c)\n{\n    count++ // semicolon missing here!\n    goto eod;\n}\n```\n\nThe `sml` code has a typo too (or maybe the language changed since then?), on page 45, right at the start:\n\n```sml\nand word_count text = (* \"and\" should be \"fun\" instead *)\n```\n\nThe same typo is repeated on page 46.\n\n### The `getopt` programs\n\n#### Mostly functional: `getopt1.sml`\n\nThe `and` typos keep on happening so many times on page 51, I'm beginning to think they are not typos, but are meant to be placed inside a bigger wrapper function for mutual recursion? Anyway, I had to change them all to `fun`.\n\nThere is a `polyEqual` warning here at `n = name`:\n\n```sml\nfun find_option opts name : (string option) option =\n    case List.find (fn (n, v) =\u003e n = name) opts of\n        NONE =\u003e NONE\n    |   SOME (n, v) =\u003e SOME v\n```\n\nWell we can't have that.\n\nIt can be fixed by adding a type annotation to `n` like this: `fn (n: string, v) =\u003e ...`\n\nThe book very stubbornly refuses to put any type annotations anywhere, so I'm going around and placing them myself. I think type annotations are especially useful in these complicated imperative programs, to understand what's happening. The book explains in long prose what the code does, but the code itself does not make it very clear to me.\n\nThe `polyEqual` warning can also be avoided by providing a type annotation for `name` instead:\n\n```sml\nfun find_option(opts: Option list)(name: string): (string option) option =\n```\n\n#### Using a hash table\n\nAnother `polyEqual` warning on page 55:\n\n```sml\nstructure STRT_key =\nstruct\n    type hash_key = string\n    val hashVal = HashString.hashString\n    fun sameKey (s1, s2) = (s1 = s2) (* here! *)\nend\n```\n\nAgain let's provide type annotations to avoid it:\n\n```sml\nstructure STRT_key =\nstruct\n    type hash_key = string\n    val hashVal = HashString.hashString\n    fun sameKey (s1: hash_key, s2: hash_key) = (s1 = s2)\nend\n```\n\nInterestingly I cannot find the structures and signatures the author is talking about in the [SML manpages](https://smlfamily.github.io/Basis/manpages.html). Specifically the `signature HASH_KEY`, `signature MONO_HASH_TABLE` and `functor HashTableFn`. The author says to see Chapter 5 for what he calls \"SML Utility libraries\". In chapter 5 he says these libraries are under-documented. Considering the book was written in 2001 and the manpages seem to be from 2000, this seems accurate. They *are* under-documented. As in, they don't exist in the manpages. But, the code loads into `sml` just fine, so they *do* exist in my SML installation... *somewhere*.\n\nSo I followed what the author says to look into the SML installation. I looked into the folder `/usr/lib/smlnj/lib/`. There are many folders here that could hold the files `hash-table-fn.sml` and `hash-string.sml` he's talking about. These files simply don't exist, so they were either renamed, or moved into another file. Here are some candidates:\n\n```bash\n/usr/lib/smlnj/lib/smlnj/smlnj-lib/.cm/x86-unix/smlnj-lib.cm\n/usr/lib/smlnj/lib/SMLNJ-LIB/Util/.cm/x86-unix/smlnj-lib.cm\n```\n\nLet's consult the `CM` book again. Second-to-last chapter, \"Available libraries\" says:\n\n| name             | description                            | installed | loaded |\n| ---------------- | -------------------------------------- | --------- | ------ |\n| `$/smlnj-lib.cm` | SML/NJ general-purpose utility library | always    | auto   |\n\nSo my suspicion is that the hash stuff is inside this.\n\n***Mystery solved!*** I was looking at the wrong manpages. `sml` has *two* websites for documentation: one [old](https://www.smlnj.org/doc/) and one [new](https://smlfamily.github.io/Basis/manpages.html). The old website often links to the new website, but for *some libraries* the new documentation is missing. So I was able to find all of `signature MONO_HASH_TABLE`, `structure HashString`, `signature HASH_KEY` and `functor HashTableFn` [here](https://www.smlnj.org/doc/smlnj-lib/Util/smlnj-lib.html). Apparently this `smlnj-lib` stuff \"contains library and utility functions that are not part of the standard Standard.\" So there is an SML Basis Library, and there is an SML/NJ Library (which comes pre-installed with SML/NJ and MLton).\n\nIt's actually documented well and properly.\n\nAt this point the author says: \"The type constraint on the table value settles the type of the table\nimmediately to save the compiler and the reader having to figure it out.\" :laughing: Could have used that earlier! But I get it.\n\n#### `getopt` with a hash table: `getopt2.sml`\n\nI cannot figure out the return types of the functions in `getopt2.sml`, specifically the helpers inside `parse_cmdline`. I'll have to come back to that. (Turns out it's just `string list`.)\n\nAgain I have to change the `.cm` file. The book says:\n\n```sml\ngroup is\n    getopt2.sml\n    /src/smlnj/current/lib/smlnj-lib.cm\n```\n\nbut I have to do:\n\n```sml\ngroup is\n    getopt2.sml\n\n    $/basis.cm\n    $/smlnj-lib.cm\n```\n\nNow `sml` and then `CM.make \"getopt2.cm\";` work perfectly (the book doesn't do this):\n\n```bash\n\u003e sml\nStandard ML of New Jersey v110.79 [built: Sat Oct 26 12:27:04 2019]\nCM.make \"getopt2.cm\";\n[autoloading]\n[library $smlnj/cm/cm.cm is stable]\n[library $smlnj/internal/cm-sig-lib.cm is stable]\n[library $/pgraph.cm is stable]\n[library $smlnj/internal/srcpath-lib.cm is stable]\n[library $SMLNJ-BASIS/basis.cm is stable]\n[library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable]\n[autoloading done]\n[scanning getopt2.cm]\n[parsing (getopt2.cm):getopt2.sml]\n[creating directory .cm/SKEL]\n[library $SMLNJ-LIB/Util/smlnj-lib.cm is stable]\n[compiling (getopt2.cm):getopt2.sml]\n[creating directory .cm/GUID]\n[creating directory .cm/x86-unix]\n[code: 7434, data: 814, env: 1306 bytes]\n```\n\nNotice `[library $SMLNJ-LIB/Util/smlnj-lib.cm is stable]` so that's where it was.\n\nOnce again, alternatively I could use\n\n```bash\nml-build getopt2.cm Main.main getopt2\n```\n\nSo we got our executable. We'll write another launch script like before (the book does not). We can reuse the script for `echo`. Literally copy-paste it into a file named `getopt2`. Now we can run it:\n\n```bash\n./getopt2 -h -v -width 10\nThe option 'height' is missing.\nUsage: [-h] [-v|-verbose] [-width width] [-height height] files\n\n./getopt2 -h -v -width 10 -height 10\nsome helpful blurb\n\n./getopt2 -v -width 10 -height 10 hello\nThe files are hello.\nThe width is 10.\nThe height is 10.\n```\n\nIt's working properly! :confetti_ball: :tada: :partying_face:\n\n#### The Deluxe `getopt`: `getopt3.sml`\n\nNow we are using the `GetOpt` structure from the SML/NJ Utility Library: https://www.smlnj.org/doc/smlnj-lib/Util/str-GetOpt.html\n\n\u003e The `GetOpt` structure provides command-line argument processing similar to the GNU **getopt** library.\n\nCode on page 63 starts with a structure that's supposed to conform to the `OPTION` signature:\n\n```sml\nstructure Option: OPTION = \nstruct\n    structure G = GetOpt (* alias *)\n\n    (* This represents an option found on the command line. *)\n    datatype Option = Verbose | Help | Width of string | Height of string\n    ...\n```\n\nThis doesn't make sense: why should this `Option` above have things like `SOME` or `NONE` or `map` or `isSome` or `valOf`?\n\nThis requires implementing all the functions in the `OPTION` signature, and the book does not do that; so `sml` rightfully gives close to 20 errors like this:\n\n```bash\n...\ngetopt3.sml:1.2-23.4 Error: unmatched value specification: getOpt\ngetopt3.sml:1.2-23.4 Error: unmatched value specification: isSome\ngetopt3.sml:1.2-23.4 Error: unmatched value specification: valOf\n...\n```\n\nAs far as I can tell, this is a straight-up error in the book. So I had to remove the `OPTION` declaration there:\n\n```sml\nstructure Option = (* removed : OPTION here *)\nstruct\n    ...\n```\n\nThe code on page 64 starts by opening a list with `[` but the list is never closed with `]`:\n\n```sml\nval options: (Option G.opt_descr) list = [\n    {\n    \tshort = \"v\", long = [\"verbose\"],\n    \tdesc = NoArg Verbose,\n    \thelp = \"Select verbose output\"\n    },\n    {\n    \tshort = \"\", long = [\"width\"],\n    \tdesc = ReqArg Width \"width\",\n    \thelp = \"The width in pixels\"\n    },\n(* no closing brackets ] *)\n```\n\nAnother typo? The author says: *\"Here is my code for **part** of the option description list.\"* So I think we're supposed to write the rest? So I have to write entries for `height`, `help`. I think.\n\n```sml\nval options: (Option G.opt_descr) list = [\n    {\n        short = \"v\", long = [\"verbose\"],\n        desc = NoArg Verbose,\n        help = \"Select verbose output\"\n    },\n    {\n        short = \"\", long = [\"width\"],\n        desc = ReqArg Width \"width\",\n        help = \"The width in pixels\"\n    },\n    {\n        short = \"\", long = [\"height\"],\n        desc = ReqArg Height \"height\",\n        help = \"The height in pixels\"\n    },\n    {\n        short = \"h\", long = [\"help\"],\n        desc = NoArg Help,\n        help = \"Some helpful blurb\"\n    },\n]\n```\n\nAlso from the indentation used in the book it's not clear if all of these are still inside the `structure Option`, I have to assume they are. \n\nOn page 66 in the `fun getWidth()` function, there is a closed parenthesis at the end, but it's not opened anywhere. The function body has a `let` block, so it should end with `end` but it ends with `)` instead:\n\n```sml\nfun getWidth(): string option =\n    let\n        val opt_width = List.find (fn Width _ =\u003e true | _ =\u003e false) (!opt_tbl)\n    in\n        case opt_width of\n            NONE =\u003e NONE\n        |   SOME(Width w) =\u003e SOME w\n        |   _ =\u003e raise Fail \"Option,getWidth\"\n    ) (* \u003c- this should be \"end\" instead *)\n```\n\nOn page 67 the dereference operator `!` is explained. Here is an example from the REPL:\n\n```sml\nval z: int list ref = ref [1, 2, 3];\nval z = ref [1,2,3] : int list ref\nz := [4]; (* mutation! *)\nval it = () : unit\nz;\nval it = ref [4] : int list ref (* mutated value *)\n!z;\nval it = [4] : int list\n```\n\nSo `!` gets rid of the `ref`.\n\nNow I'm definitely lost. The `require_option` function uses the `Usage` exception that was defined in `getopt1.sml` and `getopt2.sml` but not in `getopt3.sml`. So I think the intention is that I am really supposed to keep up with what's going on and fill in the blanks by myself; just following the code fragments in the book is not enough. I go back to `getopt2.sml` and copy the exception definition.\n\nWait, now that I looked back on `getopt2.sml` I noticed that this code was inside `structure Main`. So... where should I end `structure Option`? The code in the book never closed that structure, so I assumed all the code so far should go into it.\n\nNormally `parse_cmdline` was inside `structure Main` but `parseCmdLine` mutates the options table `opt_tbl` which, I think, should be inside the `structure Option`. What should I do? On page 67 the `main` function uses `Option.parseCmdLine` which means `main` is outside `structure Option`; so I think I just separate `main` into the `Main` structure and that's it. \n\nI am going to make a change, and move `fun show_stuff()` outside of `fun main`. This requires passing the `files, width` and `height` as parameters to `show_stuff()`:\n\n```sml\nfun show_stuff(files: string list, width: string, height: string): unit = (\n    print \"The files are\";\n    app (fn f =\u003e (print \" \"; print f)) files;\n    print \".\\n\";\n\n    if hasVerbose() then\n        print(concat[\n            \"The width is \", width, \".\\n\",\n            \"The height is \", height, \".\\n\"\n        ])\n    else ()\n)\n```\n\nUgh... there are a lot more unexplained things. For example `Option.getHeight` is missing. The author never wrote it. We'll just copy/paste `getWidth` and slightly change it I guess:\n\n```sml\nfun getHeight(): string option =\n    let\n        val opt_height = List.find (fn Height _ =\u003e true | _ =\u003e false) (!opt_tbl)\n    in\n        case opt_height of\n            NONE =\u003e NONE\n        |   SOME(Height h) =\u003e SOME h\n        |   _ =\u003e raise Fail \"Option,getHeight\"\n    end\n```\n\nI'm *NOT* having fun so far. We also need to change usages of `require_option` to `Option.require_option`, `Usage` to `Option.Usage`... or we could just `open Option` at the beginning of `structure Main` and be done with it!\n\nThere is one more undefined function in the code named `Option.usage()` and I don't know what it's supposed to do. I guess it displays the usage message in case the user inputs the wrong syntax. Let's go back to the beginning of the subsection where the author showed us the GNU default usage message:\n\n```bash\nUsage: getopt\n  -v -verbose\t\tSelect verbose output\n  -width=width\t\tThe width in pixels\n  -height=height\tThe height in pixels\n  -h -help\t\t\tShow this message.\n```\n\nI think `Option.usage()` is supposed to return this as a `string` to be passed to `Option.toErr` as the `msg` parameter. OK! First approach:\n\n```sml\nfun usage(): string =\n    concat[\n        \"Usage: getOpt\\n\",\n        \"-v -verbose\\tSelect verbose output\\n\",\n        \"-width=width\\tThe width in pixels\\n\",\n        \"-height=height\\tThe height in pixels\\n\",\n        \"-h -help\\tShow this message.\\n\"\n    ]\n```\n\nThe [GetOpt manpage](https://www.smlnj.org/doc/smlnj-lib/Util/str-GetOpt.html) has a really nice example with a much better approach to this `usage` by using the built-in `usageInfo` function; it's wrapped in a `usage()` function that actually *prints* directly; and it actually uses the text messages we put inside the `options` list, instead of my manual typing above. That's much better: just copy-paste it from the manpage, and slightly change it.\n\n```sml\nfun usage () = print (G.usageInfo{header = \"usage:\", options = options})\n```\n\nSee, if the book explained all this, it would have been nice. Oh well. At least I'm having *a little bit* of fun and feeling smart.\n\nI ended up putting this inside `structure Main` instead. Oh and I also changed the name of the structure from `Option` to `Common` like before (see further below for the reason). The overall result ended up quite different than the book. Well, there is no way for me to know that actually! Because a lot of the code is missing! So I think it ended up quite different than *what was probably intended:*\n\n```sml\nstructure Common =\nstruct\n    structure G = GetOpt (* alias *)\n\n    exception Usage of string\n\n    (* This represents an option found on the command line. *)\n    datatype Option = Verbose | Help | Width of string | Height of string\n\n    fun NoArg(opt: Option): Option G.arg_descr =\n        G.NoArg (fn () =\u003e opt)\n\n    fun ReqArg(opt: string -\u003e Option)(descr: string): Option G.arg_descr =\n        G.ReqArg (opt, descr)\n\n    val options: (Option G.opt_descr) list = [\n        {\n            short = \"v\", long = [\"verbose\"],\n            desc = NoArg Verbose,\n            help = \"Select verbose output\"\n        },\n        {\n            short = \"\", long = [\"width\"],\n            desc = ReqArg Width \"width\",\n            help = \"The width in pixels\"\n        },\n        {\n            short = \"\", long = [\"height\"],\n            desc = ReqArg Height \"height\",\n            help = \"The height in pixels\"\n        },\n        {\n            short = \"h\", long = [\"help\"],\n            desc = NoArg Help,\n            help = \"Some helpful blurb\"\n        }\n    ]\n\n    fun toErr(msg: string): unit = TextIO.output(TextIO.stdErr, msg)\n\n    val opt_tbl: (Option list) ref = ref [] (* mutable reference *)\n\n    fun parseCmdLine(argv: string list): string list =\n        let\n            val (opts, files) = G.getOpt {\n                argOrder = G.RequireOrder,\n                options = options,\n                errFn = toErr\n            } argv\n        in\n            opt_tbl := opts; (* mutation! now opt_tbl is: ref opts *)\n            files\n        end\n\n    (* The ! operator is dereference, like * in C. It gets rid of \"ref\".\n        opt_tbl is: ref [...]: list ref\n        !opt_tbl is: [...]: list           *)\n    fun hasVerbose(): bool = List.exists (fn opt =\u003e opt = Verbose) (!opt_tbl)\n\n    fun hasHelp(): bool = List.exists (fn opt =\u003e opt = Help) (!opt_tbl)\n\n    fun getWidth(): string option =\n    let\n        val opt_width = List.find (fn Width _ =\u003e true | _ =\u003e false) (!opt_tbl)\n    in\n        case opt_width of\n            NONE =\u003e NONE\n        |   SOME(Width w) =\u003e SOME w\n        |   _ =\u003e raise Fail \"Option,getWidth\"\n    end\n\n    fun getHeight(): string option =\n    let\n        val opt_height = List.find (fn Height _ =\u003e true | _ =\u003e false) (!opt_tbl)\n    in\n        case opt_height of\n            NONE =\u003e NONE\n        |   SOME(Height h) =\u003e SOME h\n        |   _ =\u003e raise Fail \"Option,getHeight\"\n    end\n\n    fun require_option(func: unit -\u003e string option)(name: string): string =\n        case func() of\n            NONE =\u003e raise Usage (concat[\"The option '\", name, \"' is missing.\"])\n        |   SOME v =\u003e v\nend\n\nstructure Main =\nstruct\n    open Common\n    \n    fun usage () = print (G.usageInfo{header = \"usage:\", options = options})\n\n    fun show_stuff(files: string list, width: string, height: string): unit = (\n        print \"The files are\";\n        app (fn f =\u003e (print \" \"; print f)) files;\n        print \".\\n\";\n\n        if hasVerbose() then\n            print(concat[\n                \"The width is \", width, \".\\n\",\n                \"The height is \", height, \".\\n\"\n            ])\n        else ()\n    )\n\n    fun main(arg0: string, argv: string list): OS.Process.status =\n    let\n        val files = parseCmdLine argv\n        val width = require_option getWidth \"width\"\n        val height = require_option getHeight \"height\"\n    in\n        if hasHelp() then\n            usage()\n        else\n            show_stuff(files, width, height);\n            OS.Process.success\n    end\n    handle Usage msg =\u003e (\n        toErr msg;\n        toErr \"\\n\";\n        usage();\n        toErr \"\\n\";\n        OS.Process.failure\n    )\n\n    val _ = SMLofNJ.exportFn(\"getopt3\", main)\nend\n```\n\n(One thing to note is that, `TextIO.output` is actually \"inherited\" from  `StreamIO` structure.)\n\nThere are no more explanations on what to do; so I have to create the `getopt3.cm` file and the `getopt3` launch script myself. Copy-pasting the `.cm` file (change `getopt2.sml` to `getopt3.sml`) and the script for`getopt2` works. Don't forget to `chmod +x getopt3` the script.\n\nRemember we will be using the neat convenient shortcut:\n\n```bash\nml-build getopt3.cm Main.main getopt3\n```\n\nAh here we go... the author's weird module naming bites us in the ass:\n\n```bash\n[scanning (52181-export.cm):getopt3.cm]\n52181-export.cm:1.45-1.55 Error: structure Option imported from $SMLNJ-BASIS/(basis.cm):basis-common.cm@310798(option.sml) and also from (52181-export.cm):(getopt3.cm):getopt3.sml\n[parsing (52181-export.cm):52181-export.sml]\nCompilation failed.\n```\n\nYep, for some reason he keeps using names that clash with SML/NJ library, such as `Option`. I'm changing it to `Common` instead.\n\nFinally it's working, I can't believe it:\n\n```bash\n./getopt3 --help\nThe option 'width' is missing.\nusage:\n  -v  --verbose        Select verbose output\n      --width=width    The width in pixels\n      --height=height  The height in pixels\n  -h  --help           Some helpful blurb\n./getopt3 -v --width=1 --height=1 file1 file2 file3\nThe files are file1 file2 file3.\nThe width is 1.\nThe height is 1.\n```\n\nOverall opinion of the `getopt` section: the `GetOpt` library seems nice, would be nicer to stick to it even closer; and I don't get the point of the mutable reference that holds the options in a list. Other than the side effects like printing, imperative programming here seems unnecessary. So I think it's done just for the sake of demonstration. In the book's defense, we first did it with minimal imperativity / library usage in `getopt1.sml` so that's fair.\n\n## Chapter 3: The Basis Library\n\n### Preliminaries\n\n#### The value restriction: a fun digression\n\nInteresting: the book says\n\n\u003e The value polymorphism restriction is not something you will likely encounter. For the purposes of this book it mainly requires you to ensure that imperative variables using the `ref` type are restricted to contain a specific declared type.\n\nWhy is this interesting? I took [Programming Languages Part A](https://www.coursera.org/learn/programming-languages) by Dan Grossman from U. of Washington, on Coursera, to learn SML. This was mentioned mostly as an optional thing: \"value restriction\":\n\n```sml\nsml\nStandard ML of New Jersey v110.79 [built: Sat Oct 26 12:27:04 2019]\nval x = ref [];\nstdIn:1.6-1.16 Warning: type vars not generalized because of\n   value restriction are instantiated to dummy types (X1,X2,...)\nval x = ref [] : ?.X1 list ref\n- \n```\n\nHere's what the course said:\n\n\u003e As described so far in this section, the ML type system is unsound, meaning that it would accept programs that when run could have values of the wrong types, such as putting an int where we expect a string. The  problem results from a combination of polymorphic types and mutable references, and the fix is a special  restriction to the type system called the value restriction. This is an example program that demonstrates the problem:\n```sml\nval r = ref NONE\n(* ’a option ref *)\nval _ = r := SOME \"hi\" (* instantiate ’a with string *)\nval i = 1 + valOf(!r) (* instantiate ’a with int *)\n```\n\u003e Straightforward use of the rules for type checking/inference would accept this program even though we should not – we end up trying to add `1` to `\"hi\"`. Yet everything seems to type-check given the types for the functions/operators `ref (’a -\u003e ’a ref)`, `:= (’a ref * ’a -\u003e unit)`, and `! (’a ref -\u003e ’a)`. To restore soundness, we need a stricter type system that does not let this program type-check. The choice ML made is to prevent the first line from having a polymorphic type. Therefore, the second and third lines will not type-check because they will not be able to instantiate an `’a` with `string` or `int`.\n\nWhy is this called *value* restriction? Because it prevents values from being polymorphic. As a result, some `fun`s that can be written as `val`s get ruled out by this. The course said:\n\n\u003e Once you have learned currying and partial application, you might try to use it to create a *polymorphic\n\u003e function*. Unfortunately, certain uses, such as these, do not work in ML:\n\n```sml\nval mapSome = List.map SOME (* not OK *)\n```\n\n\u003e Given what we have learned so far, there is no reason why this should not work, especially since all these\n\u003e functions do work:\n\n```sml\nfun mapSome xs = List.map SOME xs       (* OK *)\nval mapSome = fn xs =\u003e List.map SOME xs (* OK *)\n```\n\nThese ruled-out \"function-values\" are called *polymorphic functions types*, and interestingly, [they have been only recently added to Scala 3](https://docs.scala-lang.org/scala3/reference/new-types/polymorphic-function-types.html):\n\n\u003e A polymorphic function type is a function type which accepts type parameters. For example:\n\n```scala\n// A polymorphic method:\ndef foo[A](xs: List[A]): List[A] = xs.reverse\n\n// A polymorphic function value:\nval bar: [A] =\u003e List[A] =\u003e List[A]\n//       ^^^^^^^^^^^^^^^^^^^^^^^^^\n//       a polymorphic function type\n       = [A] =\u003e (xs: List[A]) =\u003e foo[A](xs)\n```\n\nDoes this mean Scala's type system is unsound now because of this? ([Probably not, at least in theory.](https://www.semanticscholar.org/paper/Type-soundness-for-dependent-object-types-(DOT)-Rompf-Amin/bcb109f4b7ba02a4172c012a39a578135d612cc8) Implementation is a different matter though.) The unsoundness comes when polymorphic values mix with mutation (`var` in Scala). \n\nHowever, Scala requires these polymorphic types to be *function types*; trying to replicate the above `sml` issue does not work. Here I'm trying to create a reference (`var`) to an optional value with a type parameter in it, and setting it to `None` initially (so it leaves the possibility of being `Option[String]` as well as `Option[Int]`):\n\n```scala\nscala\u003e var spam: [T] =\u003e Option[T] = [T] =\u003e None\n-- Error: ------------------------------------------------------------------------------------------------------------\n1 |var spam: [T] =\u003e Option[T] = [T] =\u003e None\n  |              ^\n  |              Implementation restriction: polymorphic function types must have a value parameter\n-- Error: ------------------------------------------------------------------------------------------------------------\n1 |var spam: [T] =\u003e Option[T] = [T] =\u003e None\n  |                                 ^\n  |                           Implementation restriction: polymorphic function literals must have a value parameter\n```\n\nYep, so that's ruled out already. Even if we *could* do this, we'd still have to instantiate the type parameter `T` with `Int` and `String` separately, so the situation where we're trying to add `1` to `\"hi\"` would not type-check. Moreover, Scala does not generalize type variables like `sml` does, instead it infers them to the \"bottom type\" `Nothing`:\n\n```sml\n ➜ sml\nStandard ML of New Jersey v110.79 [built: Sat Oct 26 12:27:04 2019]\nval x = [];\nval x = [] : 'a list (* left generalized as long as it's not a ref *)\n```\n\n```scala\nscala\u003e val y = List()\nval y: List[Nothing] = List() // not generalized, inferred to Nothing instead\n```\n\nEven if we made `y` into a `var` and then tried to reassign an integer list to it, Scala won't allow it:\n\n```scala\nscala\u003e var y = List()\nvar y: List[Nothing] = List()\nscala\u003e y = List(1)\n-- [E007] Type Mismatch Error: ---------------------------------------------------------------------------------------\n1 |y = List(1)\n  |         ^\n  |         Found:    (1 : Int)\n  |         Required: Nothing\n  |\n  | longer explanation available when compiling with `-explain`\n1 error found\n```\n\nThe `sml` equivalent of this happens only for the `ref` types:\n\n```sml\nval y = ref [];\nstdIn:2.5-2.15 Warning: type vars not generalized because of\n   value restriction are instantiated to dummy types (X1,X2,...)\nval y = ref [] : ?.X1 list ref\ny := [1];\nstdIn:3.1-3.9 Error: operator and operand don't agree [overload conflict]\n  operator domain: ?.X1 list ref * ?.X1 list\n  operand:         ?.X1 list ref * [int ty] list\n  in expression:\n    y := 1 :: nil\n```\n\nScala requires concrete type variables to be provided for values. We cannot leave them generalized (because Scala has subtyping which `sml` does not, so Scala has to worry about [variance](https://en.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science))):\n\n```scala\nscala\u003e val z = List[T]()\n-- [E006] Not Found Error: -------------------------------------------------------------------------------------------\n1 |val z = List[T]()\n  |             ^\n  |             Not found: type T\n  |\n  | longer explanation available when compiling with `-explain`\n1 error found\n```\n\nThis is a general rule not limited to references (`var`). So I tried my best to break the Scala type system, but I cannot find any problem.\n\nAnyway... interesting things happening in the functional world over the last 20 years huh? **Digression over.**\n\nBut now that weird little side thing that was mentioned in the course becomes clearer, because that course used no imperative programming, and now we're doing a lot of imperative programming in the book. ***We have to always instantiate `ref` types to some concrete type.***\n\n### General\n\nSome stuff from the [General structure](https://smlfamily.github.io/Basis/general.html) mentioned here. `exnName` and `exnMessage`, `!` for dereferencing and `:=` for assignment, `o` for function composition, and the `before` function, which I didn't know about!\n\nThe `Fatal` and `InternalError` exceptions the author mentions are not found anywhere in the `sml` library. So I manually defined them in the big code example:\n\n```sml\nexception Fatal\nexception InternalError of string\n```\n\nI had to supply a lot of other stuff too:\n\n```sml\nfun process(args: string list): unit = () (* dummy, for demonstration *)\nfun toErr(msg: string): unit = TextIO.output(TextIO.stdErr, msg)\nfun f(): unit = () (* dummy *)\n```\n\nJust keep in mind, to make the code type-check and compile, you have to provide missing stuff yourself.\n\n### Option\n\nNot much here: `valOf, isSome`. There is one interesting thing: the mention of *equality types.*\n\nWe cannot compare an optional function value to, say, `NONE` by using plain equality `=`:\n\n```sml\n- val z = SOME (fn x =\u003e x);\nval z = SOME fn : ('a -\u003e 'a) option\n- z = NONE;\nstdIn:25.1-25.9 Error: operator and operand\ndon’t agree [equality type required]\n    operator domain: ''Z * ''Z\n    operand:         ('Y -\u003e 'Y) option * 'X option\nin expression:\nz = NONE\n- Option.isSome z;\nval it = true : bool\n```\n\nNotice `equality type required`. What's an equality type? It's similar to how you can use typeclass derivation like `deriving Eq` in Haskell or `derives CanEqual` in Scala 3. In `sml` we do this by using double single-quotes before a type annotation like `''a`.  It still doesn't help us with the above problem though. We could define instead:\n\n```sml\nval z: (''a -\u003e ''a) option = SOME(fn x =\u003e x);\n```\n\nbut `NONE` itself is `'X option`, which is not an equality type; so we cannot change that!\n\n### Bool\n\nNot much here, just `Bool.toString` and `Bool.fromString`.\n\n### Text\n\nLots of stuff here. Many structures: [`String`](https://smlfamily.github.io/Basis/string.html), [`Char`](https://smlfamily.github.io/Basis/char.html) [`Text`](https://smlfamily.github.io/Basis/text.html), [`Substring`](https://smlfamily.github.io/Basis/substring.html) [`StringCvt`](https://smlfamily.github.io/Basis/string-cvt.html) and so on. Interesting bits about performance. It seems that `Substring.all` that is mentioned in the book has been renamed to `Substring.full` in my version of `sml`.\n\nNow we get to the good hard stuff: streams. We'll be processing text data in the form of streams, and use transformer functions (a bit reminiscent of monad transformers from Haskell). Character streams can come from a `string` via `StringCvt.scanString` or from a file via `TextIO.scanStream`. The code is quite complicated.\n\nThe author uses a multi-line string literal that I haven't seen before in `sml`: \n\n```sml\nval text = \"\\\n            \\ 123 true 23.4        \\n\\\n            \\ -1 false -1.3e3      \\n\\\n            \\\"\n```\n\nand the VS Code plug-in cannot do syntax coloring correctly. The rest of the code that comes after this string gets recognized as a string (probably a bug in the plug-in, with the regular expressions that define its comments?). But the code type-checks fine. You can run it, to see that it's correctly reporting the `int`, the `bool` and the `real` inside the text:\n\n```sml\nmain(\"\", []);\n123 true 23.4\nval it = 0 : OS.Process.status\n- \n```\n\nI reported it to the [extension developer](https://github.com/vrjuliao/sml-vscode-extension/issues/18).\n\n#### Bytes\n\nNot much here. [Some conversion functions.](https://smlfamily.github.io/Basis/byte.html) Will be used for reading from, and writing to, TCP/IP sockets.\n\n### Integer\n\nInteresting bit about `sml`'s garbage collector and the weird 31-bit integers. LSB (least significant bit) is used by the GB to tell whether it's a pointer or not. So `int` is actually 31-bit. Some useful conversion functions `toInt, fromInt, toString, fromString, toLarge, fromLarge` etc. \n\nAnd typical arithmetical operators. Note that `sml` does not auto-convert `int`s to `float`s when you are doing division. The `/` is reserved for `real`s whereas for `int`s you have to use the `div` operator. I'm familiar with this from Haskell. Implicit conversions can break the type-system, so it's a reasonable trade-off and only a small inconvenience.\n\n```sml\n- 5/2;\nstdIn:11.1-11.4 Error: operator and operand don't agree [overload conflict]\n  operator domain: real * real\n  operand:         [int ty] * [int ty]\n  in expression:\n    5 / 2\n- 5 div 2;\nval it = 2 : int\n- \n```\n\nHey, we can use hexadecimal with `x` in `sml`! Who knew? Also `word` and hexadecimal `word` with the `w`:\n\n```sml\n- val i: int = 0x123;\nval i = 291 : int\n- val j: word = 0w12345;\nval j = 0wx3039 : word\n- val k: word = 0wx12345;\nval k = 0wx12345 : word\n- \n```\n\nVS Code plug-in again fails to correctly syntax-color these. [Reported!](https://github.com/vrjuliao/sml-vscode-extension/issues/17)\n\n### Real\n\nIn `sml` the `real` type is not an equality type! So we cannot use the usual `=` or `\u003c\u003e` for comparing `real`s. The documentation has [some explanation about this](https://smlfamily.github.io/Basis/real.html#Real:STR:SPEC):\n\n\u003e Deciding if `real` should be an equality type, and if so,  what should equality mean, was also problematic. IEEE specifies that the sign of zeros be ignored in comparisons, and that equality evaluate to  false if either argument is `NaN`. These constraints are disturbing to the SML programmer. The former implies that `0 = ~0` is true while `r/0 = r/~0` is false. The latter implies such anomalies as `r = r` is false, or that, for a ref cell `rr`, we could have `rr = rr` but not have `!rr = !rr`. We accepted the unsigned comparison of zeros, but felt that the  reflexive property of equality, structural equality, and the equivalence of `\u003c\u003e` and `not o =` ought to be preserved. Additional complications led to the decision to not have `real` be an equality type.\n\nThe good news is that we can use `Real.==` and `Real.!=`; moreover the top-level `\u003c, \u003e, \u003c=, \u003e=` are still available.\n\n### List\n\nJust normal list stuff. The `@` operator concatenates two lists, but it has to copy the first argument, so it's not efficient:\n\n```sml\n- [1,2] @ [3,4];\nval it = [1,2,3,4] : int list\n```\n\nThe `cons` operator `::` is efficient (but it only adds 1 element to the left of the list). There are some useful functions that the book recommends:\n\n```sml\nval revAppend : 'a list * 'a list -\u003e 'a list\nval app : ('a -\u003e unit) -\u003e 'a list -\u003e unit\nval tabulate : int * (int -\u003e 'a) -\u003e 'a list\n```\n\nThe [`ListPair` structure](https://smlfamily.github.io/Basis/list-pair.html) is super useful, I used it a lot in my [pattern matching exercise in Programming Languages Part A](https://github.com/spamegg1/reviews/blob/master/courses/ProgLangA/week4/hw3/).\n\n### Array and Vector\n\nNot much to say here. There are specialized types such as `CharArray` and `CharVector`.\n\n### Portable IO API\n\nThis is the most interesting part. Lots of `IO` interfaces here: `PRIM_IO, OS_PRIM_IO, TEXT_IO, BIN_IO, TEST_STREAM_IO, IMPERATIVE_IO, STREAM_IO` with various relationships between them. And these are *just the signatures.* There are way more *structures:* `BinPrimIO, TextPrimIO, PosixBinPrimIO, Posix.IO, PosixText.IO, PosixTextPrimIO,...` The list goes on.\n\nThe documentation calls `StreamIO` a FUNCTOR. Now we're talking. A functor is not a monad but close enough.\n\nThe book says: \n\n\u003e Input streams are handled in a lazy functional manner. This means that streams are read from only upon demand (as you would expect) and the read returns a stream updated at a new position. So you can read from the same stream value multiple times and it always returns the same data from the same position. Output streams are imperative. Each write will append new data to the output stream.\n\nI bet this will bite us in the ass somewhere down the line, requiring a workaround. Otherwise very useful.\n\nAs it became a theme already, the code examples have a lot of missing stuff, undefined functions which lead to `unbound variable or constructor` errors in `sml`, so they have to be supplied by hand.\n\nOn page 87 there is code with a `let` block that's empty (also `count` is missing):\n\n```sml\nfun main(arg0: string, argv: string list): OS.Process.status =\nlet\n    (* there is nothing here *)\nin\n    case argv of\n        [] =\u003e count TextIO.stdIn \"\"\n    |   (file :: _) =\u003e\n        let\n            val strm = TextIO.openIn file\n        in\n            (count strm file) handle x =\u003e (TextIO.closeIn strm; raise x);\n            TextIO.closeIn strm\n        end;\n    OS.Process.success\nend\n```\n\nThis looks strange but it is equivalent to putting parentheses around the `case` block. If we want to do a bunch of side effects followed by a return value, like:\n\n```sml\na();\nb();\nc();\nOS.Process.success\n```\n\nthen we can either use the \"empty let block\" approach like above, or put parentheses around it. For example:\n\n```sml\nfun main(arg0: string, argv: string list): OS.Process.status = (\n    a();\n    b();\n    c();\n    OS.Process.success\n)\n```\n\nOtherwise it won't type-check. Also, if we want the block to be followed by a `handle` block, we need the parentheses (or a `let-in-end` block).\n\nThe `count` function is shown later in the book. Now we can add it above `main`. There must have been another change in the language API (or another typo in the book), because on page 89 this part:\n\n```sml\ncase TextIO.inputLine strm of\n    \"\" =\u003e (nchars, nwords, nlines)\n```\n\n gives an error:\n\n```sml\nError: case object and rules don't agree [tycon mismatch]\n  rule domain: string\n  object: string option\n  in expression:\n    (case (TextIO.inputLine strm)\n      of \"\" =\u003e (nchars,nwords,nlines)\n       | line =\u003e let val \u003cpat\u003e = \u003cexp\u003e in read (\u003cexp\u003e,\u003cexp\u003e,\u003cexp\u003e) end)\n```\n\nSo `TextIO.inputLine strm` returns an `option` in the current `sml`. We can fix the issue like so:\n\n```sml\ncase TextIO.inputLine strm of\n            NONE =\u003e (nchars, nwords, nlines) (* no lines left in file *)\n        |   SOME(line) =\u003e ...\n```\n\nand everything type-checks! :white_check_mark:\n\nI don't like the book's style of nesting functions, so I take them out (and pass the parameters of the top-function). I also remove the author's paranoid redundant parentheses everywhere. This is much nicer:\n\n```sml\nfun read(strm: TextIO.instream)(nchars: int, nwords: int, nlines: int)\n: int * int * int =\n    (* This ensures the line ends with a \\n unless we are at eof. *)\n    case TextIO.inputLine strm of\n        NONE =\u003e (nchars, nwords, nlines)\n    |   SOME(line) =\u003e\n        let\n            val words = String.tokens Char.isSpace line\n        in\n            read strm (nchars + size line, nwords + length words, nlines + 1)\n        end\n\nfun count(strm: TextIO.instream)(file: string): unit =\nlet\n    val (nchars, nwords, nlines) = read strm (0, 0, 0)\nin\n    print(concat[\n        Int.toString nlines, \" \",\n        Int.toString nwords, \" \",\n        Int.toString nchars, \" \",\n        file,\n        \"\\n\"\n    ])\nend\n```\n\nWe can load it up on the REPL and use it. It reports the correct number of lines, words, characters:\n\n```sml\nuse \"portable-io.sml\";\n[opening portable-io.sml]\nval toErr = fn : string -\u003e unit\nval read = fn : TextIO.instream -\u003e int * int * int -\u003e int * int * int\nval count = fn : TextIO.instream -\u003e string -\u003e unit\nval main = fn : string * string list -\u003e OS.Process.status\nval it = () : unit\nmain(\"\", [\"int.sml\"]);\n4 20 87 int.sml\nval it = 0 : OS.Process.status\n```\n\nAgain, if we wanted, we could write a `.cm` file and build it with `ml-build` and launch it with a script, so we'd have our own version of Unix utility `wc` written in `sml`! Neat. It's possible the author is implicitly assuming that we are doing that. I'll make the executables when we get to the more serious stuff.\n\n### Portable OS API\n\nAnother very important part of the library! We have `OS.FileSys, OS.Path, OS.Process` (which we've been using a lot already), the [Time and Date stuff](https://smlfamily.github.io/Basis/time.html) and finally the operating-system-dependent [`Unix` structure](https://smlfamily.github.io/Basis/unix.html).\n\nWe have another long code example with things that need to be fixed. At the bottom of page 91\n\n```sml\ncase FS.readDir strm of\n    \"\" =\u003e rev files (* done *)\n    | f =\u003e\n```\n\nwe have a problem similar to the above; these cases need to be `NONE` and `SOME(...)`. (So in this case [`FS.readDir`](https://smlfamily.github.io/Basis/os-file-sys.html#SIG:OS_FILE_SYS.readDir:VAL) has changed.)\n\nThere are many nested functions calling each other, and recursion, here, not to mention the total lack of type signatures to figure things out, it was tough taking them out and making it still work. The author really has a messy C-ish style.\n\nAnyway, it all type-checks and compiles! Now for the big showdown:\n\n```sml\nuse \"filesys.sml\";\n[opening filesys.sml]\nstructure FS : OS_FILE_SYS\nstructure OP : OS_PATH\nval toErr = fn : string -\u003e unit\nexception Error of string\nval open_dir = fn : string -\u003e ?.OS_FileSys.dirstream\nval get_files = fn\n  : string -\u003e ?.OS_FileSys.dirstream -\u003e string list -\u003e string list\nval show_wx = fn : string -\u003e unit\nval scan_dir = fn : string -\u003e unit\nval main = fn : string * string list -\u003e OS.Process.status\nval it = () : unit\nmain(\"\", []);\nval it = 0 : OS.Process.status\n```\n\nOh no... it's supposed to print the files in the current directory. Well that fizzled out quickly didn't it? The author says:\n\n\u003e The `show_wx` function prints the file name if it is writable and executable.\n\nTurns out none of the files satisfy this check (here `FS` is short for `OS.FileSys`):\n\n```sml\nif FS.access(file, [FS.A_WRITE, FS.A_EXEC]) then ...\n```\n\nLet's take a look at the \"access mode\" things:\n\n```sml\ndatatype access_mode = A_READ | A_WRITE | A_EXEC\n```\n\nSo I'll just change the code to use `FS.A_READ` only:\n\n```sml\n- main(\"\", []);\n./text.sml\n./filesys.sml\n./general.sml\n./option.sml\n./bool.sml\n./portable-io.sml\n./int.sml\nval it = 0 : OS.Process.status\n- \n```\n\nYAY! :confetti_ball: :tada: :partying_face: It also works in nested directories, prints all the files with their paths.\n\n### POSIX API\n\nLots and lots of stuff here! There are so many structures, types and functions it's dizzying: `Posix.FileSys, Posix.SysDB, Posix.ProcEnv`, and we are also using `SysWord, Date, Position, StringCvt`. \n\n#### Posix.FileSys\n\nWe can change file permissions with `Posix.FileSys.chmod`. The [`Posix.FileSys` signature](https://smlfamily.github.io/Basis/posix-file-sys.html) has weird sub-structure names, like `S` or `O` or `ST`. \n\nThere is code for a `stat` function that is a simplified version of the Unix `stat` utility. Again, missing code and functions are introduced later. I am seeing the use of `local` for the first time in `sml` here (here `FS` is `Posix.FileSys`):\n\n```sml\nlocal\n    val type_preds: ((stat -\u003e bool) * string) list = [\n        (FS.ST.isDir, \"Directory\"),\n        (FS.ST.isChr, \"Char Device\"),\n        (FS.ST.isBlk, \"Block Device\"),\n        (FS.ST.isReg, \"Regular File\"),\n        (FS.ST.isFIFO, \"FIFO\"),\n        (FS.ST.isLink, \"Symbolic Link\"),\n        (FS.ST.isSock, \"Socket\")\n    ]\nin\n    fun filetypeToString(st: stat) =\n    let\n        val pred = List.find (fn (pr, _) =\u003e pr st) type_preds\n    in\n        case pred of\n            SOME (_, name) =\u003e name\n        |   NONE =\u003e \"Unknown\"\n    end\nend\n```\n\nThe book says:\n\n\u003e I’ve put the list of predicates within a local block so that it is private to `filetypeToString` without being inside it. This way the list isn’t built every time that the function is called, which is wasteful. This doesn’t matter on this small program but it very well might in other programs.\n\nThat's useful! This is used many times later.\n\nOn page 101 there is a typo `pun` which should be `fun`. Maybe it's a pun intended?\n\nLots of helper functions for `stat`. The code as presented in the book type-checks, but it's impossible to understand what's happening. To add my type annotations and make it all type-check, I had to track down a lot of types:\n\n```sml\nPosix.FileSys.S.flags (* an alias for mode *)\nPosix.FileSys.S.mode \nPosix.FileSys.ST.stat\nPosix.FileSys.uid\nPosix.FileSys.gid\nPosix.ProcEnv.uid\nPosix.ProcEnv.gid\nPosix.SysDB.uid\nPosix.SysDB.gid\n```\n\nThe code is displayed first, and all these structures are explained in later sections, so it's quite confusing.\n\nWell, it *does* work:\n\n```sml\nmain(\"\", [\"myfile\"]);\nFile: myfile\nSize: 123\nType: Regular File\nMode: 644/rw-r--r--\nUid: 1000/spam\nGid: 1000/spam\nDevice: 3,1\nInode: 1838772\nLinks: 1\nAccess: Wed Nov 16 23:47:57 2022\nModify: Thu Nov 17 13:34:51 2022\nChange: Thu Nov 17 13:34:51 2022\nval it = 0 : OS.Process.status\n```\n\n:confetti_ball::tada: :partying_face: \n\n#### Posix.IO\n\nNow we get to the more monadic stuff.\n\nIt looks like `PosixTextPrimIO` and `PosixBinPrimIO` do not exist anymore, they were [\"lifted\" to `Posix.IO`](https://smlnj.sourceforge.net/NEWS/110.46-README.html) instead. Apparently this happened in 2004! So here are the changes:\n\n| Old (book, 2001)           | New (2004-now)         |\n| -------------------------- | ---------------------- |\n| `PosixBinPrimIO.mkReader`  | `PosixIO.mkBinReader`  |\n| `PosixBinPrimIO.mkWriter`  | `PosixIO.mkBinWriter`  |\n| `PosixTextPrimIO.mkWriter` | `PosixIO.mkTextReader` |\n| `PosixTextPrimIO.mkWriter` | `PosixIO.mkTextWriter` |\n\nVery sensible. If we make these changes, the code on page 102 type-checks. \n\nThe top part on page 103 still does not work. This code is supposed to be directly taken from the `Unix` structure; the code must have changed a lot since then, since I could not find it in the source code or the documentation. The `openOutFD` and `openInFD` functions don't exist anymore; they were probably renamed to `openOut` and `openIn`.\n\n`TextIO.StreamIO.mkInstream` does not accept `NONE` as its second argument, though. The type-checker wants a `TextIO.StreamIO.vector` which, in this case, [is a `string`!](https://smlfamily.github.io/Basis/stream-io.html#SIG:STREAM_IO.vector:TY) The correct version is now\n\n```sml\nfun openInFD(name: string, fd: Posix.IO.file_desc): TextIO.instream =\n    TextIO.mkInstream(\n        TextIO.StreamIO.mkInstream(fdReader(name, fd), \"\")\n    )\n```\n\nNext we see the code for [`Unix.executeInEnv`](https://smlfamily.github.io/Basis/unix.html#SIG:UNIX.executeInEnv:VAL). I'm guessing this is also too old and must have changed since 2001. It's quite long, 1.5 pages! A few typos again on page 104 (a parenthesis that gets opened after `startChild` but never gets closed). `Substring.all` needs to be changed to `Substring.full` again (happened before). \n\nThere is surprisingly little explanation for this long code. Again the structures that are used are explained in *later sections.* I have to figure out where `protect` is coming from in\n\n```sml\nfun startChild () =\n(\n    case protect P.fork() of\n        SOME pid =\u003e pid      (* parent *)\n```\n\nbecause the compiler cannot find it. `protect` does not exist anywhere in the documentation. It must have been removed or renamed to something else. `Posix.Process.fork()` already returns a `pid option` so I'm just gonna remove `protect` here.\n\nAt the end of the code, in the returned value, there is a reference to `PROC`:\n\n```sml\nPROC {\n    pid = pid,\n    ins = ins,\n    outs = outs\n}\n```\n\nbut I have no idea what it is. I thought it's a reference to an earlier alias `structure PROC = Posix.ProcEnv` but I cannot find anything [in that structure](https://smlfamily.github.io/Basis/posix-proc-env.html) that fits. `executeInEnv` is supposed to return a `('a, 'b) proc` type, which I don't know how to create. There were big overhauls to the `Unix` structure in 2004. Can't figure it out, moving on. *It's a mystery!* \n\n**EDIT:** Finally found the source code the book is talking about. It's inside [`system.tgz`](http://smlnj.cs.uchicago.edu/dist/working/110.57/system.tgz) source file, under `system/Basis/Implementation/Unix/unix.sml`. Looks very similar to the code in the book. The functions were renamed to `openTextOutFD, openBinOutFD, openTextInFD, openBinInFD` and I was right about the second argument being `\"\"`. \n\nThe mystery of the `protect`function is also solved: it's defined there, not part of the library. \n\n```sml\nfun protect(f: 'a -\u003e 'b)(x: 'a): 'b = \nlet\n    val _ = Signals.maskSignals Signals.MASKALL\n    val y = (f x) handle ex =\u003e (\n        Signals.unmaskSignals Signals.MASKALL; \n        raise ex\n    )\nin\n    Signals.unmaskSignals Signals.MASKALL; y\nend\n```\n\nThe `PROC` mystery is also solved. It's a custom datatype constructor defined in that file, with a bunch of mutable references in it: \n\n```sml\ndatatype 'stream stream =\n    UNOPENED of PIO.file_desc\n|   OPENED of { stream: 'stream, close: unit -\u003e unit }\n\ndatatype proc_status = DEAD of OS.Process.status | ALIVE of P.pid\n\ndatatype ('instream, 'outstream) proc = PROC of { \n    base: string,\n    instream: 'instream stream ref,\n    outstream: 'outstream stream ref,\n    status: proc_status ref \n}\n```\n\nThis is the 2005 version, the earliest I could find. So it was after the major 2004 overhaul of the `Unix` structure. Here `PROC` has more arguments than the book version. Versions before 2005 don't have the source code available, they are not posted on the [old releases](https://www.smlnj.org/old-news.html).\n\n*I really wish the book explained it better.* At least say: \"this is only part of the source code. with lots of stuff missing.\" I hate unrunnable code examples. I guess I am expected to look into the source code, while the snippets in the book only show little bits of it.\n\nI invented my own solution:\n\n```sml\ndatatype ('a, 'b) proc = PROC of {\n    pid: P.pid,\n    ins: 'a,\n    outs: 'b\n}\n```\n\nIt type-checks with the code *as presented in the book*, but still getting `polyEqual` warnings even though `Posix.IO.file_desc` is supposed to be an `eqtype`. Anyway, good enough! Time to move on.\n\n#### Posix.Process, Posix.ProcEnv, Posix.SysDB\n\nThese are used above in `Posix.FileSys` and `Posix.IO`, here only given a single paragraph.\n\n#### Posix.TTY\n\nThe author uses shorthands without telling: `TTY` refers to `Posix.TTY`. The code example here gives a lot of `unbound variable` errors:\n\n```sml\nposix-tty.sml:7.16-7.27 Error: unbound variable or constructor: getattr in path TTY.getattr\nposix-tty.sml:15.18-15.31 Error: unbound variable or constructor: getospeed in path TTY.getospeed\nposix-tty.sml:14.18-14.31 Error: unbound variable or constructor: getispeed in path TTY.getispeed\nposix-tty.sml:18.5-18.16 Error: unbound variable or constructor: setattr in path TTY.setattr\n```\n\nSome of these functions are inside substructures of [`Posix.TTY`](https://smlfamily.github.io/Basis/posix-tty.html) named `TC` and `CF`. So the API must have changed. The author was aware of this back in 2001:\n\n\u003e Note that at the time of writing this, the Basis library documentation for `Posix.TTY` doesn’t match SML/NJ version 110.0.7. In version 110.0.7 there is no internal structure called `Posix.TTY.CF`. Its contents appear directly in `Posix.TTY`. Similarly these functions which should be in the `Posix.TTY.TC` structure appear directly in `Posix.TTY: getattr, setattr, sendbreak, drain, flush`, and `flow`.\n\nMaking the necessary changes makes the code type-check.\n\n## Chapter 4: The SML/NJ Extensions\n\nSo here we are finally! I'd like to know if it's possible to use these in Poly/ML or MLton. I'm guessing not. I'll give it a try at least on the `poly` REPL later.\n\n### The Unsafe API\n\nThis is found under [\"Special features of SML/NJ\"](https://www.smlnj.org/doc/SMLofNJ/pages/unsafe.html) which is separate from both the [Basis Library](https://smlfamily.github.io/Basis/manpages.html) and the [`smlnj-lib` man pages](https://www.smlnj.org/doc/smlnj-lib/index.html). Weird.\n\nThere are the types `Unsafe.CharVector`, `Unsafe.Word8Vector`, `Unsafe.CharArray`, `Unsafe.Word8Array`, `Unsafe.Real64Array` which are faster.\n\n#### Memory Representation\n\nThe author laments that the library lacks functionality to manipulate memory representation of data. Then there is a 2 page long code example to estimate the size of objects in memory using `Unsafe.Object`. The [doc page for the Object structure](https://www.smlnj.org/doc/SMLofNJ/pages/object.html) is well-hidden and not linked to by other pages, I had to guess the URL. Here we can see all the things the author is using. Most importantly:\n\n```sml\ndatatype representation\n  = Unboxed\n  | Real\n  | Pair\n  | Record\n  | PolyArray\n  | ByteVector\n  | ByteArray\n  | RealArray\n  | Susp\n  | WeakPtr\n```\n\nSo `Susp` and `WeakPtr` have size 2. According to the book, a record, pair, or real-array has to be converted to a vector with `toTuple`, then folded over with `Vector.foldl`. The documentation seems to agree:\n\n\u003e `toTuple ob`\n\u003e\n\u003e If ob is a record, tuple, vector, or real-array, get a vector of its fields\n\nBut the REPL disagrees:\n\n```sml\nsmlnj-memory-rep.sml:22.36-22.74 Error: operator and operand don't agree [tycon mismatch]\n  operator domain: ?.Unsafe.object vector\n  operand:         ?.Unsafe.object list\n  in expression:\n    ((Vector.foldl foldFun) 1) (O.toTuple obj)\n```\n\nSo the API must have changed, and the documentation was not updated (I guess). It's a `list` not a `vector`. We need to use `List.foldl` instead. Then everything is fine.\n\nDue to the author's style of nesting functions inside functions inside functions, these are repeated twice:\n\n```sml\nfun sz obj = if O.boxed obj\tthen 1 + (obj_size obj)\telse 1\nval f = (fn (obj, s) =\u003e s + (sz obj))\n```\n\nIs there some other reason? I factored them out. Anyway, there is unavoidable mutual recursion here with the `and` keyword. I was able to move all the functions out of the nests. It works!\n\n```sml\n- main(\"\", []);\nSize of integer = 1 32-bit words\nSize of real = 3 32-bit words\nSize of string = 2 32-bit words\nSize of pair = 5 32-bit words\nSize of record = 9 32-bit words\nval it = 0 : OS.Process.status\n```\n\n#### The C Interface, and Miscellaneous Unsafe Operations\n\nNot much to say [here](https://www.smlnj.org/doc/SMLofNJ/pages/cinterface.html).\n\n### Signals\n\nThe author shows us how to print all the available signals on our systems. There is a basic example of an interrupt handler. We get our first glimpse into [Continuations](https://www.smlnj.org/doc/SMLofNJ/pages/cont.html), which I know nothing about but heard of quite a bit.\n\nWow! I found a compiler bug! :rofl: It's like winning the jackpot.\n\n```sml\nError: Compiler bug: Unify: unifyTy: arg ty lists wrong length\n\nuncaught exception Error\n  raised at: ../compiler/Basics/errormsg/errormsg.sml:52.14-52.19\n             ../compiler/Elaborator/types/unify.sml:310.13\n             ../compiler/Elaborator/types/unify.sml:310.13\n             ../compiler/Elaborator/types/unify.sml:310.13\n             ../compiler/Elaborator/types/unify.sml:310.13\n             ../compiler/Elaborator/types/typecheck.sml:625.12\n             ../compiler/Basics/stats/stats.sml:198.40\n             ../compiler/Elaborator/elaborate/elabmod.sml:1741.65\n             ../compiler/Elaborator/elaborate/elabmod.sml:1745.59\n             ../compiler/Basics/stats/stats.sml:198.40\n             ../compiler/TopLevel/interact/evalloop.sml:44.55\n             ../compiler/TopLevel/interact/evalloop.sml:292.17-292.20\n```\n\n[Reported!](https://github.com/smlnj/legacy/issues/262) I'm having fun!\n\n### The SMLofNJ API\n\nThese are the \"special features\" listed [here](https://www.smlnj.org/doc/features.html). Now personally I know nothing about all these low-level specialized execution options; I'm a fairly high-level programmer, I'm not very good at the low-level stuff.\n\n#### Call/cc\n\nIt's [here](https://www.smlnj.org/doc/SMLofNJ/pages/cont.html). Later explained in Chapter 6, Concurrency.\n\n#### The Interval Timer\n\nThe author gives a code example of this [crazy feature](https://www.smlnj.org/doc/SMLofNJ/pages/interval-timer.html):\n\n\u003e By returning a different continuation you can have your program switch to different code on each clock tick.\n\nWhat kind of insane person would want to do that?\n\n#### Garbage Collection Control\n\nIt's [here](https://www.smlnj.org/doc/SMLofNJ/pages/gc.html#GC:SIG:SPEC). We can manually trigger GC, and turn on/off GC messages. Nice.\n\n#### Execution Time Profiling\n\nIt's supposed to be [here](https://www.smlnj.org/doc/Compiler/pages/profile.html) but the link is dead. The book says:\n\u003eYou access execution time profiling through the `Compiler.Profile` structure, which is separate from the `SMLofNJ` structure.\n\nThere is [this](https://www.smlnj.org/doc/SMLofNJ/pages/prof-control.html#PROF_CONTROL:SIG:SPEC) which the book also explains:\n\n\u003e However the profiling uses the low-level control functions in `SMLofNJ.Internals.ProfControl`.\n\nThe documentation agrees:\n\n\u003e The [ProfControl](https://www.smlnj.org/doc/SMLofNJ/pages/internals.html#SIG:INTERNALS.ProfControl:STR:SPEC) structure is for internal use by the compiler. Programmers who want execution profiles of their programs should use the [Compiler.Profile](http://cm.bell-labs.com/cm/cs/what/smlnj/doc/Compiler/pages/compiler.html#SIG:VISCOMP.Profile:STR:SPEC) structure.  \n\nThat Bell Labs link is also dead. I found one old [link](https://flint.cs.yale.edu/cs430/smlnj/doc/Compiler/pages/profile.html).\n\nThe book says to use\n\n```sml\n\u003e CM_ROOT=profile.cm sml\nStandard ML of New Jersey, Version 110.0.7, September 28, 2000\n- Compiler.Profile.setProfMode true;\n- CM.make();\n```\n\nbut that doesn't work. The `Compiler.Profile` structure does not exist as far as I can tell. As such, the code example in the book does not work either. \n\nIt's possible that getting access to `Compiler.Profile` requires some special thing, like compiling from source with certain options, or something. The default Ubuntu installation does not have it, if this is the case.\n\nAnother possibility is that the structures were moved or renamed. For example, the [old link](https://flint.cs.yale.edu/cs430/smlnj/doc/Compiler/pages/) shows that all the structures like `Profile, PrettyPrint, Control.Print, Control.MC` *used to be* `Compiler.Profile, Compiler.PrettyPrint, Compiler.Control.Print, Compiler.Control.MC` but they were changed.\n\nSo I'm not sure if `Compiler.Profile` was entirely removed, or renamed to something else that I cannot find.\n\n*Mystery solved:* It definitely is about the installation. I have to compile from source, and enable the auto-loading of \"the Visible Compiler (old style)\". This is in the `config/preloads` file in the [installation archive](https://smlnj.org/dist/working/110.99.3/config.tgz).\n\n#### Operating System Information\n\nIt's [here](https://www.smlnj.org/doc/SMLofNJ/pages/sysinfo.html).\n\n#### Lazy Suspensions\n\nIt's [here](https://www.smlnj.org/doc/SMLofNJ/pages/susp.html).\n\n#### Weak Pointers\n\nIt's [here](https://www.smlnj.org/doc/SMLofNJ/pages/weak.html).\n\n#### The Exception History List\n\nAll you need to know is the function `SMLofNJ.exnHistory`.\n\n### The Socket API\n\n#### The Generic Socket Types\n\n#### The Specific Socket Types\n\n#### Socket Addresses\n\n#### A Simple TCP Client\n\n#### A Simple TCP Server\n\n#### Servers with Multiple Connections\n\n## Chapter 5: The Utility Libraries\n\n### Data Structures\n\n#### Trees, Maps and Sets\n\n#### Hash Tables\n\n#### Vectors and Arrays\n\n#### Queues and Fifos\n\n#### Property Lists\n\n### Algorithms\n\n#### Sorting and Searching\n\n#### Formatted Strings\n\n#### Misc. Utilities\n\n### Regular Expressions\n\n#### The Pieces of the Library\n\n#### Basic Matching\n\n#### Matching with a Back-End\n\n### Other Utilities\n\n#### Parsing HTML\n\n#### INet\n\n#### Pretty-Printing\n\n#### Reactive\n\n#### Unix\n\n## Chapter 6: Concurrency\n\n### Continuations\n\n### Coroutines\n\n### The Concurrent ML Model\n\n#### CML Threads\n\n#### CML Channels\n\n#### CML Events\n\n#### Synchronous Variables\n\n#### Mailboxes\n\n### A Counter Object\n\n### Some Tips on Using CML\n\n### Getting the Counter's Value\n\n### Getting the Value Through an Event\n\n### Getting the Value with a Time-Out\n\n### More on Time-Outs\n\n### Semaphores\n\n### Semaphores via Synchronous Variables\n\n## Chapter 7: Under the Hood\n\n### Memory Management\n\n#### Garbage Collection Basics\n\n#### Multi-Generational GC\n\n#### Run-Time Arguments for the GC\n\n#### Heap Object Layout\n\n### Performance\n\n#### Basic SML/NJ Performance\n\n#### Memory Performance\n\n#### CML Channel Communication and Scheduling\n\n#### Spawning Threads for Time-outs\n\n#### Behaviour of Timeout Events\n\n## Chapter 8: The Swerve Web Server\n\n### Introduction\n\n### The HTTP Protocol\n\n### The Resource Store\n\n### Server Configuration\n\n### The Architecture of the Server\n\n### Building and Testing the Server\n\n## Chapter 9: The Swerve Detailed Design\n\n### Introduction\n\n### The Organization of the Code\n\n### The Main Layer\n\n### The Server Layer\n\n### The Store Layer\n\n### The IETF Layer\n\n### The Config Layer\n\n### The Common Layer\n\n## Chapter 10: Conclusion\n\nThe author gives quite the praise to FP (functional programming):\n\n\u003eFunctional languages certainly scale to the size of real-world projects by nature. The productivity improvement from a functional language allows a programmer to tackle much larger projects. I estimate at least a 10 to 1 improvement in productivity over the C language just from a reduction in the number of lines of code needed. There is a large productivity boost from the absence of memory management bugs and the like that plague C programs. Further the strong static type system often means that sizable pieces of SML code just works first time.\n\nThese are very obvious points, but I didn't expect a [lifelong C programmer](http://www.anthony-shipman.id.au/resume/resume.html) from the old days to say this. The author is surely open-minded and objective about it! In fact he was open-minded all the way back in 2000 when FP wasn't even known all that much (now it's getting a lot more popular). Usually old-school C programmers don't like FP.\n\n\u003e ...SML/NJ does perform well. Without much trouble it achieved around 2/3 the speed of Apache, which is written in C. Anecdotal evidence has it that C++ is about 1/2 the speed of C. ...This suggests that SML/NJ can certainly compete with C++ or Java.\n\nWow. I wonder how much things improved since 2001. Those other languages must have improved quite a lot too. MLton is definitely the fastest ML out there.\n\n\u003e The big impediment to wider use of SML in the real-world is support for features such as databases, graphics, encryption, native languages (Unicode), true concurrency etc.\n\nValid points. SMLsharp [focuses](https://smlsharp.github.io/en/about/features/) on the databases part, especially SQL; and the concurrency part. Poly/ML addresses [concurrency](https://www.polyml.org/documentation/Reference/Thread.xml). [AliceML](https://www.ps.uni-saarland.de/alice/) addresses SQL and GTK, and addresses concurrency with futures and laziness. There are also `sml` to `js` converters in many ML implementations, such as [MLkit](https://github.com/melsman/mlkit#smltojs---the-javascript-backend). As for encryption and Unicode, there are [Unicode](https://github.com/diku-dk/sml-unicode) and [SHA-256](https://github.com/diku-dk/sml-sha256) packages available through [`smlpkg`](https://github.com/diku-dk/smlpkg).\n\n\u003e The standard basis library is looking rather dated. It hasn’t progressed in years. A future release of SML/NJ will include a native C-language interface.\n\nIs this true in 2023? I think this is referring to the \"Foreign Function Interface\". Yes, it exists [now](https://www.smlnj.org/doc/SMLNJ-C/index.html). There is a library package `libmlnlffi-smlnj` ([No Longer Foreign Function Interface](http://www.jeffvaughan.net/docs/nlffi.pdf)) on Ubuntu, which installs under `/usr/lib/smlnj/lib/c` and a lot of documentation under `/usr/share/doc/libmlnlffi-smlnj`, and a binary executable `ml-nlffigen` that generates glue code between C and `sml`:\n\n```bash\n ➜ ccat /usr/share/doc/libmlnlffi-smlnj/README\nThis is the ML-NLFFI Library, the core of a new foreign-function\ninterface for SML/NJ.\n\nLibrary $c/c.cm provides:\n\n  - an encoding of the C type system in ML\n  - dynamic linking (an interface to dlopen/dlsym)\n  - ML/C string conversion routines\n\n  This is the (only) library to be used by user code.\n\nLibrary $c/c-int.cm (subdirectory \"internals\"):\n\n  - implements all of $c/c.cm\n  - implements low-level hooks to be used by ml-nlffigen-generated code\n\nLibrary $c/memory.cm (subdirectory \"memory\"):\n\n  - encapsulates low-level details related to raw memory access\n\n  User code should NOT directly refer to this library.\n```\n\nSimilar FFIs are also available in [Poly/ML](https://www.polyml.org/documentation/Reference/Foreign.xml) and [MLton](http://mlton.org/ForeignFunctionInterface). SML# [seems to have one too.](https://smlsharp.github.io/en/about/features/) So does [Moscow ML](https://mosml.org/). So does [MLkit](https://elsman.com/mlkit/). Very nice.\n\n\u003e ...Personally I consider Concurrent ML to be a \"killer\" feature of SML/NJ. ...It’s a shame that CML is not built-in to SML/NJ instead of being an add-on.\n\n[It is now!](https://www.smlnj.org/doc/FAQ/install.html#what) \n\n\u003e **What does the SML/NJ system consist of?** \n\u003e\n\u003e SML/NJ 110 provides a native code compiler that can be used interactively and can also build stand-alone applications, along with a programming environment made up of various tools (CM, ML-Lex, ML-Yacc, CML, eXene, etc.), and a collection of libraries (smlnj-lib).\n\nIt comes bundled with the standard `smlnj` installation. It's under `/usr/lib/smlnj/lib/cml` and `/usr/lib/smlnj/lib/cml-lib`. If it's missing, it can be installed through the `libcml-smlnj` package (at least on Ubuntu).\n\nAnyway.. the author's points are all addressed by the industrial-strength OCaml (or Scala, or Haskell...) Functional languages have come a long way!\n\n\u003e The OCaml system[OCaml] is an implementation of a language in the ML family. The language is different from Standard ML and a bit on the experimental side for my taste. However it has recently seen much active\n\u003e development in the area of infrastructure...\n\nYep, that was 2000-2001. Now OCaml is full-blown in 2023. Especially [on the OCaml home page](https://ocaml.org):\n\n\u003e OCaml 5.0 is beta! OCaml 5.0 is the next  major release of OCaml, which comes with support for shared-memory parallelism through domains and direct-style concurrency through algebraic effects!\n\nFinally:\n\n\u003e The bottom line is: yes you can nowadays develop real-world applications in functional languages, getting leverage off of their special features such as greater productivity and reliability. Try it out!\n\nThank you sir! Hats off to you for trying this 22 years ago. What an absolute madlad.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspamegg1%2Funix-sml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspamegg1%2Funix-sml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspamegg1%2Funix-sml/lists"}