Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/rwxrob/cmdtab


https://github.com/rwxrob/cmdtab

Last synced: 8 months ago
JSON representation

Awesome Lists containing this project

README 

          
# Go Tab Complete Commander (`cmdtab`)



![WIP](https://img.shields.io/badge/status-wip-red)

[![Go Report Card](https://goreportcard.com/badge/github.com/rwxrob/cmdtab)](https://goreportcard.com/report/github.com/rwxrob/cmdtab)

[![GoDoc](https://godoc.org/github.com/rwxrob/cmdtab?status.svg)](https://godoc.org/github.com/rwxrob/cmdtab)



*A commander for modern command-line human-computer interactions.*



![logo](cmdtab.png)



*Tab Complete Commander* is a lightweight commander package focused on

creating human-friendly terminal command-line interfaces composed of

modular subcommands that can be imported as packages with portable

completion and embedded, dynamic documentation.



## Installation



Normally you would simply `import "github.com/rwxrob/cmdtab"`.



## Advantages



* Build the command-line interface to your application completely

  independently from your core library.



* Automatic and extendible Bash tab completion built in.



* Modular, interchangeable, readable subcommands.



* Easiest possible command documentation with support for emphasis with

  simple Markdown formatting.



* Commands can be used standalone or as subcommands of a monolith (think

  BusyBox).



# Terminology



A ***command module*** is an importable Go module hosted in a Git

repository containing both an `init` function and a *standalone* command

package with a `main` function allowing it to be used by itself or

composed into another command as a subcommand.



When referring to ***standalone*** in this context we mean a *command

module* that has been compiled into a single command from its command

package with a `main` function. 



## Ongoing Project



Although Complete Commander is fully functional now it should still be

considered beta until the final [TODO](#TODO) items are complete. Most

of them are related to better default actions and output formatting.



## How It Works



The magic of *Complete Commander* comes from the clean separation of

subcommands into their own files and composing them into an internal

command index within the main executable program. This is accomplished

through simple, judicious use of the `init()` function in each file.

This approach incurs no more performance hit than would already be

required for any other such solution and is 100% concurrency safe.



By the time the command-line application's `main()` function is called

all it has to do is `cmdtab.Execute("mycmd")` to execute the top-level

command:



1. Detects and responds to programmable shell completion context

1. Optionally adds the `help`, `usage`, and `version` builtins

1. Delegates to any subcommands, or

1. Runs its own top-level method



Because of the loose coupling, and top-down design a full subcommand

node tree can easily be displayed and documented, indeed the builtin

commands do exactly that. When care is given to maintain this loose

coupling between subcommands a file can be modularly added or moved to

any other executable directory creating very clean sharing and

composition possibilities.



## Motivation



This package scratches several "personal itches" that have come up from

the needs of modern human-computer interaction clearly passing what is

currently available:



1. No commanders exist for writing simple commands with human-friendly subcommands

1. No commanders exist that know anything about tab completion

1. Tab completion is the simplest way to provide good help to the user

1. Getopt-style options are ancient, ridiculously bad HCI

1. More users are voicing their commands rather than typing them

1. More users are using keyboard input with their thumbs

1. Modern command-line programs need to be easily distributed

1. Distributing executables with separate documentation complicates understanding the program

1. Modern monolithic command approaches allow embedded documentation and better distribution

1. Operating system package managers are overkill for distributing most command executables

1. Documentation only needs to cover what the user is specifically interested in

1. Documentation can be provided dynamically when embedded within the command it documents



### Times Have Changed



The world has outgrown the original UNIX model for both command design

and documentation, presumably due to the resource constraints of the

day. These constraints forced shared library dependencies and commands

were designed to be as minimal as possible doing "only one thing really

well." Combining several actions into a single command (think `git

clone` or `go tool dist list`) or embedding full command documentation

into the executable would have been considered monstrously wasteful at

that time. The result was lots of small commands dependent on specific

shared libraries and a separate documentation system (`man` pages).



Today Go has revolutionized applications development and respectfully

embraced modern use of resources. Static linking, cross-compilation,

built in concurrency, reflection and documentation are all evidence of

how time have changed.  It follows then, that our command interface

designs and documentation approaches should equally modernize.



There is now plenty of room to compose several commands and subcommands

into a single monolithic executable. In fact, this has become idiomatic

for Go's own tool set. Distribution and portability are more important

than memory and storage size these days.



### Sometimes You Just Need Tab



Usually completion context is triggered simply by tapping tab once or

twice from the shell when typing in the command and its different

subcommand arguments. Tab completion is supported as a standard in most

all modern shells. Tab completion is often all that is needed, even

usage would be overkill. A user simply wants to remember what the

command name is.



### Keep the Docs with the Command



Clean, standardized, extensive documentation needs to travel with the

command it documents rather than being separate from it. There is no

telling what operating systems will emerge nor how (and if) they will

implement system documentation of any kind. Indeed, with the use of

modern web and git hosting documentation the need for any system-level

documentation system is sharply diminishing. Better to embed it with the

command itself when needed.



Embedded documentation has the ability to dynamically sense the

environment and context for its use thereby creating more useful,

specific help to the user. This means that rather than dumping every

single usage option into a massive man page separate from the command

(think `gpg`), such documentation can be broken up and displayed

programmatically by the command itself.



Most modern terminals support color (ANSI escapes) and UTF-8 (yes even

Windows) and regularly display more than 80 columns of text.



### Modern Human-Computer Interaction



Voice and human-friendly textual command-line interfaces are becoming

more and more popular as more humans interact with command line

interfaces in chat applications, messaging and more. This influence has

permeated to system command design and deprecated the design decision to

use traditional getopt-like switches and options, which cannot be voiced

or tab-completed easily. (Consider how difficult it is to type or voice

a simple dash into most text applications.)



Simplicity on the command-line has been an ignored requirement for some

time. Why? Because only amazing technical people are using the command

line? Perhaps. But that is a very bad reason. Interfaces need not be

overly complex or difficult to memorize just because most users are

technical.



The *Complete Commander* approach takes these HCI considerations very

seriously placing priority on how humans use modern command-line

interfaces and developing them accordingly. Human interaction is the

single most important design decision for any application, including one

that runs on the command line.



## Design Decisions



This is a summary of the design decisions made roughly in the order they

were made. It is provided in the hopes of addressing other design

concerns anyone reviewing this package might have before choosing to use

it.



* To this day command options plague developers and users by different

  ways of dealing with single or double dashes, the equals sign, single

  letter options and more. Most are also not friendly to the use of

  UTF-8 runes in the upper ranges.



* Even through `getopt` can be problematic (having had to code for it

  and around it for two decades and still has nightmares about `gpg`'s

  interface) `MapOpts` is available for those who insist on using them

  giving the command author the choice. Parsing traditional getopt-type

  options and switches was originally dropped since the goals of this

  project are to deprecate such designs in favor of modern HCI

  approaches to command-line user interfaces. But, choice prevails over

  opinion in this case. Still, please don't use `-f ` when the

  tab-completable `file ` could be used instead. Even a single

  dash `-` is nearly impossible to voice-to-text command-line interface.



* Semantic emphasis `*bold*`, `**italic**`, `***bold-italic***` are the

  only inline formatting options allowed. This seems prudent given the

  fundamental requirement for readability as well as the complication of

  other inline formatting and escapes.  These three inline formats have

  been supported by all system documentation formats from the beginning

  and play nicely with Go's back-ticked raw-strings. Users can change

  the colors of these on terminals that use term(cap|info)-aware man

  pages and less/more (i.e.  `LESS_TERMCAP_??` or `COMPCMD_??`) without

  any need to provide theming in the package itself.



* The decision to not use other Markdown for formatting was easy given

  the minimal three formats allowed. The decision to keep documentation

  100% compatible with CommonMark was a no-brainer decision. 



* Pager application (i.e. less, more) detection seems practical since

  most everyone would want such when help output exceeds the page height

  (when such can be determined). There is a TODO to add fall-back

  builtin pager support.



* Allowing more than 80 columns and never hard-wrapping lines allows the

  user to decide the preferred width of documentation.  Lines that are

  hard-wrapped are unreadable when the column count drops to less than

  that of the hard-wrapping. With TMUX small pane widths are common.

  (This is one thing Go documentation seriously failed to consider.)



* Internationalization was a big design consideration from the

  beginning. All code containing the English defaults is separate from

  the rest allowing build tools to generate language specific versions.

  The idea of putting all supported languages into monolithic files with

  locale templates was considered but quickly dismissed given the

  potential impact to executable size. Localized builds are a common

  industry practice. The English naming of the default builtin commands

  is at least as appropriate as these standard keywords are included in

  many other contexts `help`, `version`, and `usage` are very

  ubiquitous. If needed, these can be aliased easily by adding commands

  that encapsulate them with `Call()`.



* Using `structs` instead of interfaces makes more sense given the goals

  to enable quick and easy to read and write documentation embedded in

  the source.



* Including a fair amount of output formatting and printing code seemed

  appropriate given that one of the three main goals of the project was

  to produce consistent command output formatting. It had been suggested

  to put such into another package instead but this package is so small

  that ultimately turned out to be overkill. Besides, the formatting

  used by `cmdtab` is highly specific to making output look good on a

  terminal as it relates to command documentation.



* Rather than hard-code a dependency on Bash completion, every effort

  has been made to decouple completion from any specific shell

  completion API (despite the many references to Bash, which dominates

  currently). As long as any shell completion implementation sets an

  environment variable containing the full line to be completed package

  `cmdtab` will always be able to sense completion context and perform.

  This puts the completion logic safely embedded into the command that

  needs the completion and exposes as little dependency on shell

  completion methods as possible.



* Use of aliases (`d|del|delete`) allows accessibility even if tab

  completion is not (yet) supported on a particular platform.



* Inferring the main (top-level) command to use was considered to be the

  first argument (`os.Args[0]`) for some time but further research

  revealed it can never be relied upon fully. It also initially seemed

  clever to change the behavior of an executable simply by changing its

  name but this was quickly dismissed when clearer thinking prevailed

  concerning this and other security concerns. Therefore, the first

  command must be passed as an argument to `cmdtab.Execute("mytopcmd")`. It

  is simple enough, however, to compile other executables with different

  main commands. Indeed, only the argument to need change to do so.



* Rather than add the complication of wrapping lines in a block and

  indenting the proper number of spaces for a given terminal width

  (which could be resized), the choice to keep all blocks beginning from

  the leftmost column was made. This makes the best use of screen space,

  is consistent with the help documentation from Go itself, and allows

  the output to be sized to any width without complication. Instead,

  emphasis has been given to the headers of the specific blocks. When

  indentation is truly needed raw-text (initial four spaces) can be

  used.



* Full DocOpt parsing of usage was considered and dismissed. DocOpt

  solves a different problem with different priorities (which do no

  include consideration of getopt-style command lines interfaces as an

  anti-pattern in terms of HCI). In fact, the only thing that should

  every really been in a usage string are lists of subcommands or

  arguments called out by name with `` both of which are

  automatically formatted when detected. In other words, no usage should

  ever include `-` or `--` prefixed anything. *Everything* should be

  voice-able. This makes even the most complicated commands extremely

  easy to understand quickly. 



* Support for all-caps keywords (ex: `[OPTIONS]`) in usage strings was

  considered and dismissed because angle-bracket notation already covers

  that and having single suggested usage format is simpler. It is also

  far easier for the formatting parser to determine what should be

  italicized using angle-brackets.



* Decided against any paged output of any special hidden builtin to

  allow combination with other shell scripts when quick customization is

  needed. Paged output remains for the main `help`, `usage`, and

  `version` builtins however.



* Removed all color and formatting from the special hidden builtin

  output since is will mostly likely be used by shell scripts that

  further parse it, for example to email all the authors.



* `Emphasize()`, `Wrap()`, and `Indent()` have been made public in addition to

  `Format()` for convenient use by command authors, but the values for

  italic, bold, and bolditalic have not been. This is to preserve the

  look and feel of commands that use `Emphasize()` and put the power to

  control appearance into the hands of users instead of developers. 



* At one point the internal `map[string]*Command` index was private to

  discourage tight coupling between commands. But it was decided that

  this inflexibility came at too great a cost to potential needs of

  command creators in the future who might want to inspect the Index

  directly themselves without necessarily doing anything to is, say to

  use some sort of prefix convention. `Has()` was kept as a convenience.

  This does not change the fact that subcommands should be as independent

  and uncoupled as possible. In the extreme case when subcommands need

  to communicate they should use system environment variables as would

  any other two commands normally.



## How Does Completion Work?



Reading about [Bash Programmable Completion](https://duck.com/lite?kae=t&q=Bash Programmable Completion) is probably a good idea.



For Bash all you need to add to your `~.bashrc` or `~/.bash_completion` file is the following:



```bash



complete -C mycmd mycmd



```



This will cause the shell to run `mycmd` and set the `COMP_LINE`

environment variable every time you tap the tab key once or twice. This

allows *Complete Commander* to detect completion context and only print

the words that should be possible for the last word of the command when

the tab key was pressed.



The `cmdtab` package then sees completion context it resolves it by calling

Complete(). See the package docs for more on the specific algorithm

used, but generally it does the following:



* Prints the output of any completion function found, or

* Recursively looks up any matching Subcommand names and prints them



## Machine Learning in Simple Terminal Commands?



Yep. Allowing a completion function allows incredible interesting

possibilities with completion that uses full human language semantics

rather than just prefixing the most likely candidates. Even a full

machine learning code module could be added allowing any possible

speech. Such considerations seem very absent from the HCI conversation

regarding terminal command line usage.



## Automatic (Builtin) Subcommands



Unless specifically disabled with `OmitBuiltins` (although the size to

add these builtins is trivial), the following internal subcommands are

added to any executable that is created using the `cmdtab` package:



* `help []`

* `usage`

* `version`



These subcommands are so common they have become something of an

unspoken standard for modern commands. 



Several other utility subcommands are also builtin for help integrating

with shell environments, producing documentation in other formats, and

more:



```



_authors       list names and authors

_bash_complete print line to add for bash completion

_builtins      list all cmd package builtins names and summaries

_cmdversion    print the cmd package version

_complete      force completion context

_copyrights    list names and copyrights

_descriptions  list names and descriptions

_examples      list names and examples

_gits          list names and git source repos

_help_json     dump help documentation as JSON

_index         list all names and summaries from cmd package index

_issues        list names and issue reporting URLs

_licenses      list names and licenses

_names         list names, main first

_summaries     list names and summaries

_usages        list names and usages

_versions      list names and versions



```



### Help Documentation



Help documentation is inspired by the look and design of UNIX `man`

pages so as to feel comfortable to those using such documentation for

commands for decades. It contains all the details for the given command

or ``. Rather than dump all the documentation into a single

page, however, details of subcommands can be displayed separately.



### Usage Documentation



Usage is when double-tab doesn't provide enough hints about how the

command is to be used. Usage is meant to provide only minimal usage

output while 'help' provides the full detailed information for the

command.  If an additional argument is provided the detailed help for

that specific subcommand will be provided. If the subcommand does not

exist it will be ignored and the main help information shown instead.



### Version Documentation



Having a `version` subcommand in particular is well-defined as being the

place to put all legal and authorship information in addition to just

the version. Such is required by most all free software and open-source

licenses.



## TODO



Here's some stuff we know I want to add but haven't made issues or time for yet:



* move to 1.16, root out deprecated `ioutil`

* fix broken subcommands of subcommands with tab completion and usage

* support `` syntax formatting rendered as all caps upper without

  the angle brackets and underlined, or just leave if no support in

  terminal for underlining to make compat visually with man pages

* recursive `usage` building by combining `x.Usage` into one line

* better color terminal detection and support, configurable

* `_help_md` - output a markdown file containing doc info

* `_help_html` - output a standalone HTML5 document with doc info

* `_help_http` - serves the `_help_html` document locally

* internal paging when `less` not found (`go-pager`?)

* assume use of the standard logger `log`, trap and beautify output