{"id":21667009,"url":"https://github.com/princexz/linux-shell","last_synced_at":"2026-04-17T17:31:28.477Z","repository":{"id":148304760,"uuid":"579525273","full_name":"Princexz/Linux-Shell","owner":"Princexz","description":"Linux- shell course examples","archived":false,"fork":false,"pushed_at":"2022-12-18T01:09:50.000Z","size":1093,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-20T06:48:00.267Z","etag":null,"topics":["bash","fork","linux","linux-shell","posix","shell","unix"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Princexz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-18T01:07:43.000Z","updated_at":"2024-11-10T17:34:04.000Z","dependencies_parsed_at":"2023-05-19T17:15:30.575Z","dependency_job_id":null,"html_url":"https://github.com/Princexz/Linux-Shell","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Princexz/Linux-Shell","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Princexz%2FLinux-Shell","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Princexz%2FLinux-Shell/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Princexz%2FLinux-Shell/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Princexz%2FLinux-Shell/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Princexz","download_url":"https://codeload.github.com/Princexz/Linux-Shell/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Princexz%2FLinux-Shell/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31938610,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T17:29:20.459Z","status":"ssl_error","status_checked_at":"2026-04-17T17:28:47.801Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","fork","linux","linux-shell","posix","shell","unix"],"created_at":"2024-11-25T11:32:00.557Z","updated_at":"2026-04-17T17:31:28.460Z","avatar_url":"https://github.com/Princexz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# The Shell\n\n**Material by Milad Fatenejad, Sasha Wood, Radhika Khetani and Karin Lagesen**\n\n*Modified by Shoaib Sufi for Manchester, 2014 and by Seb James for Sheffield 2015*\n\n# This tutorial\n\nThis is a tutorial to introduce you to the *shell* and how it might be\nuseful for your research.\n\nYou can use a browser to open this tutorial on github:\n    https://github.com/mikeg64/linux_shell/shell\n\n#Introduction\nAs a result Linux is available on many types of machines from super computers to PCs. \nCurrently UBUNTU is the flavour of Linux that has gained great popularity \t\nThe UNIX operating system was developed in the early 1970s by a group of enthusiasts at \nBell Laboratories in the USA. Since then it was modified by a number of different groups \nand evolved into many similar but not identical flavours. Linus Torvalds, at the time a \ncomputer science student at the university of Helsinki, started a freely available academic\n version of UNIX ‘LINUX’ that was to become the standard for all Linux implementations that \n followed. Now-a-days Unix and Linux have become almost synonymous with each other.  Linux \n is structured so that the user works within a ‘shell’ that can be configured by the system \n administrators, working behind the scenes. LINUX is a multi-user, multi-tasking operating \n system, which has the following features:\n \n\tHierarchical File System\n\tProcess Management\n\tCommand Interpreter (Shell)\n\t\n# What is a shell?\n\t\nA *shell* is a program which reads a command that you typed; decides\nwhat to do with it; does it; then prints out any text that was\ngenerated.\n\nIt's a middleman between you and the core (or *kernel*) of the computer.\n\nA *terminal* is a program that gives you access to the shell -\nthink of the terminal as the window enclosing the shell, and the shell\nas that little prompt at the bottom:\n\n    `you@somecomputer:~$`\n\nThe shell we'll use is called *bash*. Although there are others, bash\nis the most commonly used shell. The shell can be viewed as an\ninterpreted computer programming language with a focus on interactive\nuse.\n\n# What the shell does: It calls built-in commands and programs\n\nUsually, when you type a command at the shell, all you want the shell\nto do is to find, and execute a program.\n\nFor example, `ps` is a standalone program which gives you some\ninformation about the processes running on the computer:\n\n    ps\n\nWhen you type this, the shell first checks if `ps` is one of its own,\nspecial built-in keywords. It then looks in its list of \"places where\nthere might be programs\" and runs the first one it finds. (That list\nis called the PATH; more on that later).\n\nYou can see the ps program that you just ran by listing it:\n\n    ls /bin/ps\n\nSome commands you'll use are *bash builtins*. A couple of examples are\n`alias` and `source`. When you type these, you activate code which is\npart of the shell itself. This is also true of programming constructs\nsuch as conditionals (if/else), loops, variable assignments and so on.\n\nFor historical reasons, most important Linux commands are only two letters long. \nThis brevity can sometimes make them difficult to remember, and it is \nnot always easy to tell from a sequence of commands exactly what is happening. \nAlso Linux distinguishes upper case letters from lower case, and insists that many \ncommands are written in lower case. Typing a command in upper case will probably \ngenerate the response\n\n\tCommand not found\n\nA typical Linux command consists of a general command word, which may be followed \nby optional parameters that specify more precisely what you want the command to do. \nMany of these options consist of a single letter, making the command brief but not\n altogether easy to remember. If a command operates on files then the filenames \n must come after the options.\n \n\tcommand [option …] [filename …]\n\n\n\n# The Example: Manipulating Experimental Data Files\n\nWe will spend most of our time learning about the basics of the shell\nby manipulating some experimental data from a hearing test. \n\nTo get the data:\n\n    git clone https://github.com/mikeg64/linux_shell.git\n\nThe git command will grab all of the data needed for this workshop from GitHub.\n\nNow we'll *c*hange *d*irectory into the directory tree which git cloned for us:\n\n    cd linux_shell\n\n# Moving around the file system\n\nThe filesystem is like a tree. On Unix systems, there's only one root\nof the tree (the analogy ends at the ground). Windows systems may have\nseveral trees (C:\\ D:\\ and so on). The bottom of the tree is called\nthe root and in Unix, it's represented by the symbol '/'\n\nNavigating the filesystem at the shell requires some typing, but there\nare a number of shortcuts and conveniences to ease the pain.\n\nFirst we have to know where we are. The program `pwd` (print working\ndirectory) tells you where you are sitting in the directory tree. The\ncommand `ls` will list the files in the current directory. Directories\nare often called \"folders\" because of how they are represented in\nGUIs. Directories are just listings of files. They can contain other\nfiles or directories.\n\nWhen you start up a terminal on most systems, you will start in a special\ndirectory called the *home* directory. If you're using the managed desktop,\nyou'll initially find yourself in a Desktop directory, so change to your home:\n\n    cd\n\nEvery user has their own home\ndirectory where they have full access to create and delete files and\ndirectories. At the start of a session the `pwd` command tells us what\nthe name of our home directory is.  The last word in that listing\nshould also be the name of your user.  You can also find out your user\nname by entering the command `whoami`.\n\nYou can always get back to your home directory by typing `cd` (return).\n\n# File Types\n\nWhen you enter the `ls` command, it lists the contents of the current\ndirectory. There are several items in your home directory.\n\nLet's create an empty file using the `touch` command. Enter the\ncommand:\n\n    touch testfile\n\nThen list the contents of the directory again. You should see that a\nnew entry, called `testfile`, exists. The `touch` command just\ncreates an empty file.\n\n*touch is Super Useful! Why? because it updates the last-modified date\n of the file. This can be useful in scripts to check if you need to\n carry out some function or other, perhaps on data being generated by\n another program.*\n\nTo get a fuller listing, add the `-l` switch. This will show the file\nsize, the owner and information about the permissions applied to the\nfile. If the entry is a directory, then the first letter will be a\n\"d\". The fifth column shows you the size of the entries in\nbytes. Notice that `testfile` has a size of zero.\n\nTry `ls -l -h` (or equivalently `ls -lh`). That makes the file size\nshow up in \"human readable\" format.\n\nNow, let's get rid of `testfile`. To remove a file, just enter the\ncommand:\n\n    rm testfile\n\nThe `rm` command can be used to remove files. If you enter `ls` again,\nyou will see that `testfile` is gone.\n\n\n# Changing Directories\n\nNow, let's move to a different directory. The command `cd` (change\ndirectory) is used to move around. We used `cd` earlier to get us into\nthe `linux_shell` directory.  Now let's move into the\n`shell` directory. Enter the following command:\n\n    cd shell\n\nNow use the `ls` command to see what is inside this directory.  This\ndirectory contains all of the material for the shell part of this boot\ncamp. Now move to the directory containing the data for the shell\ntutorial:\n\n    cd data\n\nIf you enter the `cd` command by itself, you will return to the home\ndirectory. Try this, and then navigate back to the `shell`\ndirectory.\n\n# Arguments\n\nMost programs take additional arguments that control their exact\nbehavior. For example, `-F` and `-l` are arguments to `ls`.  The `ls`\nprogram, like many programs, take a lot of arguments. But how do we\nknow what the options are to particular commands?\n\nMost commonly used shell programs have a manual. You can access the\nmanual using the `man` program. Try entering:\n\n    man ls\n\nThis will open the manual page for `ls`. Use the space key to go\nforward and b to go backwards. When you are done reading, just hit `q`\nto exit.\n\nNote: if you are using Git Bash on Windows you will not have access to\n`man`. People have hosted the man pages at various sites which is\nuseful for people on any platform e.g\nwww.kernel.org/doc/man-pages/online_pages.html or\nwww.linuxmanpages.com\n\nPrograms that are run from the shell can get extremely complicated. To\nsee an example, open up the manual page for the `find` program, which\nwe will use later this session. No one can possibly learn all of these\narguments, of course. So you will probably find yourself referring\nback to the manual page frequently. Note: sometimes it can be pretty\ndifficult to understand what it says in a man file. However, each time\nyou read a man file you will understand more of it.\n\n# Job control\n\nMost programs, like `ls` and `cd` finish very quickly and output their\nresults immediately. Some programs last a long time and may output\ntheir results into a file, so they'll sit there holding your command\nline hostage until they finish. To allow long lived programs to be\nexecuted without losing access to the command line, most shells have a\nform of *job control* built in.\n\nIf a job is run in *the foreground*, then access to the command line\nis suspended until the job finishes. By default, a command you run at\nthe shell will run in the foreground.\n\nA job can be run in the background, in which case it will give the\ncommand line back to you for the execution of additional commands.\n\nTo run a job in the background, add an \u0026 after the command:\n\n    ps \u0026\n\nThis is particularly useful for graphical programs which open up their\nown window, or for running a program a few times in parallel.\n\n    ps \u0026 ps \u0026 \n\nIf you put an interactive program like an editor into the background,\nit'll effectively disappear. GNU Nano is a text editor which is both\ncommon and easy to use. Try it out: open it with\n\n    nano\n\nand then exit with Ctrl-x. As we don't have nano on the Managed Desktop,\nyou can use vi for this example. vi is more common than nano, but much\nmore confusing for new users:\n\n    vi\n\nYou have to exit vi with `:q!`.\n\nNow run it in the background:\n\n    nano \u0026\nor\n\n    vi \u0026\n\nNot so useful. You see the shell outputs the editor's job number and also its\nprocess id. You can list the current running jobs with\n\n    jobs\n\nIf it's job 1, then you can bring it into the foreground with\n\n    %1\n\nYou can stop the job with Ctrl-z and then put it into the\nbackground with `bg` or into the foreground with `fg`.\n\nYou can kill a stopped job with\n\n    kill %1\n\nAssuming it was job number 1.\n\n# Examining the contents of other directories\n\nBy default, the `ls` commands lists the contents of the working\ndirectory (i.e. the directory you are in). However, you can also\ngive `ls` the names of other directories to view. Navigate to the\nhome directory if you are not already there. Then enter the\ncommand:\n\n    ls linux_shell\n\nThis will list the contents of the `linux_shell` directory without\nyou having to navigate there. Now enter:\n\n    ls linux_shell/shell\n\nThis prints the contents of `shell`. The `cd` command works in a\nsimilar way. Try entering:\n\n    cd linux_shell/shell\n\nand you will jump directly to `shell` without having to go through\nthe intermediate directory.\n\n# Absolute vs. Relative Paths\n\nThe `cd` command takes an argument which is the directory\nname. Directories can be specified using either a *relative path* or\nan *absolute path*. The directories on the computer are arranged into\na hierarchy. The absolute path tells you where a directory is in that\nhierarchy, all the way from the root and up.\n\nNavigate to the home directory. Now, enter the `pwd` command and you\nshould see the full name of your home directory.  This tells you that\nyou are in a directory that is named the same as your user, which sits\ninside one or more other directories. The very top of the hierarchy is\na directory called `/` which is usually referred to as the *root\ndirectory*.\n\nFirst, figure out again what the absolute path to your home directory\nwas. Now enter the following command (replace the stuff in \u003c\u003e with the\nresults from `pwd`).\n\n    cd \u003cpwd-results\u003e/linux_shell/shell\n\nThis jumps to `shell`. Now go back to the home directory. We saw\nearlier that the command\n\n    cd linux_shell/shell\n\nhad the same effect - it took us to the `shell` directory. But,\ninstead of specifying the absolute path which started with a /, we\nspecified a *relative path*. In other words, we specified the path\nrelative to our current directory. A absolute path always starts with\na `/`. A relative path does not. You can usually use either an\nabsolute path or a relative path depending on what is most\nconvenient. If we are in the home directory, it is more convenient to\njust enter the relative path since it involves less typing.\n\nNow, list the contents of the `/bin` directory. Do you see anything\nfamiliar in there?\n\n# Saving time with shortcuts, wild cards, and tab completion\n\n## Shortcuts\n\nThere are some shortcuts which you should know about. Referring to the\nhome directory is very common. The shell recognises the tilde\ncharacter, `~`, as a shortcut for your home directory. Navigate to the\n`shell` directory, then enter the command:\n\n    ls ~\n\nThis prints the contents of your home directory, without you having to\ntype the absolute path. The shortcut `..` always refers to the directory\nabove your current directory. Thus: \n\n    ls ..\n\nprints the contents of the ~/linux_shell directory. You can chain\nthese together, so:\n\n    ls ../../\n\nprints the contents of what should be your home\ndirectory. Finally, the special directory `.` always refers to your\ncurrent directory. So, `ls`, `ls .`, and `ls ././././.` all do the\nsame thing, they print the contents of the current directory. This may\nseem like a useless shortcut right now, but we'll see when it is\nneeded in a little while.\n\nTo summarize, the commands `ls ~`, `ls ~/.`, `ls ../../`, and `ls\n\u003cabsolute path to home directory\u003e` all do exactly the same\nthing. These shortcuts are not necessary, they are provided for your\nconvenience.\n\n## Our data set: Cochlear Implants\n\nA cochlear implant is a small electronic device that is surgically\nimplanted in the inner ear to give deaf people a sense of\nhearing. More than a quarter of a million people have them, but there\nis still no widely-accepted benchmark to measure their effectiveness.\nIn order to establish a baseline for such a benchmark, teenagers with\nCIs were asked to listen to audio files on their computer and report:\n\n1.  the quietest sound they could hear\n2.  the lowest and highest tones they could hear\n3.  the narrowest range of frequencies they could discriminate\n\nTo participate, test subjects were played an audio sample by a lab\ntech who then recorded their data - when the subjects\nfirst heard the sound, or first heard a difference in the sound.  Each\nset of test results were written out to a text file, one set per file.\nEach participant has a unique subject ID, and a made-up subject name.\nEach experiment has a unique experiment ID. The experiment has\ncollected 351 files so far.\n\nThe data is a bit of a mess! There are inconsistent file names, there\nare extraneous \"NOTES\" files that we'd like to get rid of, and the\ndata is spread across many directories. We are going to use shell\ncommands to get this data into shape. By the end we would like to:\n\n1.  Put all of the data into one directory called \"alldata\"\n\n2.  Have all of the data files in there, and ensure that every file\n    has a \".txt\" extension\n\n3.  Get rid of the extraneous \"NOTES\" files\n\n\n## Globbing with wild cards [Super useful]\n\nNavigate to the `~/linux_shell/shell/data/THOMAS` directory. This\ndirectory contains our hearing test data for THOMAS. If we type `ls`,\nwe will see that there are a bunch of files which are just four digit\nnumbers. By default, `ls` lists all of the files in a given\ndirectory. The `*` character is a shortcut for \"everything\". Thus, if\nyou enter `ls *`, you will again see all of the contents of a given\ndirectory. This * can be combined with other characters. Now try this command:\n\n    ls *1\n\nThis lists every file that ends with a `1`. This command:\n\n    ls /usr/bin/*.sh\n\nLists every file in `/usr/bin` that ends in the characters `.sh`.\n\nThis command:\n\n    ls *4*1\n\nlists every file in the current directory which contains the number\n`4`, and ends with the number `1`. There are four such files: `0241`,\n`0341`, `0431`, and `0481`. \n\nSo how does this actually work? When the shell (and this is a\nbash-specific explanation; other shells may vary) sees a word that\ncontains the `*` character, it automatically looks for files that\nmatch this wild card; * means \"anything\". In this case, it identified\nfour such files. Then, it replaced the `*4*1` with the list of files,\nseparated by spaces *and passes that as the argument list to the\nprogram*. In other words, the two commands:\n\n    ls *4*1\n    ls 0241 0341 0431 0481\n\nresult in an identical call to `ls`. The `ls` command cannot tell the\ndifference between these two things.\n\nThe expansion of wild cards by the shell is known as *globbing*. There\nare some other wild cards beyond the * character which you can search\nup on the internet.\n\n* * * *\n## Short Exercise\n\nDo each of the following using a single `ls` command without\nnavigating to a different directory.\n\n1.  List all of the files in `/bin` that contain the letter `a`\n2.  Can you figure out if there are any directories inside of /bin?\n\n* * * *\n\n## More on expansion: quotation marks and spaces in filenames [Super Useful]\n\nSpaces are important when you type commands in the shell:\n\n    cd ../.. # To go back to the 'shell' directory\n\nThis command lists two files:\n\n    ls 4fileone 4filetwo\n\nThis command lists one file:\n\n    ls \"4fileone 4filetwo\"\n\nThe shell interprets the quotation marks and passes the string\n\"4fileone 4filetwo\" to the `ls` program.\n\nThis is how you can refer to a file containing a space. Another way is\nto \"escape the space\":\n\n    ls 4fileone\\ 4filetwo\n\nThe backslash there tells the shell to interpret the space as part of\nthe string and not as the separator between one argument and another.\n\nDealing with spaces is a bit annoying on the command line, which is\nwhy most Linux and Unix users tend to avoid spaces in their file\nnames.\n\nLastly,\n\n    ls '4fileone 4filetwo'\n\nIs similar to `ls \"4fileone 4filetwo\"`, but has an important\ndifference. Try this:\n\n    ls \"$HOME\"\n\nand this:\n\n    ls '$HOME'\n\n* * * *\n## Short Exercise\n\nWork through the above examples which used `ls`, replacing `ls` with `cat`.\nThis should prove that the quotation marks are important and that the use of\nspaces in filenames is inadvisable.\n\n* * * *\n\n## Tab Completion [Super useful]\n\nNavigate to the home directory. Typing out directory names can waste a\nlot of time. When you start typing out the name of a directory, then\nhit the tab key, the shell will try to fill in the rest of the\ndirectory name. For example, enter:\n\n    cd 2\u003ctab\u003e\n\nThe shell will fill in the rest of the directory name for\n`linux_shell`. Press enter to enter the boot camp directory. Next, go\ninto the shell directory and do:\n\n    ls 3\u003ctab\u003e\u003ctab\u003e\n\nWhen you hit the first tab, nothing happens. The reason is that there\nare multiple file in this directory which start with\n3. Thus, the shell does not know which one to fill in. When you hit\ntab again, the shell will list the possible choices. \n\nTab completion can also fill in the names of programs. For example,\nenter `e\u003ctab\u003e\u003ctab\u003e`. You will see the name of every program that\nstarts with an `e`. One of those is `echo`. If you enter `ec\u003ctab\u003e` you\nwill see that tab completion works.\n\n# Command History\n\nYou can easily access previous commands. `history` (a bash built-in)\nlists the command history.\n\n    [seb@sebpad 08:48:28 shell]$ history\n     (Extra output snipped)\n     2053  ls\n     2054  which history\n     2055  history\n    [seb@sebpad 08:48:45 shell]$ \n\nYou can re-call a command from that history like this:\n\n    !2053\n\nThis will call `ls` from the short list above.\n\n## Navigating and searching the command history [Super useful]\n\nYou can quickly access recent commands from the history and search the\nhistory:\n\nHit the up arrow.  Hit it again.  You can step backwards through your\ncommand history.  The down arrow takes you forwards in the command\nhistory.\n\n^-C will cancel the command you are writing, and give you a fresh prompt.\n\n^-R will do a reverse-search through your command history. This is\nvery useful. If you find a partial match you can keep pressing ^-R\nuntil you find the instance you are interested in.\n\n# Which program?\n\nCommands like `ls`, `rm`, and `cd` are just ordinary programs on the\ncomputer. A program is just a file that you can *execute*. The program\n`which` tells you the location of a particular program. For example:\n\n    which ls\n\nWill return \"/bin/ls\". Thus, we can see that `ls` is a program that\nsits inside of the `/bin` directory. Now enter:\n\n    which find\n\nYou will see that `find` is a program that sits inside of the\n`/bin` directory (it might be `/usr/bin` on some platforms).\n\nYou could have an executable program anywhere on the filesystem. When\nwe enter a program name, like `ls`, and hit enter, how does the shell\nknow where to look for that program?\n\nHow does it know to run `/bin/ls` when we enter `ls` and not\n`/home/me/usr/bin/ls`?\n\nThe answer is that when we enter a program name and hit enter, there\nare a few standard places that the shell automatically looks. If it\ncan't find the program in any of those places, it will print an error\nsaying \"command not found\". Enter the command:\n\n    echo $PATH\n\nThis will print out the value of the `PATH` environment variable.\nNotice that a list of directories, separated by colon characters, is\nreturned. These are the places the shell looks for programs to run. If\nyour program is not in this list, then an error is printed. The shell\nONLY checks in the places listed in the `PATH` environment variable.\n\n*Note: You can modify the PATH environment variable; adding special\n directories containing the programs you have written; this is super\n useful on Iceberg, where you can't install programs into /bin or\n /usr/bin*\n\nNavigate to the `shell` directory and list the contents. You will\nnotice that there is a program (executable file) called `hello` in\nthe shell directory. Now, try to run the program by entering:\n\n    hello\n\nYou *should* get an error saying that hello cannot be found. That is\nbecause the directory `\u003cyour home\ndirectory\u003e/linux_shell/shell` is not in the `PATH` (this is\nactually a security feature).  However, it turns\nout that in Windows git bash, the current working directory IS in the path.\n\nIn any case, you can run the `hello` program by entering:\n\n    ./hello\n\nRemember that `.` is a shortcut for the current working\ndirectory. This tells the shell to run the `hello` program which is\nlocated right here. So, you can run any program by entering the path\nto that program. You can run `hello` equally well by specifying:\n\n    \u003cpath to home directory\u003e/linux_shell/shell/hello\n\nOr by entering:\n\n    ../shell/hello\n\nWhile you're at it; try:\n\n    $HOME/linux_shell/shell/hello\n\nand\n\n    /home/$USER/linux_shell/shell/hello\n\nNeat huh? HOME and USER are two more examples of environment\nvariables.\n\nWhen there are no `/` characters, the shell assumes you want to look\nin one of the default places for the program.\n\n# Calling MatLab from the shell\n\n* This section can't be run with the Sheffield Windows Managed Desktop *\n\nIn can be convenient to call MatLab from the command line.\n\nFor me the location of `matlab` was\n*/Applications/MATLAB_R2013a.app/bin/matlab*. If running `matlab`\nproduces a *command not found* type of message then you can either use\nthe absolute path for matlab or you can add it to your *PATH* which is\nthe list of locations that the shell searches for commands and\nprograms:\n\n    MATLAB_LOCATION=/Applications/MATLAB_R2013a.app/bin\n    export PATH=$MATLAB_LOCATION:$PATH\n\n*You can put these lines into your .bashrc file to have them run every\n time you log in*\n\nNow using `nano` or `notepad` make a file called `hello.m` and put the following\ncontents in and save the file:\n\n    disp('hello')\n    exit()\n\nThen from the same directory call the following command:\n\n    matlab -nosplash -nodesktop -r hello -logfile out.txt\n   \nThis calls matlab without a GUI, the filename is `hello.m` but the\nargument to -r is just *hello* and the output is written to stdout and\na file called `out.txt` - you can use `nano` or `cat` to check the\ncontents.\n\nAnother way of calling MatLab is by using a *Here Document*. The Here\nDocument redirects the output of a command block into the *stdin* of a\nprogram or command. In our current example, rather than putting the\nlist of commands in a file and then calling `matlab` you can place\nthem in the following way:\n\n    matlab -nosplash -nodesktop \u003c\u003cHEREMARKER\n    disp('hello')\n    exit()\n    HEREMARKER         \n\nOne benefit of this approach is that you can keep all of your MatLab\nand bash commands in one file if this were a script.\n\nFor more information about *Here Documents* please refer to [The LDP's\nAdvanced Bash-Scripting\nGuide](http://www.tldp.org/LDP/abs/html/here-docs.html)\n\n* * * *\n# Short Exercise\n\nUsing the above example write a *Here Document* based script for\nMatLab, modify it's permissions and run it to test that it works.\n\n* * * *\n\n# Calling the shell from matlab\n\nLearning about the shell is useful because it makes it easier to call\nexternal programs from matlab scripts. When you use the system() call\nin matlab, it actually launches a shell, then passes the content of\nthe system() call's argument to the shell, meaning you can do things\nlike:\n\n    [status, stdout] = system ('ls ~/somedir/');\n\nThis assumes that your matlab is installed on Linux or Mac; the\nWindows matlab system call won't call a bash shell; instead it calls\nthe standard DOS shell.\n\nYou can shortcut to calling a shell command with the ! character in matlab:\n\n    ! ls\n\n# Examining Files\n\nWe now know how to switch directories, run programs, and look at the\ncontents of directories, but how do we look at the contents of files?\n\nThe easiest way to examine a file is to just print out all of the\ncontents using the program `cat`. Enter the following command:\n\n    cat ex_data.txt\n\nThis prints out the contents of the `ex_data.txt` file. This file \ncontains an example of how our data is formatted. If you enter:\n\n    cat ex_data.txt ex_data.txt\n\nIt will print out the contents of `ex_data.txt` twice. `cat` just\ntakes a list of file names and writes them out one after another (this\nis where the name comes from, `cat` is short for concatenate). \n\n* * * *\n# Short Exercises\n\n1.  Print out the contents of the `~/linux_shell/shell/dictionary.txt`\n    file. What does this file contain?\n\n2.  Without changing directories, (you should still be in `shell`),\n    use one short command to print the contents of all of the files in\n    the `\u003cyour home directory\u003e/linux_shell/shell/data/THOMAS` directory.\n\n* * * *\n\n`cat` is a terrific program, but when the file is really big, it can\nbe annoying to use. Try this:\n\n    cat  ~/linux_shell/shell/dictionary.txt\n\nAll you can see is about the last 25 lines of that file. How to see\nthe previous lines?\n\nOne way is to scroll up in your terminal or [Super useful tip] press\nShift-PgUp), but this has limitations (one is that neither works on\nthe Windows Managed Desktop's installation of git bash!).\n\nThe file viewing program, `less`, is a good tool for viewing long\nfiles. Enter the following command:\n\n    less ~/linux_shell/shell/dictionary.txt\n\n`less` opens the file, and lets you navigate through it. The commands\nare identical to the `man` program. Use \"space\" to go forward and hit\nthe \"b\" key to go backwards. The \"g\" key goes to the beginning of the\nfile and \"G\" goes to the end. When you are done, hit \"q\" to quit.\n\n`less` also gives you a way of searching through files. Just hit the\n\"/\" key to begin a search. Enter the word you would like\nto search for and hit enter. It will jump to the next location where\nthat word is found. Try searching the `dictionary.txt` file for the\nword \"cat\". If you hit \"/\" then \"enter\", `less` will just repeat\nthe previous search. `less` searches from the current location and\nworks its way forward. If you are at the end of the file and search\nfor the word \"cat\", `less` will not find it. You need to go to the\nbeginning of the file and search.\n\nRemember, the `man` program uses the same commands (in fact, it uses\nless as its viewer!), so you can search documentation using \"/\" as well.\n\n* * * *\n# Short Exercise\n\nUse the commands we've learned so far to figure out how to search\nin reverse while using `less`.\n\n* * * * \n\n\n# Redirection\n\nLet's turn to the experimental data from the hearing tests. \nThis data is located in the `~/linux_shell/shell/data`\ndirectory. Each subdirectory corresponds to a particular participant\nin the study. Navigate to the `Bert` subdirectory in `data`.  First,\npress `ls` to look at the files. There\nare a bunch of text files which contain experimental data\nresults. Lets print them all:\n\n    cat *\n\nNow enter the following command:\n\n    cat * \u003e ../all_data\n\nThis tells the shell to take the output from the `cat *` command and\ndump it into a new file called `../all_data`. To verify that this\nworked, examine the `all_data` file. If `all_data` had already\nexisted, we would overwritten it. So the `\u003e` character tells the shell\nto take the output from whatever is on the left and dump it into the\nfile on the right. The `\u003e\u003e` characters do almost the same thing,\nexcept that they will append the output to the file if it already\nexists.\n\n* * * *\n# Short Exercise\n\nUse `\u003e\u003e`, to append the contents of all of the files whose name contains the\nnumber 4 in the directory:\n\n    \u003cyour home directory\u003e/linux_shell/shell/data/gerdal\n\nto the existing `all_data` file. Thus, when you are done `all_data`\nshould contain all of the experiment data from Bert and any\nexperimental data file from gerdal that contains the number 4.\n\n* * * *\n\n\n# Creating, moving, copying, and removing\n\nWe've created a file called `all_data` using the redirection operator\n`\u003e`. This file is critical - it's our analysis results - so we want to\nmake copies so that the data is backed up.\nLets copy the file using the `cp` command. The `cp`\ncommand backs up the file. Navigate to the `data` directory and enter:\n\n    cp all_data all_data_backup\n\nNow `all_data_backup` has been created as a copy of `all_data`. We can\nmove files around using the command `mv`. Enter this command:\n\n    mv all_data_backup /tmp/\n\nThis moves `all_data_backup` into the directory `/tmp`. The directory\n`/tmp` is a special directory that all users can write to. It is a\ntemporary place for storing files. Data stored in `/tmp` may be\nautomatically deleted when the computer shuts down.\n\nThe `mv` command is also one way to rename files. Since this file is so\nimportant, let's rename it:\n\n    mv all_data all_data_IMPORTANT\n\nType in `ls`, and you will see that file name has been changed to\nall_data_IMPORTANT. Let's delete the backup file now:\n\n    rm /tmp/all_data_backup\n\nThe `mkdir` command is used to create a directory. Just enter `mkdir`\nfollowed by a space, then the directory name. \n\n* * * *\n# Short Exercise\n\nDo the following:\n\n1.  Rename the `all_data_IMPORTANT` file to `all_data`.\n2.  Create a directory in the `data` directory called `foo`\n3.  Then, copy the `all_data` file into `foo`\n4.  Do `ls foo` to have a look inside the new directory \n\n* * * *\n\nBy default, `rm`, will NOT delete directories. You can tell `rm` to\ndelete a directory using the `-r` option. Enter the following command:\n\n    rm -r foo\n\n\n# Count the words\n\nThe `wc` program (word count) counts the number of lines, words, and\ncharacters in one or more files. Make sure you are in the `data`\ndirectory, then enter the following command:\n\n    wc Bert/* gerdal/*4*\n\nFor each of the files indicated, `wc` has printed a line with three\nnumbers and also the relative file name. The first is the number of lines in that file. The second is\nthe number of words. Third, the total number of characters is\nindicated. The bottom line contains this information summed over all of\nthe files. Thus, there were 10445 characters in total. \n\nRemember that the `Bert/*` and `gerdal/*4*` files were merged\ninto the `all_data` file. So, we should see that `all_data` contains\nthe same number of characters:\n\n    wc all_data\n\nEvery character in the file takes up one byte of disk space (as it contain's `ASCII text`. Thus, the\nsize of the file in bytes should also be 10445. Let's confirm this:\n\n    ls -l all_data\n\nRemember that `ls -l` prints out detailed information about a file and\nthat the fifth column is the size of the file in bytes.\n\n* * * *\n\n# The awesome power of the Pipe\n\nSuppose I wanted to only see the total number of character, words, and\nlines across the files `Bert/*` and `gerdal/*4*`. I don't want to\nsee the individual counts, just the total. Of course, I could just do:\n\n    wc all_data\n\nSince this file is a concatenation of the smaller files. Yes, this\nworks, but I had to create the `all_data` file to do this. Thus, I\nhave wasted a precious 10445 bytes of hard disk space. We can do this\n*without* creating a temporary file, but first I have to show you two\nmore commands: `head` and `tail`. These commands print the first few,\nor last few, lines of a file, respectively. Try them out on\n`all_data`:\n\n    head all_data\n    tail all_data\n\nThe `-n` option to either of these commands can be used to print the\nfirst or last `n` lines of a file. To print the first/last line of the\nfile use:\n\n    head -n 1 all_data\n    tail -n 1 all_data\n\nLet's turn back to the problem of printing only the total number of\nlines in a set of files without creating any temporary files. To do\nthis, we want to tell the shell to take the output of the `wc Bert/*\ngerdal/*4*` and send it into the `tail -n 1` command. The `|`\ncharacter (called pipe) is used for this purpose. Enter the following\ncommand:\n\n    wc Bert/* gerdal/*4* | tail -n 1\n\nThis will print only the total number of lines, characters, and words\nacross all of these files. What is happening here? Well, `tail`, like\nmany command line programs will read from the *standard input* when it\nis not given any files to operate on. In this case, it will just sit\nthere waiting for input. That input can come from the user's keyboard\n*or from another program*. Try this:\n\n    tail -n 2\n\nNotice that your cursor just sits there blinking. Tail is waiting for\ndata to come in. Now type:\n\n    French\n    fries\n    are\n    good\n\nthen Ctrl-d. You should get the lines:\n\n    are\n    good\n\nprinted back at you due to you asking tail to return the last two by\ndoing -n 2. The Ctrl-d keyboard shortcut inserts an *end-of-file*\ncharacter. It is sort of the standard way of telling the program \"I'm\ndone entering data\". The `|` character replaces the data from the\nkeyboard with data from another command. You can string all sorts of\ncommands together using the pipe.\n\nThe philosophy behind these command line programs is that none of them\nreally do anything all that impressive. BUT when you start chaining\nthem together, you can do some really powerful things really\nefficiently. If you want to be proficient at using the shell, you must\nlearn to become proficient with the pipe and redirection operators:\n`|`, `\u003e`, `\u003e\u003e`, `2\u003e` and `2\u003e\u003e`.\n\n\n## A sorting example\n\nLet's create a file with some words to sort for the next example. We\nwant to create a file which contains the following names:\n\n    Bob\n    Alice\n    Diane\n    Charles\n\nNavigate to `/tmp` and open an empty file with nano or notepad:\n\n    nano toBeSorted\n\nNow enter the four names as shown above. When you are done, press\nCtrl-o to write out the file. Press enter to use the file name\n`toBeSorted`. Then press Ctrl-x to exit `nano` (or do equivalent\nthings in notepad).\n\nWhen you are back to the command line, enter the command:\n\n    sort toBeSorted\n\nNotice that the names are now printed in alphabetical order.\n\nTry looking at this file with `less` - note that the file itself has not changed.\n\n* * * *\n# Short Exercise\n\nUse the `echo` command and the append operator, `\u003e\u003e`, to append your\nname to the file, then sort it and send the output to a new file\ncalled Sorted.\n\nOnce you have looked at the new file, remove both toBeSorted and Sorted.\n\n* * * *\n\nLet's navigate back to `~/linux_shell/shell/data`. Enter the\nfollowing command:\n\n    wc Bert/* | sort -k 3 -n\n\nWe are already familiar with what the first of these two commands\ndoes: it creates a list containing the number of characters, words,\nand lines in each file in the `Bert` directory. This list is then\npiped into the `sort` command, so that it can be sorted. Notice there\nare two options given to sort:\n\n1.  `-k 3`: Sort based on the third column\n2.  `-n`: Sort in numerical order as opposed to alphabetical order\n\nNotice that the files are sorted by the number of characters.\n\n* * * *\n# Short Exercise\n\nUse the `man` command to find out how to sort the output from `wc` in\nreverse order.\n\n* * * * \n# Short Exercise\n\nCombine the `wc`, `sort`, `head` and `tail` commands so that only the\n`wc` information for the largest file is listed\n\nHint: To print the smallest file, use:\n\n    wc Bert/* | sort -k 3 -n | head -n 1\n\n* * * * \n\n# Putting commands into a script and execution permission [Super Useful]\n\nPrinting the smallest file seems pretty useful. We don't want to type\nout that long command often. Let's create a simple script, a simple\nprogram, to run this command. The program will look at all of the\nfiles in the current directory and print the information about the\nsmallest one. Let's call the script `smallest`. We'll use `nano` or `notepad` to\ncreate this file. Navigate to the `data` directory, then:\n\n    nano smallest\n\nThen enter the following text:\n\n    #!/bin/bash\n    wc * | sort -k 3 -n | head -n 1\n\nNow, `cd` into the `Bert` directory and enter the command\n`../smallest`. Notice that it says permission denied. This happens\nbecause we haven't told the shell that this is an executable\nfile. If you do `ls -l ../smallest`, it will show you the permissions on \nthe left of the listing.\n\nEnter the following commands:\n\n    chmod a+x ../smallest\n    ../smallest\n\nThe `chmod` command is used to modify the permissions of a file. This\nparticular command modifies the file `../smallest` by giving all users\n(notice the `a`) permission to execute (notice the `x`) the file. If\nyou enter:\n\n    ls -l ../smallest\n\nYou will see that the file permissions have changed. \nCongratulations, you just created your first shell script!\n\n# Searching files\n\nYou can search the contents of a file using the command `grep`. The\n`grep` program is very powerful and useful especially when combined\nwith other commands by using the pipe. Navigate to the `Bert`\ndirectory. Every data file in this directory has a line which says\n\"Range\". The range represents the smallest frequency range that can be\ndiscriminated. Lets list all of the ranges from the tests that Bert\nconducted:\n\n    grep Range *\n\nNow add the --color switch:\n\n    grep --color Range *\n\n* * * * \n# Short Exercise\n\nCreate an executable script called `smallestrange` in the `data`\ndirectory, that is similar to the `smallest` script, but prints the\nfile containing the file with the smallest Range. Use the commands\n`grep`, `sort`, and `tail` to do this.\n\n* * * * \n\n# Finding files\n\nThe `find` program can be used to find files based on arbitrary\ncriteria. Navigate to the `data` directory and enter the following\ncommand:\n\n    find . -print\n\nThis prints the name of every file or directory, recursively, starting\nfrom the current directory. Let's exclude all of the directories:\n\n    find . -type f -print\n\nThis tells `find` to locate only files. Now try this command:\n\n    find . -type f -name \"*1*\"\n\nThe `find` command can acquire a list of files and perform some\noperation on each file. Try this command out:\n\n    find . -type f -exec grep Volume {} \\;\n\nThis command finds every file starting from `.`. Then it searches each\nfile for a line which contains the word \"Volume\". The `{}` refers to\nthe name of each file. The trailing `\\;` is used to terminate the\ncommand.\n\nUsing find like this can be slow, because it is calling a new instance\nof `grep` for each item the `find` returns. It's possible to use find\nwith xargs to overcome this problem if necessary:\n\n    find . -type f -print | xargs grep Volume\n\n`find` generates a list of all the files we are interested in, \nthen we pipe them to `xargs`.  `xargs` takes the items given to it \nand passes them as arguments to `grep`.  `xargs` generally only creates\na single instance of `grep` (or whatever program it is running).\n\n* * * * \n# Short Exercise\n\nNavigate to the `data` directory. Use one `find` command to perform each\nof the operations listed below (except number 2, which does not\nrequire a `find` command):\n\n1.  Find any file whose name is \"NOTES\" within `data` and delete it \n\n2.  Create a new directory called `cleaneddata` (note: this should in the same  directory as `data`)\n\n3.  Move all of the files within `data` to the `cleaneddata` directory\n\n4.  Rename all of the files to ensure that they end in `.txt` (note:\n    it is ok for the file name to end in `.txt.txt`\n\nHint: If you make a mistake and need to start over just do the\nfollowing:\n\n1.  Navigate to the `shell` directory\n\n2.  Delete the `data` directory with `rm -r data`\n\n3.  Enter the command: `git checkout -- data` You should see that the\n    data directory has reappeared in its original state\n\n## BONUS FUN\n\nRedo exercise 4, except rename only the files which do not already end\nin `.txt`. You will have to use the `man` command to figure out how to\nsearch for files which do not match a certain name.\n\n* * * *\n\n# What did we miss out? [Super Useful]\n\nThere's more we could have fitted in to this session if it were\nlonger. Sometimes it's just useful to know that something exists so\nthat you can research its use later. Here's a list of programs we find\nsuper useful on the command line:\n\n## awk\n\nThe Microsoft Excel of the command line. Can print elements from a row\nof text. Often used with grep and sed.\n\n## locate\n\nA program which finds files in your filesystem by consulting a database.\n\n## sed\n\nThe stream editor - edit files on the fly in your scripts. Often used\nwith awk and grep.\n\n## tail -f\n\nThe -f argument to tail \"follows\" a file. You can use this to watch a\nfile grow. Often used to watch system log files.\n\n## sudo\n\n\"Super User DO\". Run a command as if you were the *root user* of the\nsystem. For admin tasks.\n\n## ssh, scp and sftp\n\nNetwork transparency. Jump from machine to machine with ssh and use it\nto execute commands on remote machines. Use scp and sftp to transfer files.\n\n## Regular expressions\n\nMany programs use regular expressions, which are a more advanced\nversion of the wild cards used by the shell. Annoyingly, there are\nmany slightly different flavours of regular expressions, but\neventually, you have to get your teeth into them (especially useful\nwith sed and grep).\n\n## .bashrc and .bash_profile\n\nThis files are executed when the shell starts. You can put any command\nin there. Often used to set an alias for a long command and to set\nenvironment variables like PATH so they are correct each time you open\na shell.\n\n## ` characters\n\n` characters have a special action. E.g.:\n\n    ls -l `which ls`\n\n## clear and reset\n\nTo clear your terminal, type `clear`. If your terminal goes all wierd,\ntry `reset`.\n\n## rename\n\nIf you have lots of files to rename and you need to do it according to\nsome pattern, then `rename` is your tool.\n\n## script\n\nAllows you to log a session.\n\n## screen\n\nRun several shells in a single terminal window in such a way that the\nsession is *immune to the network connection being lost*.\n\n\n\n* * * * \n\nPlease take a look at the [Shell Cheat Sheet]\n(https://github.com/mikeg64/linux_shell/blob/master/shell/shell_cheatsheet.md)\nto refresh what you have learned and to get a quick overview of some\nother topics.\n\nIf you have reached this part of the document and you have time left\nor you would like some further practice after the workshop then please\nattempt the [1000 Genome Shell\nexercises](https://github.com/mikeg64/linux_shell/blob/master/shell/exercises/shell.markdown)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprincexz%2Flinux-shell","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprincexz%2Flinux-shell","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprincexz%2Flinux-shell/lists"}