“Bash” stands for “Bourne Again Shell”, which is a punny way of saying that it’s a re-imagining of the Bourne shell. None of which has anything to do with spy novels.
Back in the early days of Unix, there were three main shells: The Bourne shell(“sh”), the C shell(“csh”), and the Korn shell(“ksh”). The Bourne shell was always the vanilla, “standard” shell, while the C shell had a syntax resembling the C programming language and the Korn shell was packed with a lot more complexities and features than the Bourne shell.
Personally, I always used the Korn shell, and I have to say that the Bash shell feels very much like the Korn shell. They are very similar in functionality.
We’re going to look at the Bash shell here, and not worry about the others. Bash is the default that you’ll find in most Linux environment nowadays, anyways.
What Is a Shell?
A shell is just a program, like any other, except that it’s interactive. It accepts what you type in, executes programs for you and handles the input and output to and from those programs, and it displays stuff on the screen. It’s also special because it’s the program that is invoked by the “login” program after you’ve typed in the correct password. If you kill off your shell, your connection to the system will be dropped.
Linux commands have a fairly common structure:
$ command [option][option]...[option] arguments
That probably seems pretty cryptic:
- This is the name of the command. Something like
- Options are generally expressed with a hyphen (
-), and one or two letters. That’s the Unix way. You just need to know what the letters mean. Some utilities support alternate forms with double hyphens and long names. If an option has an argument that goes with it, then it will generally be preceded by a space.
- Most utilities are going to assume some quantity of information is going to always be needed. Like a file name to work on, or a search string. Arguments don’t have any special tags, and you generally just need to know what they are and what order they go in.
Here’s the command to search for a string in a file:
$ grep "abc" fred.data
The command is “grep” and there are no options, but two arguments. The first is the string to search for, and the second is the file to search in. This command will print every line in
fred.data that has the string “abc” in it.
We can add an option:
$ grep -v "abc" fred.data
Here we’ve added the
-v option, which reverses the sense of the search. Now it will print every line in
fred.data that does NOT have “abc” in it.
If the options don’t have any parameters, you can string them together behind a single
-. Here we’ll add the option to count the lines instead of printing them:
$ grep -vc "abc" fred.data
Every program that runs in Linux is passed pointers to three “files”, which all normally point to the terminal. You really, really need to understand these:
- Standard In(stdin)
- This is an input file. Data will be read from here.
- Standard Out(stdout)
- This is an output file, data will be written to here.
- Standard Error(stderr)
- This is an output file, any data related to errors that happen should be written to here.
Simple, right? Many, many Linux utilities are designed to read their data from
stdin, write the results on
stdout and write any errors on
What’s so important about that?
You can manipulate all of these files through the shell. Let’s look at the key operations:
Redirecting stdout to Disk File
The first thing you can do is that you can tell the shell to take all of the output from a utility and redirect it to a disk file. This allows you to save the output from a program. You do this by using the “redirect out” operator
>. Let’s look at this in action:
dave@MyComputer:~/demo $ ls /opt/turtl blink_image_resources_200_percent.pak locales content_resources_200_percent.pak natives_blob.bin content_shell.pak pdf_viewer_resources.pak icon.png resources icudtl.dat snapshot_blob.bin libffmpeg.so turtl libnode.so ui_resources_200_percent.pak LICENSE version LICENSES.chromium.html views_resources_200_percent.pak dave@MyComputer:~/demo $ ls /opt/turtl > test.out dave@MyComputer:~/demo $ cat test.out blink_image_resources_200_percent.pak content_resources_200_percent.pak content_shell.pak icon.png icudtl.dat libffmpeg.so libnode.so LICENSE LICENSES.chromium.html locales natives_blob.bin pdf_viewer_resources.pak resources snapshot_blob.bin turtl ui_resources_200_percent.pak version views_resources_200_percent.pak
In this example we’ve done a listing of
/opt/turtl, once to see what it looks like, and then again but saving the results to
test.out. Then we’ve used the
cat command to print the results to the screen. The
cat command is intended to be used to concatenate two files together, joining them one after the other and writing the results to
stdout. But if you just give it one file then it will essentially just write that file to
stdout - in this case, the screen.
One thing you’ll see is that the you get two columns of output just running the
ls command without any redirection, but you get only one column if you redirect it to a file. So the
ls program is aware that
stdout isn’t going to be the screen. In most cases, if you redirect the output of
ls to a file, you’re going to find it’s easier to deal with in a single column if you’re going to run that output through some other utility.
Redirect stdin From a Disk File
Now we have some test data, let’s run it through a utility to do something with it. For this example, we’ll use
wc, a program that counts the lines, words and characters in a file.
wc can actually be told to read a file, but if you don’t specify a file, then it assumes it’s
stdin. Let’s try both ways:
dave@MyComputer:~/demo $ wc test.out 18 18 318 test.out dave@MyComputer:~/demo $ wc abt efge otd to utean onot 3 6 27
Here we’ve run
wc twice. The first time we told it to count
test.out. The results were 18 lines, 18 words and 318 characters. Then we just entered the command
wc with no arguments, so it assumed that we meant
stdin. In case though,
stdin is just the terminal, so it drops down a line and just blinks at you. Then I typed a few random letters and hit
<Enter>, and it just blinked at me some more. So I typed a few more lines of random stuff, and I hit
<Ctrl>D is the code for “End of File”, or in technical terms
That signaled to
wc that the end of the input had come and it generated the results: 3 lines, 6 words and 27 characters.
Now lets redirect
stdin to come from
test.out. You do this with the “redirect in” operator,
<. Notice how it’s just the opposite angle bracket from redirect out. Think of them as arrows. “Out” points away from the command, “In” points towards it.
The command looks like this:
dave@MyComputer:~/demo $ wc < test.out 18 18 318
And it gives the same results as
Redirecting stdout From One Command into stdin of a Second
This is probably the most common use of all of this redirection stuff. Do in and out at the same time to put the output of one command into the input of another command. This is called “piping”, and it’s done using the “piping symbol”,
First we’ll look at using
cat to read a file from disk and write it to
stdout, and then we’ll pass it to
wc as its input:
dave@MyComputer:~/demo $ cat test.out | wcresources 18 18 318resources
As it stands, this isn’t terribly useful, since we know that
wc can just take a filename as an argument. But there are commands that only read
stdin, and this is a viable technique to use them with a disk file. Just pipe it through
More often, though, you are actually going to run a command that does something interesting, and then run it through one or more commands to manipulate the data and end up with even more interesting results.
So we’ll try that. Instead of using
test.out, we’ll create the
ls listing from scratch, then run it through
dave@MyComputer:~/demo $ ls /opt/turtl | wc 18 18 318
It’s the same results as before, just as you would expect.
But what if we only wanted to know how many files were in
/opt/turtl? How would we do that? In that case, we only want the first number that comes out of
wc, the line count. Here we can use another utility called
cut, which is used for pulling particular fields or columns out of an input file.
But we have a problem.
cut can use the spaces as a delimiter between the values, but there’s lots of extra spaces in there which are probably going to change if the numbers get bigger or smaller. You can see that there are less spaces between the “18” and the “318” than between the two “18”’s. We can use another utility,
tr (which stands for “translate”), to get rid of extra spaces.
Let’s try it:
dave@MyComputer:~/demo $ ls /opt/turtl | wc |tr -s " " 18 18 318
That looks better. The
-s option is the “squish” option which removes repeated instances of whatever characters are specified, in this case, it’s spaces.
So now we’re ready to run it through
dave@MyComputer:~/demo $ ls /opt/turtl | wc |tr -s " "|cut -d" " -f2 18
And there you go. The
-d option specifies the delimiter, and the
-f says what field to grab. Since we have a leading space, we want field 2, which is the first “18”.
This is just a demonstration to show piping, but in real life you’d just use the
-l option of
wc to report just the lines:
dave@MyComputer:~/demo $ ls /opt/turtl | wc -l 18
It’s always good to read the man pages!