A variable within a shell script is just some data referenced by name. The data is always stored as a string of alpha numerical data. The shell does not work with concepts like “integers”, “reals” or “floating point variables”, everything is a string. Shell variable names may consist of upper- or lowercase letters, digits and the underscore. White space is not allowed in variable names. Some variables are predefined by the shell - a number of them are listed later on. All others can be user defined.

To set a variable within a script an equal sign is used:

A_VARIABLE=a_value

White space is not allowed anywhere in this construct. If you want to assign a string to a variable that contains white space, you either have to prepend it with a backslash, or put the string within double or single quotes, e.g:

A_VARIABLE="a value"

You can de-reference the variable name (“fetch its contents”) by using a dollar-sign in front of the name of the variable. For example, to print the contents of the variable set in our previous example, you could use:

echo $A_VARIABLE

Sometimes, we will need to delimit the variable name itself to prevent confusion. Suppose we have a variable named A and another one named AA - the following statement can mean either “print the contents of the variable named A followed by the literal string ‘A’” or it could mean “print the contents of the variable named AA”:

echo $AA

To prevent uncertainty, use curly braces to delimit the variable name:

echo ${A}A  # $A and string "A"
echo ${AA}  # $AA

If a variable is not set, the NULL value is returned by default if the variable is de-referenced. However, the shell provides a number of notations that allow substitutions if the NULL value was found:

Here are some examples of the use of substitutions: suppose we have a script that needs the variable MAILTO set. If we want the script to exit if that variable was not set, we could use:

MAILBOX=${MAILTO:?"MAILTO needs to be set"}

Or we might decide that the MAILBOX defaults to “root”, like this:

MAILBOX=${MAILTO:=root}

Variables values may be de-referenced and reset at will. But a variable can be declared to be read-only, using the readonly command:

$ READONLY="Concrete"
$ readonly READONLY
$ echo ${READONLY}
Concrete
$ READONLY="Brick"
bash: READONLY: readonly variable

The previous example was run using bash, the default shell on Linux systems. It is fully compatible with the original Bourne shell.

The shell uses a number of internal variables. The values for these are generally set at login by parsing the contents of a startup configuration file, e.g., the /etc/profile file or the .login or .bashrc script within the home directory of the account. Environment variables are by no means set in stone and can be altered by the user in most cases.

The scope of variables within a shell is global (in bash, you can use the local keyword to define local variables within functions). However, there is a catch: subshells - new shells, spawned by the current shell - do not automatically inherit the variables of the calling shell(script). To enable inheritance of a variable, you need to use the export keyword. When a variable is exported its data is accessible by its child-processes. Such a variable is called an “environment variable”. See the example captured below:

 
$ cat test
#!/bin/sh
echo ${a}
exit0
$ a=4
$ ./test

$ export a
$ a=4
$ ./test
4
$ _

Also, a subshell is not able to change the content of the variable in the calling shell - in other words: the subshell uses a copy of the variable. By using parentheses, you can force a subshell to be started. See this example:

#!/bin/sh
export a=4
echo $a
( a=9; echo $a )
echo $a
exit 0

This will produce the output:

4
9
4

#!/bin/sh
for parameter in "$@"
do
        echo $parameter
done       
exit 0

$ sh partest 1 2 3 4 5 6 "seven is a nice number" 8 9 10 11 12 
1 
2
3
4
5
6
seven is a nice number
8
9
10
11
12
$ _

$ sh partest 1 2 3 4 5 6 "seven is a nice number" 8 9 10 11 12
1 2 3 4 5 6 seven is a nice number 8 9 10 11 12
$ _   

$ sh partest 1 2 3 4 5 6 "seven is a nice number" 8 9 10 11 12
1 
2 
3 
4 
5 
6 
seven
is
a
nice
number
8
9
10
11
12
$ _  

#!/bin/sh
while [ ! -z "$1" ]
do
        echo "{" $1 "}"
        shift
done
exit 0

#!/bin/sh
while [ $# -ne 0 ]
do
        echo "$1"
        shift
done
exit 0

It is often necessary to conditionally execute parts of your scripts. Additionally, a number of methods is implemented which allow you to perform controlled iteration of blocks of code (looping).

The shell supports many branching options. The if is used most often. It enables you to conditionally execute blocks of code. The condition should immediately follow the if statement. Next comes the block of code to execute, delimited by the keywords then and fi:

if {condition}
then
        {code}
        {more code}
fi

The condition should evaluate to a value of zero (“true”) or one (“false”). The block of code will be executed if the condition is true. Optionally, you can use the else keyword to provide a container for a block of code that will be executed if the condition evaluates to false:

if {condition}
then
        {code executed if condition evaluates true}
else
        {code executed if condition evaluates false}
fi

.. or, to be concrete:

if test 100 -lt 1
then
        echo "100 is less than 1 - fascinating.."
else
        echo "1 is less than 100. So, what else is new?" 
fi

We made use of the shell built-in command test in this example. test evaluates logical expressions and is often used to form the condition for looping and branching instructions. test takes boolean statements as its argument and returns a value of zero or one. The shortened form of the test command is formed by enclosing an expression in square brackets [], like this:

if [ 100 -lt 1 ]
then
        echo "100 is less than 1 - fascinating.."
else
        echo "1 is less than 100. So, what else is new?" 
fi

Be certain to include white space between the square brackets and the statement. The test command is often used in conjunction with the shell's boolean operators. This relies on the shell's behaviour to evaluate just enough parts of an expressions to ensure its outcome:

[ $YOUR_INPUT = "booh" ] && echo "Are you a cow?"

The part of the expression to the right of the double ampersands (boolean AND) will never evaluate if the part to the left evaluates to false. The else keyword can be followed by another if keyword, which allows nesting:

if [ 100 -lt 1 ]
then
        echo "100 is less than 1. Fascinating.."
else 
     if [ 2 -gt 1 ]
     then
        echo "2 is bigger than 1. Figures."
     else
        echo "1 is bigger than 2. Fascinating.."
     fi
fi

The examples given so far are, of course, not very realistic, but serve the purpose of clarifying the concepts. In practice, testing is mostly done against variables, as demonstrated in the following example (which demonstrates a shorter form of the else ... if statements: the elif statement. It allows sequential matching of conditions without the need to nest if.. then.. else .. fi constructs):

#!/bin/sh

VAR=2
 
if [ $VAR -eq 1 ]
then
     echo "One."
elif [ $VAR -eq 2 ]
then
    echo "Two."
else
    echo "Not 1, not 2, but $VAR"
fi

Another branching mechanism is the case construct. The syntax for this construct is:

case {variable} in:

      {expression-1}) {code} ;;

      {expression-2}) {code} ;;
esac

The case construct is used to test the contents of a variable against a regular expression (as described in the section called “Regular Expressions”). If the expression matches, a corresponding block of code will be executed. That block starts after the closing parenthesis and should be terminated using a sequence of two semi-colons (;;). If a matching expression is found, the script will continue just after the esac statement. An example:

Language="Cobol"
 
case ${Language} in

      Java|java) echo "Java";;
      BASIC)     echo "Basic";;
      C)         echo "Ah, a master..";;
      *)         echo "Another language";;
esac  

Note the use of the the wild card “*)”, which matches any pattern. If none of the other expressions match the variable, the script prints a default message.

Loops iterate through a block of code until or while some condition is met. Conditions might be, for example, that a list of values is exhausted, a command returned a true or false value or a given variable or set of variables reached a value. Within a POSIX shell there are three types of loops: the for, while and until constructs. Previously, we gave you two examples of looping: the for and the while loop. A few additional examples will, hopefully, clarify their proper use:

for loops.  the syntax for a for loop is

for {variablename} in {list-of-values}
do
    {command} ...
done

{list-of-values} is a whitespace separated list of values. This type of loop fills the variable named in {variablename} with the first element of the list of values and iterates through the command(s). It keeps iterating until all values of the list have been processed, or the loop is terminated otherwise, e.g., using the break command. For example:

$ cat list.sh
#!/bin/sh

# this loop lists some numbers
# 
for VALUE in 19 2 1 3 4 9 2 1 9 11 14 33 
do
        echo -n "$VALUE "
done
echo
exit 0
$ sh list.sh
19 2 1 3 4 9 2 1 9 11 14 33 
$ _

It is possible to use the output of a command to generate a list of variables, and this can be combined with elements you define yourself, as is demonstrated in this capture of a session:

$ cat ./count
#!/bin/sh

# this loop counts from 1 to 10
# 
for VALUE in zero `seq 3`
do
        echo $VALUE
done
exit 0
$ sh count
zero
1
2
3
$ _

A for loop expects its parameters to be literal strings. Hence, execution of the seq program is forced by putting its name and parameters between backticks. Otherwise, the loop would just iterate the loop using the strings “zero”, “seq” and “3” to fill the VALUE variable.

Any loop can be terminated prematurely using the break statement. Execution of the code will be resumed on the first line after the loop, for example, this code snippet will print just “A” (and not “B” and “C”):

for var in A B C
do
        break
        echo "this will never be echoed"
done
echo $var

If, for some reason, the remaining part of the code within a loop needs not be executed, you can use the continue statement.

for var in A B C
do
        echo $var      
        continue
        echo "this will never be echoed"
done

The code snippet above will print

A
B
C

while loops.  If you want to execute code while some condition holds true, you can use the while statement. A while function executes statements within a corresponding do .. done block, e.g.:

while true
do
        echo "this will never be echoed"
done
echo "this is echoed"

In our example, we used the common program true, which will just exit with an exit code 0 (zero). Its counterpart, false will just exit with the value 1. Note that a while loop expects a list, much as the for loop does. The while loop, however, expects a list of executable commands rather than a list of variables. As long as the last command in the list returns zero, the commands delimited by the do .. done keywords will be executed:

while 
        false          # returns 1
        echo "before"  # returns 0 -> code within do..done block will be executed
do
        break
        echo "this will never be echoed"
done
echo "this is echoed"

Execution of this code would result into the following output:

before
this is echoed

Note, however, that you almost never see more then one command within the list. Often, this will be the test program. As you see, this loop-type can also be terminated prematurely using the break command.

while true
do
        break
        echo "this will never be echoed"
done
echo "this is echoed"

You can force execution of the next iteration by using the continue command:

stop=0
while test $stop -ne 1
do
        stop=1
        continue
        echo "this will never be echoed"
done
echo "this is echoed"

The example above also demonstrates the aforementioned usage of the program called “test”. Remember: test validates the expression formed by its arguments and exits with a value of true (0) or false (1) accordingly, see the manual page for test(1).

until loops.  The until function is very similar to the while function. It too expects a list of executable commands, but the corresponding block of code - again delimited by the do .. done statements - will be executed as long as the last command in the list returns a non-zero value:

until true 
do
        echo "this will never be echoed"
done
echo "this is echoed"

The following example lists the values from 1 to 10:

A=0
until [ $A -eq 11 ]
do
        A=`expr $A + 1`
        echo $A
done

Note the method used to increment the value of a variable, using the execution (backticks) of the expr tool.

Functions are just a set of shell commands grouped together under a unique name. Once a function has been defined, you can call it using its name as if it was a standalone program. Shell functions allow programmers to organize actions into logical blocks. Instead of having to repeat the same lines over and over again, which would result in long scripts, performance penalties and a far bigger chance of erroneous code, similar lines of code can be grouped together and called by name. This stimulates modular thinking on behalf of the programmer and provides reuse of code. It is also easier to follow the script's flow.

A function is defined by declaring its name, followed by a set of empty parentheses () and a body, denoted by curly braces {}. Within that body the commands are grouped, e.g.:

#!/bin/sh

dprint() {
        if [ $DEBUG ] 
        then
                echo "Debug: $*"
        fi
}

dprint "at begin of main code"

exit 0

Save this script in a file named td. Try running it. It will output nothing. Now set and export the DEBUG variable and run it again:

$ export DEBUG=1
$ ./td
Debug: at begin of main code
$ _

The if ... then construct used within the function will be explained later on. A number of things can be observed here. First of all, the function needs to be declared before it can be called. Also, the method to call a function is demonstrated: simply use its name. As you see, you can specify parameters. Note that the definition of the function does not require specification of the number of parameters. When calling a function you are free to specify as many parameters as you would like. You will need to write code within the function to ensure that the proper number of parameters is passed. The $* pseudo-variable can be used within the function to list (if any) the argument(s). Indeed, pseudo-variables $1..$9, $#, $@ and $* are all available within the function as well, and differ from the pseudo-variables with the same name within the script. To demonstrate this, save the following program in a file named “td2”:

#!/bin/sh
f1 () {
        echo "in function   at=" "$@"
        echo "in function hash=" "$#"
        echo "in function star=" "$*"
}
 
echo "in main script at=" "$@"
echo "in main script hash=" "$#"
echo "in main script star=" "$*"

f1 one flew over the cuckoos nest

exit 0

Running it, with the specified command, will have the following result:

$ sh ./td2 coo coo 
in main script at= coo coo
in main script hash= 2
in main script star= coo coo
in function   at= one flew over the cuckoos nest
in function hash= 6
in function star= one flew over the cuckoos nest
$ _

Though functions do not share the set of pseudo-variables with the rest of the script, they do share all other variables with the main program. If a function changes the content of a variable or defines a new one, its contents will be available in the main script as well. The following script clarifies this:

#!/bin/sh

SET_A () {
        A="new"
        B="bee"
}

A="old"

echo $A         # will display "old"

SET_A           # sets variable in script

echo $A         # will display "new" now
echo $B         # and this will display "bee"

exit 0

On most modern POSIX-compliant shells they will work just fine, to avoid confusion, it is best to refrain from using variables and functions with the same name in one script.

You are allowed to use recursion - in other words: you are allowed to call a function within itself. However, it is important to provide some sort of escape, otherwise your script will never terminate:

#!/bin/sh

call_me_4_ever() {

        echo "hello!"
        call_me_4_ever
}

call_me_4_ever
exit 0

This script will run forever, happily printing out greetings. A more usable example follows. Please study it carefully - it demonstrates various concepts we will explain later on.

   /---------------------------------------------------------------
01 | #!/bin/sh
02 | 
03 | seqpes() {
04 | 
05 |     fail()
06 |     {
07 |         echo failed: $*
08 |         exit 1
09 |     }
10 | 
11 |     [ $# -eq 3 ] || fail argument error
12 | 
13 |     local z=`expr $1 + $3`
14 | 
15 |     echo $1
16 |     [ $z -le $2 ] && seqpes $z $2 $3
17 |     echo $1
18 | }
19 |  
20 | seqpes $1 $2 $3 
21 |  
22 | exit 0                
   /---------------------------------------------------------------

Can you figure out what this program does? Probably not. Many people need to run this first to understand its function.

The example illustrates the concept of recursion, nested functions, local variables and alternate testing syntax. For those of you who look at this script with awe, it may come as a surprise to learn that this is a very typical example of illegible code. You are not supposed to write code like this (see the section called “Some words on coding style” for an explanation why not).

That said, lets look at the improved version:

   /---------------------------------------------------------------
01 | #!/bin/sh
02 | 
03 | # This script prints sequences of values, counting up first,
04 | # then down again. It needs 3 parameters: the start value, 
05 | # the maximal value and the increment. It uses recursion
06 | # to achieve this. See the comment before the function
07 | # seqpes() found below for an explanation. 
08 | 
09 | # fail() prints its arguments and exits the script with an
10 | # error code (1) to signal to the calling program that 
11 | # something went wrong.
12 | #
13 | fail()
14 | {
15 |     echo failed: $*
16 |     exit 1
17 | }
18 | 
19 | 
20 | # seqpes() needs 3 arguments: the current value, the maximal 
21 | # value and the increment to use. It will print the current 
22 | # value, increment it with the increment to use and then will 
23 | # create a new instance of itself, which acts likewise, 
24 | # until the printed result is equal or bigger than the maximal 
25 | # value. Each time the function is called by itself, a new set 
26 | # of pseudo-variables ($1..$3) is created and kept in memory. 
27 | # When the maximal value finally has been reached, the last 
28 | # function exits and gives control back to the function that 
29 | # called it, which acts likewise until the calling
30 | # program finally regains control. This ASCII art tries to clarify 
31 | # this:
32 | # 
33 | #  seqpez 1 3 1  
34 | #  print $1 ............ (1)
35 | #  |  seqpez 2 3 1
36 | #  |  print $1 ......... (2)
37 | #  |  |  seqpez 3 3 1
38 | #  |  |  print $1 ...... (3)
39 | #  |  |  print $1 ...... (3)
40 | #  |  |  $1=3
41 | #  |  print $1 ......... (2)
42 | #  |  <--exit
43 | #  print $1 ............ (1)
44 | #  <--exit
45 | #
46 | seqpes() {
47 | 
48 |     # print the current value and calculate the next value
49 |     #
50 |     echo $1
51 |     next_value=`expr $1 + $3`
52 | 
53 |     # check if we already surpassed the maximal value
54 |     #
55 |     if [ $next_value -le $2 ] 
56 |     then
57 |    # we need another iteration.
58 |         seqpes $next_value $2 $3
59 |     fi
60 | 
61 |     # Done. Print this value once more:
62 |     echo $1
63 | }
64 | 
65 | # The script starts here, and checks its parameters. Note that
66 | # the code that validates the arguments is very incomplete! We 
67 | # just check for the number of arguments, not for their values,
68 | # for example. This segment definitely lacks quality.
69 | # 
70 | if [ $# -ne 3 ] 
71 | then 
72 |     # print and exit
73 |     fail argument error
74 | else 
75 |     # call recursive function
76 |     seqpes $1 $2 $3 
77 | fi
78 | 
79 | exit 0     # tell calling program all went well
   /---------------------------------------------------------------

As you see, this code has almost four times the amount of lines our previous example had. However, it uses a more generic syntax, has many lines of comments and does not use local variables and nested functions. It works just as well as the previous example and is self-explanatory - hence, nothing more needs to be said about it.

A so-called here document allows you to use a series of text-lines directly from a shell script as input for a command. The following example will print a help-text:

   /---------------------------------------------------------------
01 |#!/bin/sh
02 |
03 |usage()
04 |{
05 |    cat << ENDCAT
06 |    Use $0 as follows:
06 |        $0 spec
08 |    where spec is one of the following:
09 |        red     - to select red output
10 |        green   - to select green output
11 |        blue    - to select blue output
12 |ENDCAT
13 |
14 |    exit 1
15 |}
16 |
17 |test -z "$1" && usage
18 |
19 |.... the rest of the script ...
   /--------------------------------------------------------------

This program will check in line 17 if the first argument is present (a way to test if any arguments were given at all). If not, the program will call a function called usage. Inside usage a here-document is used to feed the usage text to cat. The here-document starts in line 05 and ends in line 12. It consists of:

The text between the begin and end of the here document (that is text in lines 06 until 11) will be passed to cat, which will display it. The output of the usage function (supposing that the script is called mysetcolour) will be:

    Use mysetcolour as follows:
        mysetcolour spec
    where spec is one of the following:
        red     - to select red output
        green   - to select green output
        blue    - to select blue output

By now you should have learned enough to pass the LPIC-2 exam. However, shell programming is an art in itself. It goes well beyond the scope of this book to teach you everything there is to know about it. On a number of topics, if you would like to read further, please check the bibliography. Amongst the topics perhaps warranting further study are:

To trace a script's execution, you can use the command set. The option -x causes the shell to print each command as it is evaluated. Setting this option also enables a scripter to see the results of variable substitution and file name expansion. The -v option instructs the shell to print a command before it is evaluated. Using both parameters enables you to study the inner workings of the script, however it results in a rather elaborate and sometimes confusing output. To switch off debugging, use the “+x” option.

Experienced shell programmers often use debug functions and statements to debug a script. For example:

 
#!/bin/sh

DEBUG=1  # remove this line to suppress debugging output

debug()
{
        [ $DEBUG ] && echo DEBUG: $*
}


WORLD="World"
debug "WORLD = $WORLD"
echo "Hello World"
debug "End"

By setting the DEBUG variable, you can control whether or not debugging output will be sent. If you are done debugging, you can either remove the “debug” lines and function or leave them alone and remove the line that sets DEBUG.

You can use your own set of debugging functions to temporarily stop or delay execution of your script. By using the power that Unix gives you, you can even write a very basic debugger that enables you to set breakpoints and interactively view the environment. See the example below:

 
#!/bin/sh
 
DEBUG=1  # remove this line to suppress debugging output
 
debug_print()
{
        [ -z "$DEBUG" ] || echo DEBUG: $*
}
 
debug_breakpoint()
{
        while [ ! -z "$DEBUG" ] 
        do
           echo "DEBUG: HELD EXECUTION. (q)uit, (e)nvironment or (c)ontinue?"
           stty raw
           stty -echo
           A=`dd if=/dev/tty count=1 ibs=1 2>/dev/null`
           stty -raw
           stty echo
           case $A in
                q|Q) exit;;
                e|E) set;;
                c|C) break;
           esac
        done
}

debug_delay()
{
        [ -z "$DEBUG" ] || sleep $1
}
 
 
WORLD="World"
debug_print "WORLD = $WORLD"
echo "Hello World"
debug_print "End"
while true
do
        echo "loop.."
        debug_breakpoint
        debug_delay 1
done                 

This script demonstrates a number of techniques - we leave it up to you to study them as an exercise. Note, that the techniques used above will only work for interactive scripts - that is: scripts that have a controlling tty. Try running this script, both with and without the DEBUG flag.

The power of the Unix programming environment tends to seduce a lot of programmers into writing unreadable code. Often, the argumentation is that terse code will be more efficient. In practice, this is often not true. Additionally, many novice programmers tend to try to “show off” their knowledge. Many scripts are written that are barely readable and have no indentation. Bad code uses shortcuts, nesting and recursion whenever possible and uses uncommon constructs. All this is then spangled with esoteric regular expressions. You must never write code like this. Remember: you are almost a certified professional. The main reason you will be hired is not to impress your peers, but to enable others to do their work - including your peers.

Good programmers are consistent in how they write. They judge code readability to be more important than (minor) performance gains. They refrain from things like nested functions or functions that bear the same name as a variable. They user proper indentation and test their programs. They also test if their newly-written program really works. execute, e.g. For example:

.. code .. 

cd $WORKDIR
if [ $? -ne 0 ]
then
        echo could not change to $WORKDIR
        exit 1
fi

.. more code .. 

A good programmer spends a lot of time documenting his work, both inside the code and in separate documentation. He is aware that some of his peers may lack his skills and therefore tries to use the simplest approach to solve a problem. He uses plain English to clearly and succinctly explain how his code works. He refrains from snide remarks, the usage of weird acronyms and emoticons. In general: be brief, but complete.