Chapter 11. Advanced Shell Programming

Chapter Syllabus

11.1 Arithmetic and Logic Operations

11.2 The while-do-done Loop

11.3 The until-do-done Loop

11.4 The for-do-done Loop

11.5 Breaking a Loop

11.6 Text Processing

Loops are used to perform an operation repeatedly until a condition becomes true or false. The test or let command is used to check the condition every time a repetition is made. All loop structures used in shell programming start with a keyword. The block of commands that is executed repeatedly is enclosed by the do-done keywords.

There are three basic types of loops. The first one is the for-do-done loop, which is used to execute a block of commands for a fixed number of times. The while-do-done loop checks for a condition and goes on executing a block of commands until that condition becomes false. The until-do-done loop repeats the execution of a block of commands until a condition becomes true. As soon as the condition becomes true, the loop terminates.

All of these loops are controlled by a variable known as the control variable. This variable gets a new value on every repetition of the loop. The let command is also used to make arithmetic, logic, and assignment operations inside the loops and to change the value of the control variable.

In this chapter, we will start with arithmetic and logic operations performed with the let command. The three loops will be discussed one-by-one. You will find

the general syntax of each loop as well as a flow diagram. In the end, you will find some text processing examples and their use in loops.

11.1 Arithmetic and Logic Operations

The let command performs both arithmetic and logic operations. The use of the let command is important because all loops depend on the control variable. The value of this control must be changed during the execution of the loop. Usually this value is incremented or decremented with the help of the let command. The loop structures also need logic operations, used for the testing value of the control variable. This is the second use of the let command. Like the test command, the let command also has explicit and implicit modes.

Explicit Mode let Command

In the explicit mode, the word let is used in the command line. Consider the following example of the use of the command.

$ A=5

$ B=3

$ let "C=A+B"

$ echo $C

8

$

You created two new shell variables A and B and assigned these variables numeric values. Then you used the let command to sum these two values and assign the result to a third variable C. To display the value of this variable, you used the echo command. Like this arithmetic operation, you can also perform logic operations with the let command as shown in the following example.

$ var1=5

$ var2=3

$ let "var1"<>

$ echo $?

1

$ let "var1>var2"

$ echo $?

0

$

In this example, you compared two variables. The first comparison was not true, so the result code returned is 1. The second comparison is true and the result code is zero.

Implicit Mode let Command

You can replace the word let with double parentheses on each side of the expression. The above example, where you added two variables, can also be accomplished as follows.

$ A=5

$ B=3

$ ((C=A+B))

$ echo $C

8

$

The let command can also perform complex operations like the one shown here.

((A=A+(3*B)/(A-1)))

While evaluating the result of an expression, the usual arithmetic rules are applied. Parentheses can be used to alter the order of evaluation.

Table 11-1 lists the operators that can be used with the let command.

The first two operators are unary operators that need only one operand. All other operators are binary operators and need two operands. You will find many examples of the use of the let command in this chapter.

Table 11-1. Operators Used with the let Command

Operator

Description

-

Unary minus

!

Unary negation (same value but with a negative sign)

=

Assignment

+

Addition

-

Subtraction

*

Multiplication

/

Integer division

%

Remainder

Table 11-1. Operators Used with the let Command

Operator Description

<

Less than

>

Greater than

<=

Less than or equal to

>=

Greater than or equal to

==

Comparison for equality

!=

Comparison for nonequality

11.2 The while-do-done Loop

The while-do-done loop is used to test a condition before the execution of the block of commands contained inside the loop. The command block is executed if the test is successful and returns a true value. It may happen that the command block never executes if the test fails the very first time. The loop continues to execute as long as the condition remains true. The general syntax of the while-do-done loop is shown here.

while condition

do

command block

done

The condition is usually an expression containing a test or let command. Both of these commands are usually used in implicit mode. The while-do-done loop can be represented as a flow diagram as shown in Figure 11-1.

Figure 11-1. The while-do-done loop.

Let us see an example of the loop. We start with assigning the value 1 to a variable VAR1. Every time the loop executes, we double the value of the variable. The loop continues as long as the value of the variable is less than 100. As soon as the variable value reaches this limit, the loop execution terminates, and the next command after the done keyword is executed. The shell program script-20 follows.

#!/usr/bin/sh

echo "The while loop example"

echo

VAR1=1

while ((VAR1 < 100))

do

echo "Value of the variable is : $VAR1"

((VAR1 = VAR1 * 2))

done

echo

echo "The loop execution is finished"

You can also use the test command instead of the let command in the comparison made in the while condition. In that case, this line will be:

while [ VAR1 -lt 100 ]

When you execute this program, you will see the output shown here.

$ ./script-20

The while loop example

Value of the variable is : 1

Value of the variable is : 2

Value of the variable is : 4

Value of the variable is : 8

Value of the variable is : 16

Value of the variable is : 32

Value of the variable is : 64

The loop execution is finished

$

A while loop may become an infinite loop if you make a mistake while making a test decision. For example, consider the following program where you start with a value of VAR1 equal to 1. You add 2 to the value of VAR1 at each step. You compare the value of the variable with 10. This condition is never fulfilled because the value of the variable never becomes 10. It goes from 9 to 11, skipping the value to which the comparison is made. By changing "!=" to "<=", you can solve the problem. The program script-21 is shown here.

#!/usr/bin/sh

echo "The while loop example"

echo

VAR1=1

while ((VAR1 != 10))

do

echo "Value of the variable is : $VAR1"

((VAR1 = VAR1 + 2))

done

echo

echo "The loop execution is finished"

Another example of an infinite loop is when you forget to modify the control variable inside the loop, such as in the code segment that follows.

VAR1=1

while ((VAR1 != 10))

do

echo "Value of the variable is : $VAR1"

done

Here the value of VAR1 is always 1, and the condition remains true, resulting in an infinite loop.

11.3 The until-do-done Loop

The until-do-done loop is like the while-do-done loop. The only difference is that it tests the condition and goes on executing as long as the condition remains false. It terminates execution as soon as the condition becomes true. The general syntax of this loop is:

until condition

do

command block

done

The flow diagram of the until-do-done loop is shown in Figure 11-2.

Figure 11-2. The until-do-done loop.

As you may have noticed, the only difference between Figure 11-1 and Figure 11-2 is that the "True" and "False" positions have been interchanged. Here is script-22, which has the same result as script-20 but was implemented using an until-do-done loop.

#!/usr/bin/sh

echo "The until loop example"

echo

VAR1=1

until (( VAR1 > 100 ))

do

echo "Value of the variable is : $VAR1"

((VAR1 = VAR1 * 2))

done

echo

echo "The loop execution is finished"

11.4 The for-do-done Loop

The for-do-done loop is executed on a list of elements. The list of elements is assigned to a variable one-by-one. The value of this variable is processed inside the loop. The loop continues to execute until all of the list elements are processed and there are no more elements in the list. The general syntax of the for-do-done loop is:

for var in list

do

command block

done

The for-do-done loop flow diagram is shown in Figure 11-3.

Figure 11-3. The for-do-done loop.

As an example of the use of this loop, if you want to list all executable files in your home directory, you can use the following program (script-23) for this purpose.

#!/usr/bin/sh

echo "List of all executable files in home directory"

cd $HOME

for F in *

do

if [ -x $F ]

then

ll $F

fi

done

The asterisk character represents all files in this directory. When you run this program, the result is shown as follows. You may have a different result on your system. There may be other uses of this program. You can utilize this script to find all files that have the SUID bit set or some other type of file with slight modifications.

$ ./script-23

List of all executable files in home directory

-rwxr-xr-x 1 boota users 267 Oct 18 19:23 script-00

-rwxr-xr-x 1 boota users 131 Oct 18 19:53 script-01

-rwxr-xr-x 1 boota users 198 Oct 18 20:01 script-02

-rwxr-xr-x 1 boota users 100 Oct 18 20:07 script-03

-rwxr-xr-x 1 boota users 121 Oct 18 20:16 script-04

-rwxr-xr-x 1 boota users 132 Oct 18 21:25 script-05

-rwxr-xr-x 1 boota users 232 Oct 18 23:11 script-06

-rwxr-xr-x 1 boota users 177 Oct 18 22:04 script-07

-rwxr-xr-x 1 boota users 142 Oct 19 17:43 script-08

-rwxr-xr-x 1 boota users 170 Oct 19 18:04 script-09

-rwxr-xr-x 1 boota users 638 Oct 19 18:30 script-10

-rwxr-xr-x 1 boota users 313 Oct 19 19:31 script-11

-rwxr-xr-x 1 boota users 195 Oct 20 23:16 script-20

-rwxr-xr-x 1 boota users 193 Oct 20 23:00 script-21

-rwxr-xr-x 1 boota users 195 Oct 21 17:04 script-22

-rwxr-xr-x 1 boota users 140 Oct 21 17:07 script-23

$

The script-24 is another example of the for-do-done loop, where a list is provided to the for command. This list contains the names of weekdays, which the program reads one-by-one and prints them on your terminal screen.

#!/usr/bin/sh

for DAY in Sunday Monday Tuesday Wednesday Thursday Friday

Saturday

do

echo "The day is : $DAY"

done

The result of this program is:

$ ./script-24

The day is : Sunday

The day is : Monday

The day is : Tuesday

The day is : Wednesday

The day is : Thursday

The day is : Friday

The day is : Saturday

$

Changing File Access Date and Time

Let's suppose you want to change the access time of all files in your current directory to the current time. You can use the touch command with a small shell script as shown here.

for FILE in *

do

touch $FILE

done

Accessing Command Line Parameters

To process all command line parameters one-by-one using a for-do-done loop, the following code segment may be used.

for ARG in $*

do

echo $ARG

done

You can replace the echo command with any command or a block of commands to get a desired result.

Study Break

Use of Shell Loops

All of the three shell loops have their own applications. However, the while-do-done and until-do-done loops can be used interchangeably in many cases. Let's have some practice with these loops. Using a while-do-done loop, write a shell program that takes a number as input and then prints its table from 1 to 10. Now change the while-do-done

loop to an until-do-done loop to get the same functionality. Use the for-do-done loop and pass a list of numbers from 1 to 10 to the for statement. Again print the table with this arrangement.

11.5 Breaking a Loop

There may be situations when you want to break or discontinue the execution of commands inside the command block of a loop. This is done when a particular condition is met and you don't want any further execution of commands in the command block. You may also need to check an error condition and discontinue execution of the program depending on that error condition.

The shell provides three mechanisms for breaking the normal execution of loops. These are break, continue, and exit.

The break Command

The break command discontinues the execution of loop immediately and transfers control to the command following the done keyword. You can pass a number n as an argument to the break command. In that case, the break command jumps to the command after nth occurrence of the done keyword. Consider script-25 shown here. It asks you to enter a file name. If the file is a regular file, it uses the cat command and displays its contents. If the file is not a regular file, it displays a message and then quits execution after displaying the message "Good bye." The program uses an infinite loop and will not terminate until you enter a nonregular file name, such as a directory name.

#!/usr/bin/sh

while true

do

echo "Enter name of file to be displayed: \c"

read FILE

if [ ! -f $FILE ]

then

echo "This is not a regular file"

break

fi

cat $FILE

done

echo "Good bye"

Let us execute and enter a nonregular file such as /etc.

$ ./script-25

Enter name of file to be displayed: /etc

This is not a regular file

Good bye

$

The continue Command

The continue command is slightly different from the break command. When encountered, it skips the remaining part of the loop and transfers the control to the start of the loop for the next iteration. The script-26 does the same job as script-25 but with the use of the continue command. In this script, we have changed the test condition, and the loop goes on executing until you enter a file name that is not a regular file. At this point, the loop breaks and the command after the loop is executed.

#!/usr/bin/sh

while true

do

echo "Enter name of file to be displayed: \c"

read FILE

if [ -f $FILE ]

then

cat $FILE

continue

fi

echo "This is not a regular file"

break

done

echo "Good bye"

The exit Command

The exit command completely terminates the program. It returns an exit code that is optionally provided as its argument in the program. If the exit command doesn't have any arguments, it returns the exit code of the command executed just before it. This command is used when a critical error is encountered, and further execution of the program may cause faulty results. For example, dividing a number by zero is illegal, so you want to check this condition before a

command is executed that divides a number by zero. Program script-27 reads a number entered by the user and then divides 100 by this number. It then displays the quotient and the remainder. If you try to divide by zero, the program displays an error message and terminates immediately.

#!/usr/bin/sh

NUM=100

while true

do

echo "Enter a divisor for integer 100 : \c"

read DIV

if [ $DIV -eq 0 ]

then

echo "Divide by zero is not permitted"

exit 1

fi

(( QUO = NUM / DIV ))

(( REM = NUM % DIV ))

echo "The quotient is : $QUO"

echo "The remainder is : $REM"

done

11.6 Text Processing

You have used the grep command as a filter to extract or delete lines containing a particular text pattern. Here you will learn two more commands that are useful for text processing. The sed command is a stream editor that takes text from stdin and sends it to stdout after editing it. The cut command is used to extract a desired part of text from a line. It also takes its input from stdin and sends its output to stdout.

Using sed

The stream editor is a useful tool to edit large amounts of text at one time. For example, you may need to search for a word in a large file and replace it with another word. Let's try to replace the word "echo" with "ECHO" in script-27. The sed command will do the job as follows.

$ sed s/echo/ECHO/g script-27

#!/usr/bin/sh

NUM=100

while true

do

ECHO "Enter a divisor for integer 100 : \c"

read DIV

if [ $DIV -eq 0 ]

then

ECHO "Divide by zero is not permitted"

exit 1

fi

(( QUO = NUM / DIV ))

(( REM = NUM % DIV ))

ECHO "The quotient is : $QUO"

ECHO "The remainder is : $REM"

done

If you want to do an operation on all files, you can write a shell program to accomplish the job. Program script-28 shown here replaces "echo" with "ECHO" in all files of the current directory.

#!/usr/bin/sh

for FILE in *

do

cat $FILE |sed s/echo/ECHO/g >tempfile

cp tempfile $FILE

done

rm tempfile

As you can see, this is a very useful tool to make changes to a large number of files that could take a long time otherwise. Consider you are writing a book and want to change "figure" to "Fig" in all chapters. If you don't know how to do it in an efficient way, you may start editing all files manually, spending hours on a job that need take only a few minutes.

There are many ways to use sed that make it a very useful tool. For additional information, consult the sed manual pages.

Using cut

The cut command is used to extract a particular part of data from a line of text. If the data are in the form of fields, you can extract particular fields. For example, if you want to list all user names on your system, you can use the cut command on the /etc/passwd file as follows:

cut -f 1 -d : /etc/passwd

or

cat /etc/passwd | cut -f 1 -d :

Here the -f 1 option tells the command that you want to extract field number 1. The -d : option shows that the fields in the data are separated by a delimiter

colon ":". Since user names are in the start of each line in /etc/passwd and they are followed by a colon, the command extracts all user names from the file.

You may also use the cut command to extract a particular number of characters from a file. To extract the first eight characters from every line in the /etc/passwd file, you may use the following command.

cat /etc/passwd | cut -c 1-8

Here you specified a range of characters using the -c 1-8 option. See the manual pages for more information on the cut command.

Let us use the cut command in a shell program script-29. This script is used to send an email message to all users on a system. The message contents are stored in a file mailfile. You use the mailx command to send the message.

#!/usr/bin/sh

for USER in $(cut -f 1 -d : /etc/passwd)

do

mailx -s "Test mail" $USER

done

You have used the cut command to create a list of user names and then send a mail message to each name in the list.

The sleep Command

The sleep command is used to suspend execution for a certain amount of time. You provide the number of seconds as the argument to the sleep command. The following code segment lists all files in the current directory with a pause of five seconds between every file.

for FILE in *

do

ll $FILE

sleep 5

done

Test Your Knowledge

1:

Which command will you use to add the values of two variables VAR1 and VAR2, and store the result in VAR3?

A. [ $VAR3 = $VAR1 + $VAR2 ]

B. $VAR3 = [ $VAR1 + $VAR2 ]

C. $VAR3 = (( VAR1 + VAR2 ))

D. $VAR3 = VAR1 + VAR2 ))

2:

You want to wait for 10 seconds at the end of the loop in each loop cycle. Which command will you use?

A. sleep

B. pause

C. wait

D. Any of the three commands can be used.

3:

Consider the following code segment. How many times does the loop execute?

A=1

until [ $A < 10 ]

do

echo $A

(( $A=$A+1))

done

A. zero

B. one

C. nine

D. ten

4:

What will be the output of the program shown here?

#!/usr/bin/sh

A=1

while [ $A -lt 10 ]

do

B=1

while [ $B -lt 10 ]

do

break 2

echo "Inner loop"

done

echo "Outer Loop"

done

A. "Inner Loop" will be printed 10 times.

B. "Outer Loop" will be printed 10 times.

C. "Outer Loop" will be printed 9 times.

D. Nothing will be printed.

5:

While writing a program, you meet a situation where you want to break the normal execution and shift control to the beginning of the loop, skipping the remaining commands in the loop. Which command will you use?

A. break

B. continue

C. exit

D. shift