[ Table of Contents ] [ Front Page ] [ Prev ] [ Linux Gazette FAQ ] [ Next ]

Talkback: Discuss this article with peers

"Linux Gazette...making Linux just a little more fun!"

Introduction to Shell Scripting--The Basics

By

``Here's a hint. When you think your code to exec a shell function is just not working, never, repeat NEVER send it "/etc/reboot" just to see what happens.''
 -- Elliott Evans


INTRO

Shell scripting is a fascinating combination of art and science that gives you access to the incredible flexibility and power of Linux with very simple tools. Back in the early days of PCs, I was considered quite an expert with DOS's "batch files", something I now realize was a weak and gutless imitation of Unix's shell scripts. I'm not usually much given to Microsoft-bashing - I believe that they have done some absolutely awesome stuff in their time - but their BFL ("Batch File Language") was a joke by comparison. It wasn't even a funny one.

Since shell scripting is an inextricable part of the shell itself, quite a bit of the material in here will deal with shell quirks, methods, and specifics. Be patient; it's all a part of the knowledge that is necessary for writing good scripts.
 

PHILOSOPHY OF SCRIPTING

Linux - Unix in general - is not a warm and fuzzy, non-knowledgeable-user oriented system. Rather than specifying exact motions and operations that you must perform, it provides you with a myriad of small tools which can be connected in a literally infinite number of combinations, to achieve any result that is necessary (I find Perl's motto of "TMTOWTDI" - There's More Than One Way To Do It - highly apropos for all of Unix). That sort of power and flexibility, of course, carries a price - increased complexity and a requirement for higher competence in the user. Just as there is an enormous difference between operating, say, a bicycle versus a super-sonic jet fighter, so is there an enormous difference between blindly following the rigid dictates of a standardized GUI and creating your own program, or shell script, that performs exactly the functions you need in exactly the way you need them done.

Shell scripting is programming - but it is programming made easy, with little, if any, formal structure. It is an interpreted language, with its own syntax - but it is only the syntax that you use when invoking programs from your command line; something I refer to as "recyclable knowledge". This, in fact, is what makes shell scripts so useful: in the process of writing them, you continually learn more about the specifics of your shell and the operation of your system - and this is knowledge that truly pays for itself in the long run as well as the short.
 

REQUIREMENTS

Since I have a strong preference for `bash', and it happens to be by far the most commonly used shell, that's what these scripts are written for. Even if you use something else, that's still fine: as long as you have `bash' installed, these scripts will execute correctly. As you will see, scripts invoke the shell that they need; it's part of what a well-written script does.

I'm going to assume that you're in your home directory, since we don't want these files scattered all over the place where you can't find them later. I'm also going to assume that you know enough to hit the "Enter" key after each line that you type in, and that, once you have selected a name for your shell script, you will check that you do not have an executable with that same name in your path (Hint: type "which bkup" to check for an executable called "bkup"). For this specific reason, you should never name your scripts "test". This is one of the FAQs of Unix, a.k.a. "why doesn't my shell script/program do anything?" There's an executable in /bin called "test" that does nothing (nothing obvious, that is) when invoked...

It goes without saying that you have to know the basics of file operations - copying, moving, etc. - as well as being familiar with the basic assumptions of the file system, i.e., "." is the current directory, ".." is the parent (the one above the current), "~" is your home directory, etc. You didn't know that? You do now! <chuckle>

Whatever editor you use, whether `vi', `emacs', `mcedit' (the default editor in Midnight Commander and one of my favorite tools), or any other text editor is fine; just don't save this work in some word-processing format.

In order to avoid constant repetition of material, I'm going to number the lines as we go through and discuss different parts of a script file. I'll be putting it all together at the end, anyway.
 

BUILDING A SCRIPT

Let's go over the very basics of creating a script. Those of you who find this obvious and simplistic are invited to follow along anyway; as we progress, the material will become more complex - and a "refresher" never hurts. As it is, the projected audience for this article is a Linux newbie, someone who has never created a shell script before - but wishes to become a Script Guru in 834,657 easy steps. :)
 

In its simplest form, a shell script is nothing more than a shortcut - a list of commands that you would normally type in, one after another, to be executed at your shell prompt - plus a bit of "magic" to notify the shell that it is indeed a script.

The "magic" consists of two simple things: a notation at the beginning of the script that specifies the program that is used to execute it, and a change in the permissions of the file containing the script in order to make it executable.

As a practical example, let's create a script that will "back up" a specified file to a selected directory; we'll go through the steps and the reasoning that makes it all happen.

First, let's create the file and set the permissions. Type

>bkup
chmod +x bkup
The first line creates a file called "bkup" in your current directory. The second line makes it executable; note that the "+x" option of `chmod' makes this script executable by everyone - if you wish to restrict that, you'll need to run `chmod' with "u+x" or "ug+x" (see the "chmod" man page). In most cases, though, just plain "+x" is fine.

Next, we'll need to actually create the script. Start your editor and open up the file you've just made:

mcedit bkup
The first line in all of the script files we create will be this one (again, remember to ignore the number and the colon at the start of the line): 

  

  
1: #!/bin/bash



  

  
I've heard this referred to as the 'hash-bang hack'. The interesting thing about it is that the pound character is actually a comment

  marker - everything following a '#' on a line is supposed to be ignored by the shell - but the '#!' construct is unique in that respect, and is interpreted as a prefix to the name of the executable that will actually process the lines that follow.
  
This is a subtle but important point, by the way: when a script runs, it actually starts an additional bash process that runs under the

  current one; that process executes the script and exits, dropping you back in the original shell that spawned it. This is why a script that,

  for example, changes directories as it executes will not leave you in that new directory when it exits: the original shell has not been told to change directories, and you're right where you were when you started - even though the change is effective while the script runs.

  
To continue with our script:

  

  

  
2:  # "bkup" - copies specified files to the user's ~/Backup
3:  # directory after checking for name conflicts.



  

  
As I've mentioned, the `#' character is a comment marker. It's a good idea, since you'll probably create a number of shell scripts in the

  future, to insert some comments at the beginning of each one to indicate what it does - or at some point, you'll be scratching your

  head and trying to remember why you wrote it. In later columns, we'll explore ways to make that reminder a bit more automatic... but let's go on.
  

  

  
4: cp -i $1 ~/Backup



  

  

  


  The "-i" syntax of the `cp' command makes it interactive; that is, if we run "bkup file.txt" and a file called "file.txt" already exists in

  the ~/Backup directory, `cp' will ask you if you want to overwrite it - and will abort the operation if you hit anything but the 'y' key.

  
The "$1" is a "positional parameter" - it denotes the first thing that you type after the script name. In fact, there's an entire list of

  these variables:

  
$0 - The name of the script being executed - in this case, "bkup".
$1 - The first parameter - in this case, "file.txt"; any parameter may
     be referred to by $<number> in this manner.
#@ - The entire list of parameters - "$1 $2 $3..."
$# - The number of parameters.
There are several other ways to address and manipulate positional parameters (see the `bash' man page) - but these will do us for now.

   
  
MAKING IT SMARTER

  
So far, our script doesn't do very much; hardly worth bothering, right? All right; let's make it a bit more useful. What if you wanted

  to both keep the file in the ~/Backup directory and save the new one - perhaps by adding an extension to show the "version"? Let's try

  that; we'll just add a line, and modify the last line as follows:

  

  

  
4: a=$(date +%T-%d_%m_%Y)
5: cp -i $1 ~/Backup/$1.$a



  

  
Here, we are beginning to see a little of the real power of shell scripts: the ability to use the results of other Linux tools, called "command substitution". The effect of the $(command) construct is to execute the command inside the parentheses and replace the entire "$(command)" string with the result. In this case, we have asked `date' to print the current time and date, down to the seconds, and pass the result to a variable called 'a'; then we appended that variable to the filename to be saved in ~/Backup. Note that when we assign a value to a variable, we use its name ( a=xxx ), but when we want to use that value, we must prepend a '$' to that name (.../$1.$a). The names of variables may be almost anything, with these exceptions:
  
  In my experience, if you confine your variable names to lower-case letters, dashes, and underscores, there won't be any problems.

   
  
The effect of the last two lines in the script is to create a unique filename - something like file.txt.01:00:00-01_01_2000 - that should not conflict with anything else in ~/Backup. Note that I've left in the "-i" switch as a "sanity" check: if, for some truly strange reason, two file names do conflict, "cp" will give you a last-ditch chance to abort. Otherwise, it won't make any difference - like dead yeast in beer, it causes no harm even if it does nothing useful.

  
By the way, the older version of the $(command) construct - the `command` (note that "back-ticks" are being used rather than single quotes) - is deprecated, for a good reason. $()s are easily nested - $(cat $($2$(basename file1 txt))), for example; something that cannot be done with back-ticks, as the second back-tick would "close" the first one, and the command would fail, or do something unexpected. You can still use them, though - in single, non-nested substitutions (the most common kind), or as the innermost or outermost pair of the nested set - but if you use the new method exclusively, you'll always avoid that error.

   

  
So, let's see what we have so far, with whitespace added for readability and the line numbers removed (hey, an actual script!):

  

  

  
#!/bin/bash

  
# "bkup" - copies specified files to the user's ~/Backup
# directory after checking for name conflicts.

  
a=$(date +%T-%d_%m_%Y)
cp -i $1 ~/Backup/$1.$a



  

  
Yes, it's only a two-line script - but one that's starting to become useful. We'll continue playing with it in the next issue.

   
  
Oh, one last thing; another "Unix FAQ". Should you try to execute your newly-created script by typing

  
bkup
at the prompt, you'll get this familiar reproof:
  
bash: bkup: command not found
 -- "HEY! Didn't we just sweat, and labor, and work hard... What happened?"
  
Unlike DOS, the execution of commands and scripts in the current directory is disabled by default - as a security feature. Imagine what would happen if someone created a script called "ls", containing "rm -rf *" ("erase everything") in your home directory and you typed "ls"! If the current directory (".") came before "/bin" in your PATH variable, you'd be in a sorry state indeed...

  
Due to this, and a number of similar "exploits" that can be pulled off, you have to specify the path to all executables that you wish to run there - a wise restriction. You can also move your script into a directory that is in your path, once you're done tinkering with it; "/usr/local/bin" is a good candidate for this (Hint: type "echo $PATH" to see which directories are listed).

  
Meanwhile, in order to execute it, simply type

  
./bkup file.txt

  
- the "./" just says that the file to be run is in the current directory. Use "~/", instead, if you're calling it from anywhere else;

  the point here is that you have to give a complete path to the executable, since it is not in any of the directories listed in your

  PATH variable.

  
This assumes, of course, that you have a file in your current directory called "file.txt", and that you have created a subdirectory

  called "Backup" in your home directory. Otherwise, you'll get an error.

   

  
REVIEW

  
In this article, we've looked at some of the basics involved in creating a shell script, as well as some specifics:

   

  
  
WRAP-UP

  
Well, that's a good bit of information for a start. Play with it, experiment; shell scripting is a large part of the fun and power of

  Linux. Next month, we'll talk about error checking - the things your script should do if the person using it makes an error in syntax, for example - as well as getting into loops and conditional execution, and maybe dealing with a few of the "power tools" that are commonly used in shell scripts.

  
Please feel free to send me suggestions for any corrections or improvements, as well as your own favorite shell-scripting tips or any really neat scripting tricks you've discovered; just like anyone whose ego hasn't swamped their good sense, I consider myself a student, always ready to learn something new. If I use any of your material, you will be credited.

  
Until then -

  
Happy Linuxing!

   

  

  

  

  REFERENCES
  
"man" pages for 'bash', 'cp', 'chmod'

   

  
``Not me, guy. I read the Bash man page each day like a Jehovah's Witness reads the Bible. No wait, the Bash man page IS the bible.

  Excuse me...''

  -- More on confusing aliases, taken from comp.os.linux.misc

  

  

  
Copyright © 2000, Ben Okopnik

  Published in Issue 52 of Linux Gazette, April 2000

  

  Talkback: Discuss this article with peers
  
[ Table of Contents ] [ Front Page ] [ Prev ] [ Linux Gazette FAQ ] [ Next ]