A Word About Operating Systems

Welcome to Perl.
Perl is the Swiss Army chainsaw of scripting languages: powerful and adaptable. It was first
developed by Larry Wall, a linguist working as a systems administrator for NASA in the late 1980s, as
a way to make report processing easier. Since then, it has moved into a large number of roles:
automating system administration, acting as glue between different computer systems; and, of course,
being one of the most popular languages for CGI programming on the Web.
Why did Perl become so popular when the Web came along? Two reasons: First, most of what is
being done on the Web happens with text, and is best done with a language that's designed for text
processing. More importantly, Perl was appreciably better than the alternatives at the time when
people needed something to use. C is complex and can produce security problems (especially with
untrusted data), Tcl can be awkward and Python didn't really have a foothold.
It also didn't hurt that Perl is a friendly language. It plays well with your personal programming style.
The Perl slogan is ``There's more than one way to do it,'' and that lends itself well to large and small
problems alike.
In this first part of our series, you'll learn a few basics about Perl and see a small sample program.
A Word About Operating Systems
In this series, I'm going to assume that you're using a Unix system and that your Perl interpreter is
located at /usr/local/bin/perl. It's OK if you're running Windows; most Perl code is platform-
independent.
Your First Perl Program
Take the following text and put it into a file called first.pl:
#!/usr/local/bin/perl
print "Hi there!\n";
(Traditionally, first programs are supposed to say Hello world!, but I'm an iconoclast.)
Now, run it with your Perl interpreter. From a command line, go to the directory with this file and type
perl first.pl. You should see:
Hi there!
The \n indicates the ``newline'' character; without it, Perl doesn't skip to a new line of text on its own.
Functions and Statements
Perl has a rich library of functions. They're the verbs of Perl, the commands that the interpreter runs.
You can see a list of all the built-in functions on the perlfunc main page. Almost all functions can be
given a list of parameters, which are separated by commas.
The print function is one of the most frequently used parts of Perl. You use it to display things on the
screen or to send information to a file (which we'll discuss in the next article). It takes a list of things to
output as its parameters.
print "This is a single statement.";
print "Look, ", "a ", "list!";
A Perl program consists of statements, each of which ends with a semicolon. Statements don't need
to be on separate lines; there may be multiple statements on one line or a single statement can be
split across multiple lines.
print "This is "; print "two statements.\n"; print "But this ",
"is only one statement.\n";
Numbers, Strings and Quotes
There are two basic data types in Perl: numbers and strings.
Numbers are easy; we've all dealt with them. The only thing you need to know is that you never insert
commas or spaces into numbers in Perl. always write 10000, not 10,000 or 10 000.
Strings are a bit more complex. A string is a collection of characters in either single or double quotes:
'This is a test.'
"Hi there!\n"
The difference between single quotes and double quotes is that single quotes mean that their
contents should be taken literally, while double quotes mean that their contents should be interpreted.
For example, the character sequence \n is a newline character when it appears in a string with double
quotes, but is literally the two characters, backslash and n, when it appears in single quotes.
print "This string\nshows up on two lines.";
print 'This string \n shows up on only one.';
(Two other useful backslash sequences are \t to insert a tab character, and \\ to insert a backslash into
a double-quoted string.)
Variables
If functions are Perl's verbs, then variables are its nouns. Perl has three types of variables: scalars,
arrays and hashes. Think of them as ``things,'' ``lists,'' and ``dictionaries.'' In Perl, all variable names
are a punctuation character, a letter or underscore, and one or more alphanumeric characters or
underscores.
Scalars are single things. This might be a number or a string. The name of a scalar begins with a
dollar sign, such as $i or $abacus. You assign a value to a scalar by telling Perl what it equals, like so:
$i = 5;
$pie_flavor = 'apple';
$constitution1776 = "We the People, etc.";
You don't need to specify whether a scalar is a number or a string. It doesn't matter, because when
Perl needs to treat a scalar as a string, it does; when it needs to treat it as a number, it does. The
conversion happens automatically. (This is different from many other languages, where strings and
numbers are two separate data types.)
If you use a double-quoted string, Perl will insert the value of any scalar variables you name in the
string. This is often used to fill in strings on the fly:
$apple_count = 5;
$count_report = "There are $apple_count apples.";
print "The report is: $count_report\n";
The final output from this code is The report is: There are 5 apples..
Numbers in Perl can be manipulated with the usual mathematical operations: addition, multiplication,
division and subtraction. (Multiplication and division are indicated in Perl with the * and / symbols, by
the way.)
$a = 5;
$b = $a + 10; # $b is now equal to 15.
$c = $b * 10; # $c is now equal to 150.
$a = $a - 1; # $a is now 4, and algebra teachers are cringing.
You can also use special operators like ++, --, +=, -=, /= and *=. These manipulate a scalar's value
without needing two elements in an equation. Some people like them, some don't. I like the fact that
they can make code clearer.
$a = 5;
$a++; # $a is now 6; we added 1 to it.
$a += 10; # Now it's 16; we added 10.
$a /= 2; # And divided it by 2, so it's 8.
Strings in Perl don't have quite as much flexibility. About the only basic operator that you can use on
strings is concatenation, which is a $10 way of saying ``put together.'' The concatenation operator is
the period. Concatenation and addition are two different things:
$a = "8"; # Note the quotes. $a is a string.
$b = $a + "1"; # "1" is a string too.
$c = $a ."1"; # But $b and $c have different values!
Remember that Perl converts strings to numbers transparently whenever it's needed, so to get the
value of $b, the Perl interpreter converted the two strings "8" and "1" to numbers, then added them.
The value of $b is the number 9. However, $c used concatenation, so its value is the string "81".
Just remember, the plus sign adds numbers and the period puts strings together.
Arrays are lists of scalars. Array names begin with @. You define arrays by listing their contents in
parentheses, separated by commas:
@lotto_numbers = (1, 2, 3, 4, 5, 6); # Hey, it could happen.
@months = ("July", "August", "September");
The contents of an array are indexed beginning with 0. (Why not 1? Because. It's a computer thing.)
To retrieve the elements of an array, you replace the @ sign with a $ sign, and follow that with the
index position of the element you want. (It begins with a dollar sign because you're getting a scalar
value.) You can also modify it in place, just like any other scalar.
print$months[0]; # This prints "July".
$months[2] = "Smarch"; # We just renamed September!
If an array doesn't exist, by the way, you'll create it when you try to assign a value to one of its
elements.
$winter_months[0] = "December"; # This implicitly creates @winter_months.
Arrays always return their contents in the same order; if you go through @months from beginning to
end, no matter how many times you do it, you'll get back July, August and September in that order. If
you want to find the length of an array, use the value $#array_name. This is one less than the number
of elements in the array. If the array just doesn't exist or is empty, $#array_name is -1. If you want to
resize an array, just change the value of $#array_name.
print$#months; # This prints 2.
$a1 = $#autumn_months; # We don't have an @autumn_months, so this is -1.
$#months = 0; # Now @months only contains "July".
Hashes are called ``dictionaries'' in some programming languages, and that's what they are: a term
and a definition, or in more correct language a key and a value. Each key in a hash has one and only
one corresponding value. The name of a hash begins with a percentage sign, like %parents. You
define hashes by comma-separated pairs of key and value, like so:
%days_in_month = ( "July" => 31, "August" => 31, "September" => 30 );
You can fetch any value from a hash by referring to $hashname{key}, or modify it in place just like any
other scalar.
print $days_in_month{"September"}; # 30, of course.
$days_in_month{"February"} = 29; # It's a leap year.
If you want to see what keys are in a hash, you can use the keys function with the name of the hash.
This returns a list containing all of the keys in the hash. The list isn't always in the same order, though;
while we could count on @months to always return July, August, September in that order, keys
%days_in_summer might return them in any order whatsoever.
@month_list = keys %days_in_summer;
# @month_list is now ('July', 'September', 'August') !
The three types of variables have three separate namespaces. That means that $abacus and
@abacus are two different variables, and $abacus[0] (the first element of @abacus) is not the same
as $abacus{0} (the value in abacus that has the key 0).
Comments
Notice that in some of the code samples from the previous section, I've used code comments. These
are useful for explaining what a particular piece of code does, and vital for any piece of code you plan
to modify, enhance, fix, or just look at again. (That is to say, comments are vital for all code.)
Anything in a line of Perl code that follows a # sign is a comment. (Except, of course, if the # sign
appears in a string.)
print"Hello world!\n"; # That's more like it.
# This entire line is a comment.
Loops
Almost every time you write a program, you'll need to use a loop. Loops allow you run a particular
piece of code over and over again. This is part of a general concept in programming called flow
control.
Perl has several different functions that are useful for flow control, the most basic of which is for.
When you use the for function, you specify a variable that will be used for the loop index, and a list of
values to loop over. Inside a pair of curly brackets, you put any code you want to run during the loop:
for $i (1, 2, 3, 4, 5) {
print "$i\n";
}
This loop prints the numbers 1 through 5, each on a separate line.
A handy shortcut for defining loops is using .. to specify a range of numbers. You can write (1, 2, 3, 4,
5) as (1 .. 5). You can also use arrays and scalars in your loop list. Try this code and see what
happens:
@one_to_ten = (1 .. 10);
$top_limit = 25;
for $i (@one_to_ten, 15, 20 .. $top_limit) {
print "$i\n";
}
The items in your loop list don't have to be numbers; you can use strings just as easily. If the hash
%month_has contains names of months and the number of days in each month, you can use the keys
function to step through them.
for $i (keys %month_has) {
print "$i has $month_has{$i} days.\n";
}
for $marx ('Groucho', 'Harpo', 'Zeppo', 'Karl') {
print "$marx is my favorite Marx brother.\n";

}
The Miracle of Compound Interest
You now know enough about Perl - variables, print, and for() - to write a small, useful program.
Everyone loves money, so the first sample program is a compound-interest calculator. It will print a
(somewhat) nicely formatted table showing the value of an investment over a number of years. (You
can see the program at compound_interest.pl)
The single most complex line in the program is this one:
$interest = int (($apr / 100) * $nest_egg * 100) / 100;
$apr / 100 is the interest rate, and ($apr / 100) * $nest_egg is the amount of interest earned in one
year. This line uses the int() function, which returns the integer value of a scalar (its value after any
fractional part has been stripped off). We use int() here because when you multiply, for example,
10925 by 9.25%, the result is 1010.5625, which we must round off to 1010.56. To do this, we multiply
by 100, yielding 101056.25, use int() to throw away the leftover fraction, yielding 101056, and then
divide by 100 again, so that the final result is 1010.56. Try stepping through this statement yourself to
see just how we end up with the correct result, rounded to cents.
Play Around!
At this point you have some basic knowledge of Perl syntax and a few simple toys to play with - print,
for(), keys(), and int(). Try writing some simple programs with them. Here are two suggestions, one
simple and the other a little more complex:
• A word frequency counter. How often does each word show up in an array of words? Print out
a report. (Hint: Use a hash to count of the number of appearances of each word.)
• Given a month and the day of the week that's the first of that month, print a calendar for the
month. (Remember, you need \n to go to a new line.)
Comparison operators
There's one important element of Perl that we skipped in the last article: comparison operators. Like
all good programming languages, Perl allows you ask questions such as `Ìs this number greater than
that number?'' or `Àre these two strings the same?'' and do different things depending on the answer.
When you're dealing with numbers, Perl has four important operators: <, >, == and !=. These are the
``less than,'' ``greater than,'' `èqual to'' and ``not equal to'' operators. (You can also use <=, ``less
than or equal to,'' and >=, ``greater than or equal to.)
You can use these operators along with one of Perl's conditional keywords, such as if and unless.
Both of these keywords take a condition that Perl will test, and a block of code in curly brackets that
Perl will run if the test works. These two words work just like their English equivalents - an if test
succeeds if the condition turns out to be true, and an unless test succeeds if the condition turns out to
be false:
if ($year_according_to_computer == 1900) {
print"Y2K has doomed us all! Everyone to the compound.\n";
}
unless ($bank_account > 0) {

print "I'm broke!\n";
}
Be careful of the difference between = and ==! One equals sign means `àssignment'', two means
``comparison for equality''. This is a common, evil bug:
if ($a = 5) {
print "This works - but doesn't do what you want!\n";
}
Instead of testing whether $a is equal to five, you've made $a equal to five and clobbered its old
value. (In a later article, we'll discuss a way to make sure this bug won't occur in running code.)
Both if and unless can be followed by an else statement and code block, which executes if your test
failed. You can also use elsif to chain together a bunch of if statements:
if ($a == 5) {
print "It's five!\n";
} elsif ($a == 6) {
print "It's six!\n";
} else {
print "It's something else.\n";
}
unless ($pie eq 'apple') {

print "Ew, I don't like $pie flavored pie.\n";
} else {
print"Apple! My favorite!\n";
}
while and until
Two slightly more complex keywords are while and until. They both take a condition and a block of
code, just like if and unless, but they act like loops similar to for. Perl tests the condition, runs the
block of code and runs it over and over again for as long as the condition is true (for a while loop) or
false (for a until loop).
Take a look at the following code and try to guess what it will do before reading further:
$a = 0;
while ($a != 3) {
$a++;
print "Counting up to $a...\n";
}
until ($a == 0) {
$a--;
print "Counting down to $a...\n";
}
Here's what you see when you run this program:
Counting up to 1...
Counting up to 2...
Counting up to 3...
Counting down to 2...
String comparisons
So that's how you compare numbers. Now, what about strings? The most common string comparison
operator is eq, which tests for string equality - that is, whether two strings have the same value.
Remember the pain that is caused when you mix up = and ==? Well, you can also mix up == and eq.
This is one of the few cases where it does matter whether Perl is treating a value as a string or a
number. Try this code:
$yes_no = "no";
if ($yes_no == "yes") {
print "You said yes!\n";
}
Why does this code think you said yes? Remember that Perl automatically converts strings to
numbers whenever it's necessary; the == operator implies that you're using numbers, so Perl converts
the value of $yes_no (``no'') to the number 0, and ``yes'' to the number 0 as well. Since this equality
test works (0 is equal to 0), the if block gets run. Change the condition to $yes_no eq "yes", and it'll do
what it should.
Things can work the other way, too. The number five is numerically equal to the string " 5 ", so
comparing them to == works. But when you compare five and " 5 " with eq, Perl will convert the
number to the string "5" first, and then ask whether the two strings have the same value. Since they
don't, the eq comparison fails. This code fragment will print Numeric equality!, but not String equality!:
$a = 5;
if ($a == " 5 ") { print "Numeric equality!\n"; }
if ($a eq " 5 ") { print "String equality!\n"; }
More fun with strings
You'll often want to manipulate strings: Break them into smaller pieces, put them together and change
their contents. Perl offers three functions that make string manipulation easy and fun: substr(), split()
and join().
If you want to retrieve part of a string (say, the first four characters or a 10-character chunk from the
middle), use the substr() function. It takes either two or three parameters: the string you want to look
at, the character position to start at (the first character is position 0) and the number of characters to
retrieve. If you leave out the number of characters, you'll retrieve everything up to the end of the
string.
$a = "Welcome to Perl!\n";
print substr($a, 0, 7); # "Welcome"
print substr($a, 7); # " to Perl!\n"
A neat and often-overlooked thing about substr() is that you can use a negative character position.
This will retrieve a substring that begins with many characters from the end of the string.
$a = "Welcome to Perl!\n";
print substr($a, -6, 4); # "Perl"
(Remember that inside double quotes, \n represents the single new-line character.)
You can also manipulate the string by using substr() to assign a new value to part of it. One useful
trick is using a length of zero to insert characters into a string:
$a = "Welcome to Java!\n";
substr($a, 11, 4) = "Perl"; # $a is now "Welcome to Perl!\n";
substr($a, 7, 3) = ""; # ... "Welcome Perl!\n";
substr($a, 0, 0) = "Hello. "; # ... "Hello. Welcome Perl!\n";
Next, let's look at split(). This function breaks apart a string and returns a list of the pieces. split()
generally takes two parameters: a regular expression to split the string with and the string you want to
split. (We'll discuss regular expressions in more detail in the next article; for the moment, we're only
going to use a space. Note the special syntax for a regular expression: / /.) The characters you split
won't show up in any of the list elements.
$a = "Hello. Welcome Perl!\n";
@a = split(/ /, $a); # Three items: "Hello.", "Welcome", "Perl!\n"
You can also specify a third parameter: the maximum number of items to put in your list. The splitting
will stop as soon as your list contains that many items:
$a = "Hello. Welcome Perl!\n";
@a = split(/ /, $a, 2); # Two items: "Hello.", "Welcome Perl!\n";
Of course, what you can split, you can also join(). The join() function takes a list of strings and
attaches them together with a specified string between each element, which may be an empty string:
@a = ("Hello.", "Welcome", "Perl!\n");
$a = join(' ', @a); # "Hello. Welcome Perl!\n";
$b = join(' and ', @a); # "Hello. and Welcome and Perl!\n";
$c = join('', @a); # "Hello.WelcomePerl!\n";
Filehandles
Enough about strings. Let's look at files - after all, what good is string manipulation if you can't do it
where it counts?
To read from or write to a file, you have to open it. When you open a file, Perl asks the operating
system if the file can be accessed - does the file exist if you're trying to read it (or can it be created if
you're trying to create a new file), and do you have the necessary file permissions to do what you
want? If you're allowed to use the file, the operating system will prepare it for you, and Perl will give
you a filehandle.
You ask Perl to create a filehandle for you by using the open() function, which takes two arguments:
the filehandle you want to create and the file you want to work with. First, we'll concentrate on reading
files. The following statement opens the file log.txt using the filehandle LOGFILE:
open (LOGFILE, "log.txt");
Opening a file involves several behind-the-scenes tasks that Perl and the operating system undertake
together, such as checking that the file you want to open actually exists (or creating it if you're trying to
create a new file) and making sure you're allowed to manipulate the file (do you have the necessary
file permissions, for instance). Perl will do all of this for you, so in general you don't need to worry
about it.
Once you've opened a file to read, you can retrieve lines from it by using the <> construct. Inside the
angle brackets, place the name of your filehandle. What is returned by this depends on what you want
to get: in a scalar context (a more technical way of saying `ìf you're assigning it to a scalar''), you
retrieve the next line from the file, but if you're looking for a list, you get a list of all the remaining lines
in the file. (One common trick is to use for $lines (<FH>) to retrieve all the lines from a file - the for
means you're asking a list.)
You can, of course, close a filehandle that you've opened. You don't always have to do this, because
Perl is clever enough to close a filehandle when your program ends or when you try to reuse an
existing filehandle. It's a good idea, though, to use the close statement. Not only will it make your
code more readable, but your operating system has built-in limits on the number of files that can be
open at once, and each open filehandle will take up valuable memory.
Here's a simple program that will display the contents of the file log.txt, and assumes that the first line
of the file is its title:
open (LOGFILE, "log.txt") or die "I couldn't get at log.txt";
# We'll discuss the "or die" in a moment.
$title = <LOGFILE>;
print "Report Title: $title";
for $line (<LOGFILE>) {
print $line;
}
close LOGFILE;
Writing files
You also use open() when you are writing to a file. There are two ways to open a file for writing:
overwrite and append. When you open a file in overwrite mode, you erase whatever it previously
contained. In append mode, you attach your new data to the end of the existing file without erasing
anything that was already there.
To indicate that you want a filehandle for writing, you put a single > character before the filename you
want to use. This opens the file in overwrite mode. To open it in append mode, use two > characters.
open (OVERWRITE, ">overwrite.txt") or die "$! error trying to overwrite";
# The original contents are gone, wave goodbye.
open (APPEND, ">>append.txt") or die "$! error trying to append";

# Original contents still there, we're adding to the end of the file
Once our filehandle is open, we can use the humble print statement to write to it. Specify the
filehandle you want to write to and a list of values you want to write:
print OVERWRITE "This is the new content.\n";
print APPEND "We're adding to the end here.\n", "And here too.\n";
Live free or die!
You noticed that most of our open() statements are followed by or die "some sort of message". This is
because we live in an imperfect world, where programs don't always behave exactly the way we want
them to. It's always possible for an open() call to fail; maybe you're trying to write to a file that you're
not allowed to write, or you're trying to read from a file that doesn't exist. In Perl, you can guard
against these problems by using or and and.
A series of statements separated by or will continue until you hit one that works, or returns a true
value. This line of code will either succeed at opening OUTPUT in overwrite mode, or cause Perl to
quit:
open (OUTPUT, ">$outfile") or die "Can't write to $outfile: $!";
The die statement ends your program with an error message. The special variable $! contains Perl's
explanation of the error. In this case, you might see something like this if you're not allowed to write to
the file. Note that you get both the actual error message (``Permission denied'') and the line where it
happened:
Can't write to a2-die.txt: Permission denied at ./a2-die.pl line 1.
Defensive programming like this is useful for making your programs more error-resistant - you don't
want to write to a file that you haven't successfully opened!
Here's an example: As part of your job, you write a program that records its results in a file called
vitalreport.txt. You use the following code:
open VITAL, ">vitalreport.txt";
If this open() call fails (for instance, vitalreport.txt is owned by another user who hasn't given you write
permission), you'll never know it until someone looks at the file afterward and wonders why the vital
report wasn't written. (Just imagine the joy if that ``someone'' is your boss, the day before your annual
performance review.) When you use or die, you avoid all this:
open VITAL, ">vitalreport.txt" or die "Can't write vital report: $!";
Instead of wondering whether your program wrote your vital report, you'll immediately have an error
message that both tells you what went wrong and on what line of your program the error occurred.
You can use or for more than just testing file operations:
($pie eq 'apple') or ($pie eq 'cherry') or ($pie eq 'blueberry')
or print "But I wanted apple, cherry, or blueberry!\n";
In this sequence, if you have an appropriate pie, Perl skips the rest of the chain. Once one statement
works, the rest are ignored. The and operator does the opposite: It evaluates your chain of
statements, but stops when one of them doesn't work.
open (LOG, "log.file") and print "Logfile is open!\n";
This statement will only show you the words Logfile is open! if the open() succeeds - do you see why?
Subs
So far, our Perl programs have been a bunch of statements in series. This is OK if you're writing very
small programs, but as your needs grow, you'll find it's limiting. This is why most modern programming
languages allow you to define your own functions; in Perl, we call them subs.
A sub is defined with the sub keyword, and adds a new function to your program's capabilities. When
you want to use this new function, you call it by name. For instance, here's a short definition of a sub
called boo:
sub boo {
print "Boo!\n";
}
boo(); # Eek!
(Older versions of Perl required that you precede the name of a sub with the & character when you
call it. You no longer have to do this, but if you see code that looks like &boo in other people's Perl,
that's why.)
Subs are useful because they allow you to break your program into small, reusable chunks. If you
need to analyze a string in four different places in your program, it's much easier to write one
&analyze_string sub and call it four times. This way, when you make an improvement to your string-
analysis routine, you'll only need to do it in one place, instead of four.
In the same way that Perl's built-in functions can take parameters and can return values, your subs
can, too. Whenever you call a sub, any parameters you pass to it are placed in the special array @_.
You can also return a single value or a list by using the return keyword.
sub multiply {
my (@ops) = @_;
return $ops[0] * $ops[1];
}
for $i (1 .. 10) {
print "$i squared is ", multiply($i, $i), "\n";
}
Why did we use the my keyword? That indicates that the variables are private to that sub, so that any
existing value for the @ops array we're using elsewhere in our program won't get overwritten. This
means that you'll evade a whole class of hard-to-trace bugs in your programs. You don't have to use
my, but you also don't have to avoid smashing your thumb when you're hammering nails into a board.
They're both just good ideas.
You can also use my to set up local variables in a sub without assigning them values right away. This
can be useful for loop indexes or temporary variables:
sub annoy {
my ($i, $j);
for $i (1 .. 100) {
$j .= "Is this annoying yet?\n";
}
print $j;
}
If you don't expressly use the return statement, the sub returns the result of the last statement. This
implicit return value can sometimes be useful, but it does reduce your program's readability.
Remember that you'll read your code many more times than you write it!
Putting it all together
At the end of the first article we had a simple interest calculator. Now let's make it a bit more
interesting by writing our interest table to a file instead of to the screen. We'll also break our code into
subs to make it easier to read and maintain.
[Download this program]
#!/usr/local/bin/perl -w
# compound_interest_file.pl - the miracle of compound interest, part 2
# First, we'll set up the variables we want to use.

$outfile = "interest.txt"; # This is the filename of our report.
$nest_egg= 10000; # $nest_egg is our starting amount
$year = 2000; # This is the starting year for our table.
$duration = 10; # How many years are we saving up?
$apr= 9.5; # This is our annual percentage rate.
&open_report;
&print_headers;
&interest_report($nest_egg, $year, $duration, $apr);
&report_footer;
sub open_report {
open (REPORT, ">$outfile") or die "Can't open report: $!";
}
sub print_headers {
# Print the headers for our report.
print REPORT "Year", "\t", "Balance", "\t", "Interest", "\t",
"New balance", "\n";
}
sub calculate_interest {
# Given a nest egg and an APR, how much interest do we collect?
my ($nest_egg, $apr) = @_;
return int (($apr / 100) * $nest_egg * 100) / 100;

}
sub interest_report {
# Get our parameters. Note that these variables won't clobber the
# global variables with the same name.
my ($nest_egg, $year, $duration, $apr) = @_;
# We have two local variables, so we'll use my to declare them here.

my ($i, $line);
# Calculate interest for each year.

for $i (1 .. $duration) {
$year++;
$interest = &calculate_interest($nest_egg, $apr);
$line = join("\t", $year, $nest_egg, $interest,

$nest_egg + $interest) . "\n";
print REPORT $line;
$nest_egg += $interest;
}
}
sub report_footer {
print REPORT "\n Our original assumptions:\n";
printREPORT " Nest egg: $nest_egg\n";
printREPORT " Number of years: $duration\n";
printREPORT " Interest rate: $apr\n";
close REPORT;
}
Notice how much clearer the program logic becomes when you break it down into subs. One nice
quality of a program written as small, well-named subs is that it almost becomes self-documenting.
Take a look at these four lines from our program:
open_report;
print_headers;
interest_report($nest_egg, $year, $duration, $apr);
report_footer;
Code like this is invaluable when you come back to it six months later and need to figure out what it
does - would you rather spend your time reading the entire program trying to figure it out or read four
lines that tell you the program 1) opens a report file, 2) prints some headers, 3) generates an interest
report, and 4) prints a report footer?
You'll also notice we use my to set up local variables in the interest_report and calculate_interest
subs. The value of $nest_egg in the main program never changes. This is useful at the end of the
report, when we output a footer containing our original assumptions. Since we never specified a local
$nest_egg in report_footer, we use the global value.
Play around!
In this article, we've looked at files (filehandles, open(), close(), and <>), string manipulation (substr(),
split() and join()) and subs. Here's a pair of exercises - again, one simple and one complex:
• You have a file called dictionary.txt that contains dictionary definitions, one per line, in the
format ``word space definition''. (Here's a sample.) Write a program that will look up a word
from the command line. (Hints: @ARGV is a special array that contains your command line
arguments and you'll need to use the three-argument form of split().) Try to enhance it so that
your dictionary can also contain words with multiple definitions in the format ``word space
definition:alternate definition:alternate definition, etc...''.
• Write an analyzer for your Apache logs. You can find a brief description of the common log
format at http://www.w3.org/Daemon/User/Config/Logging.html. Your analyzer should count
the total number of requests for each URL, the total number of results for each status code
and the total number of bytes output.
Simple matching
The simplest regular expressions are matching expressions. They perform tests using keywords like
if, while and unless. Or, if you want to be really clever, tests that you can use with and and or. A
matching regexp will return a true value if whatever you try to match occurs inside a string. When you
want to use a regular expression to match against a string, you use the special =~ operator:
$user_location = "I see thirteen black cats under a ladder.";
if ($user_location =~ /thirteen/) {
print "Eek, bad luck!\n";
}
Notice the syntax of a regular expression: a string within a pair of slashes. The code $user_location
=~ /thirteen/ asks whether the literal string thirteen occurs anywhere inside $user_location. If it does,
then the test evaluates true; otherwise, it evaluates false.
Metacharacters
A metacharacter is a character or sequence of characters that has special meaning. We've discussed
metacharacters in the context of double-quoted strings, where the sequence \n mean the newline
character, not a backslash, and the character n and \t means the tab character.
Regular expressions have a rich vocabulary of metacharacters that let you ask interesting questions
such as, ``Does this expression occur at the end of a string?'' or ``Does this string contain a series of
numbers?''
The two simplest metacharacters are ^ and $. These indicate ``beginning of string'' and `ènd of
string,'' respectively. For example, the regexp /^Bob/ will match ``Bob was here,'' ``Bob'' and ``Bobby.''
It won't match `Ìt's Bob and David,'' because Bob doesn't show up at the beginning of the string. The
$ character, on the other hand, means that you are matching the end of a string. The regexp /David$/
will match ``Bob and David,'' but not ``David and Bob.'' Here's a simple routine that will take lines from
a file and only print URLs that seem to indicate HTML files:
for $line (<URLLIST>) {
# "If the line starts with http: and ends with html...."
if (($line =~ /^http:/) and
($line =~ /html$/)) {
print $line;
}
}
Another useful set of metacharacters is called wildcards. If you've ever used a Unix shell or the
Windows DOS prompt, you're familiar with wildcards characters like * and ?. For example when you
type ls a*.txt, you see all filenames that begin with the letter a and end with .txt. Perl is a bit more
complex, but works on the same general principle.
In Perl, the generic wildcard character is .. A period inside a regular expression will match any
character, except a newline. For example, the regexp /a.b/ will match anything that contains a,
another character that's not a newline, followed by b - `àab,'' `à3b,'' `à b,'' and so forth.
If you want to literally match a metacharacter, you must escape it with a backslash. The regex /Mr./
matches anything that contains ``Mr'' followed by another character. If you only want to match a string
that actually contains ``Mr.,'' you must use /Mr\./.
On its own, the . metacharacter isn't very useful, which is why Perl provides three wildcard quantifiers:
+, ? and *. Each quantifier means something different.
The + quantifier is the easiest to understand: It means to match the immediately preceding character
or metacharacter one or more times. The regular expression /ab+c/ will match `àbc,'' `àbbc,''
`àbbbc'' and so on.
The * quantifier matches the immediately preceding character or metacharacter zero or more times.
This is different from the + quantifier! /ab*c/ will match `àbc,'' `àbbc,'' and so on, just like /ab+c/ did,
but it'll also match `àc,'' because there are zero occurences of b in that string.
Finally, the ? quantifier will match the preceding character zero or one times. The regex /ab?c/ will
match `àc'' (zero occurences of b) and `àbc'' (one occurence of b). It won't match `àbbc,'' `àbbbc''
and so on.
We can rewrite our URL-matching code to use these metacharacters. This'll make it more concise.
Instead of using two separate regular expressions (/^http:/ and /html$/), we combine them into one
regular expression: /^http:.+html$/. To understand what this does, read from left to right: This regex
will match any string that starts with ``http:'' followed by one or more occurences of any character, and
ends with ``html''. Now, our routine is:
for $line (<URLLIST>) {
if ($line =~ /^http:.+html$/) {
print $line;
}
}
Remember the /^something$/ construction - it's very useful!
Character classes
We've already discussed one special metacharacter, ., that matches any character except a newline.
But you'll often want to match only specific types of characters. Perl provides several metacharacters
for this. <\d> will match a single digit, \w will match any single ``word'' character (which, to Perl,
means a letter, digit or underscore), and \s matches a whitespace character (space and tab, as well
as the \n and \r characters).
These metacharacters work like any other character: You can match against them, or you can use
quantifiers like + and *. The regex /^\s+/ will match any string that begins with whitespace, and /\w+/
will match a string that contains at least one word. (But remember that Perl's definition of ``word''
characters includes digits and the underscore, so whether or not you think _ or 25 are words, Perl
does!)
One good use for \d is testing strings to see whether they contain numbers. For example, you might
need to verify that a string contains an American-style phone number, which has the form 555-1212.
You could use code like this:
unless ($phone =~ /\d\d\d-\d\d\d\d/) {
print "That's not a phone number!\n";
}
All those \d metacharacters make the regex hard to read. Fortunately, Perl allows us to improve on
that. You can use numbers inside curly braces to indicate a quantity you want to match, like this:
unless ($phone =~ /\d{3}-\d{4}/) {
print "That's not a phone number!\n";
}
The string \d{3} means to match exactly three numbers, and \d{4} matches exactly four digits. If you
want to use a range of numbers, you can separate them with a comma; leaving out the second
number makes the range open-ended. \d{2,5} will match two to five digits, and <\w{3,}> will match a
word that's at least three characters long.
You can also invert the \d, \s and \w metacharacters to refer to anything but that type of character. \D
matches nondigits; \W matches any character that isn't a letter, digit or underscore; and \S matches
anything that isn't whitespace.
If these metacharacters won't do what you want, you can define your own. You define a character
class by enclosing a list of the allowable characters in square brackets. For example, a class
containing only the lowercase vowels is [aeiou]. /b[aeiou]g/ will match any string that contains ``bag,''
``beg,'' ``big,'' ``bog'' or ``bug''. You use dashes to indicate a range of characters, like [a-f]. (If Perl
didn't give us the \d metacharacter, we could do the same thing with [0-9].) You can combine
character classes with quantifiers:
if ($string =~ /[aeiou]{2}/) {
print "This string contains at least
two vowels in a row.\n";
}
You can also invert character classes by beginning them with the ^ character. An inverted character
class will match anything you don't list. [âeiou] matches every character except the lowercase
vowels. (Yes, ^ can also mean ``beginning of string,'' so be careful.)
Flags
By default, regular expression matches are case-sensitive (that is, /bob/ doesn't match ``Bob''). You
can place flags after a regexp to modify their behaviour. The most commonly used flag is i, which
makes a match case-insensitive:
$greet = "Hey everybody, it's Bob and David!";
if ($greet =~ /bob/i) {
print "Hi, Bob!\n";
}
We'll talk about more flags later.
Subexpressions
You might want to check for more than one thing at a time. For example, you're writing a ``mood
meter'' that you use to scan outgoing e-mail for potentially damaging phrases. You can use the pipe
character | to separate different things you are looking for:
# In reality, @email_lines would come from your email text,
# but here we'll just provide some convenient filler.
@email_lines = ("Dear idiot:",
"I hate you, you twit. You're a dope.",
"I bet you mistreat your llama.",
"Signed, Doug");
for $check_line (@email_lines) {

if ($check_line =~ /idiot|dope|twit|llama/) {
print"Be careful! This line might
contain something offensive:\n",
$check_line, "\n";
}
}
The matching expression /idiot|dope|twit|llama/ will be true if `ìdiot,'' ``dope,'' ``twit'' or ``llama'' show
up anywhere in the string.
One of the more interesting things you can do with regular expressions is subexpression matching, or
grouping. A subexpression is like another, smaller regex buried inside your larger regexp, and is
placed inside parentheses. The string that caused the subexpression to match will be stored in the
special variable $1. We can use this to make our mood meter more explicit about the problems with
your e-mail:
if ($check_line =~ /(idiot|dope|twit|llama)/) {
print"Be careful! This line contains the
offensive word $1:\n",
$check_line, "\n";
}
}
Of course, you can put matching expressions in your subexpression. Your mood watch program can
be extended to prevent you from sending e-mail that contains more than three exclamation points in a
row. We'll use the special {3,} quantifier to make sure we get all the exclamation points.
if ($check_line =~ /(!{3,})/) {
print "Using punctuation like '$1'
is the sign of a sick mind:\n",
$check_line, "\n";
}
}
If your regex contains more than one subexpression, the results will be stored in variables named $1,
$2, $3 and so on. Here's some code that will change names in ``lastname, firstname'' format back to
normal:
$name = "Wall, Larry";
$name =~ /(\w+), (\w+)/;
# $1 contains last name, $2 contains first name
$name = "$2 $1";

# $name now contains "Larry Wall"
You can even nest subexpressions inside one another - they're ordered as they open, from left to
right. Here's an example of how to retrieve the full time, hours, minutes and seconds separately from
a string that contains a timestamp in hh:mm:ss format. (Notice that we're using the {1,2} quantifier so
that a timestamp like ``9:30:50'' will be matched.)
$string = "The time is 12:25:30 and I'm hungry.";
$string =~ /((\d{1,2}):(\d{2}):(\d{2}))/;
@time = ($1, $2, $3, $4);
Here's a hint that you might find useful: You can assign to a list of scalar values whenever you're
assigning from a list. If you prefer to have readable variable names instead of an array, try using this
line instead:
($time, $hours, $minutes, $seconds) = ($1, $2, $3, $4);
Assigning to a list of variables when you're using subexpressions happens often enough that Perl
gives you a handy shortcut:
($time, $hours, $minutes, $seconds) =
($string =~ /((\d{1,2}):(\d{2}):(\d{2}))/);
Watch out!
Regular expressions have two traps that generate bugs in your Perl programs: They always start at
the beginning of the string, and quantifiers always match as much of the string as possible.
Here's some simple code for counting all the numbers in a string and showing them to the user. We'll
use while to loop over the string, matching over and over until we've counted all the numbers.
$number = "Look, 200 5-sided, 4-colored pentagon maps.";
while ($number =~ /(\d+)/) {
print "I found the number $1.\n";
$number_count++;
}
print "There are $number_count numbers here.\n";
This code is actually so simple it doesn't work! When you run it, Perl will print I found the number 200
over and over again. Perl always begins matching at the beginning of the string, so it will always find
the 200, and never get to the following numbers.
You can avoid this by using the g flag with your regex. This flag will tell Perl to remember where it was
in the string when it returns to it. When you insert the g flag, our code looks like this:
$number = "Look, 200 5-sided, 4-colored pentagon maps.";
while ($number =~ /(\d+)/g) {
print "I found the number $1.\n";
$number_count++;
}
print "There are $number_count numbers here.\n";
Now we get the results we expected:
I found the number 200.
There are 3 numbers here.
The second trap is that a quantifier will always match as many characters as it can. Look at this
example code, but don't run it yet:
$book_pref = "The cat in the hat is where it's at.\n";
$book_pref =~ /(cat.*at)/;
print $1, "\n";
Take a guess: What's in $1 right now? Now run the code. Does this seem counterintuitive?
The matching expression (cat.*at) is greedy. It contains cat in the hat is where it's at because that's
the largest string that matches. Remember, read left to right: ``cat,'' followed by any number of
characters, followed by `àt.'' If you want to match the string cat in the hat, you have to rewrite your
regexp so it isn't as greedy. There are two ways to do this:
1. Make the match more precise (try /(cat.*hat)/ instead). Of course, this still might not work - try using
this regexp against The cat in the hat is who I hate.
2. Use a ? character after a quantifier to specify nongreedy matching. .*? instead of .* means that Perl
will try to match the smallest string possible instead of the largest:
# Now we get "cat in the hat" in $1.
$book_pref =~ /(cat.*?at)/;
Search and replace
Now that we've talked about matching, there's one other thing regular expressions can do for you:
replacing.
If you've ever used a text editor or word processor, you're familiar with the search-and-replace
function. Perl's regexp facilities include something similar, the s/// operator, which has the following
syntax: s/regex/replacement string/. If the string you're testing matches regex, then whatever matched
is replaced with the contents of replacement string. For instance, this code will change a cat into a
dog:
$pet = "I love my cat.\n";
$pet =~ s/cat/dog/;
print $pet;
You can also use subexpressions in your matching expression, and use the variables $1, $2 and so
on, that they create. The replacement string will substitute these, or any other variables, as if it were a
double-quoted string. Remember our code for changing Wall, Larry into Larry Wall? We can rewrite it
as a single s/// statement!
$name = "Wall, Larry";
$name =~ s/(\w+), (\w+)/$2 $1/; # "Larry Wall"
s/// can take flags, just like matching expressions. The two most important flags are g (global) and i
(case-insensitive). Normally, a substitution will only happen once, but specifying the g flag will make it
happen as long as the regex matches the string. Try this code, and then remove the g flag and try it
again:
$pet = "I love my cat Sylvester, and my other cat Bill.\n";
$pet =~ s/cat/dog/g;
print $pet;
Notice that without the g flag, Bill doesn't turn into a dog.
The i flag works just as it did when we were only using matching expressions: It forces your matching
search to be case-insensitive.
Putting it all together
Regular expressions have many practical uses. We'll look at a httpd log analyzer for an example. In
our last article, one of the play-around items was to write a simple log analyzer. Now, let's make it a bit
more interesting: a log analyzer that will break down your log results by file type and give you a list of
total requests by hour.
(Complete source code.)
First, let's look at a sample line from a httpd log:
127.12.20.59 - - [01/Nov/2000:00:00:37 -0500]
"GET /gfx2/page/home.gif HTTP/1.1" 200 2285
The first thing we want to do is split this into fields. Remember that the split() function takes a regular
expression as its first argument. We'll use /\s/ to split the line at each whitespace character:
@fields = split(/\s/, $line);
This gives us 10 fields. The ones we're concerned with are the fourth field (time and date of request),
the seventh (the URL), and the ninth and 10th (HTTP status code and size in bytes of the server
response).
First, we'd like to make sure that we turn any request for a URL that ends in a slash (like /about/) into
a request for the index page from that directory (/about/index.html). We'll need to escape out the
slashes so that Perl doesn't mistake them for terminators in our s/// statement.
$fields[6] =~ s/\/$/\/index.html/;
This line is difficult to read, because anytime we come across a literal slash character we need to
escape it out. This problem is so common, it has acquired a name: leaning-toothpick syndrome.
Here's a useful trick for avoiding the leaning-toothpick syndrome: You can replace the slashes that
mark regular expressions and s/// statements with any other matching pair of characters, like { and }.
This allows us to write a more legible regex where we don't need to escape out the slashes:
$fields[6] =~ s{/$}{/index.html};
(If you want to use this syntax with a matching expression, you'll need to put a m in front of it. /foo/
would be rewritten as m{foo}.)
Now, we'll assume that any URL request that returns a status code of 200 (request OK) is a request
for the file type of the URL's extension (a request for /gfx/page/home.gif returns a GIF image). Any
URL request without an extension returns a plain-text file. Remember that the period is a
metacharacter, so we need to escape it out!
if ($fields[8] eq '200') {
if ($fields[6] =~ /\.([a-z]+)$/i) {
$type_requests{$1}++;
} else {
$type_requests{'txt'}++;
}
}
Next, we want to retrieve the hour each request took place. The hour is the first string in $fields[3] that
will be two digits surrounded by colons, so all we need to do is look for that. Remember that Perl will
stop when it finds the first match in a string:
# Log the hour of this request
$fields[3] =~ /:(\d{2}):/;
$hour_requests{$1}++;
Finally, let's rewrite our original report() sub. We're doing the same thing over and over (printing a
section header and the contents of that section), so we'll break that out into a new sub. We'll call the
new sub report_section():
sub report {
print ``Total bytes requested: '', $bytes, ``\n''; print "\n";
report_section("URL requests:", %url_requests);
report_section("Status code results:", %status_requests);
report_section("Requests by hour:", %hour_requests);
report_section("Requests by file type:", %type_requests);
}
The new report_section() sub is very simple:
sub report_section {
my ($header, %type) = @_; print $header, "\n";
for $i (sort keys %type) {
print $i, ": ", $type{$i}, "\n";
}
print "\n";
}
We use the keys function to return a list of the keys in the %type hash, and the sort function to put it in
alphabetic order. We'll play with sort a bit more in the next article.
Play around!
As usual, here are some sample exercises:
1. A rule of good writing is `àvoid the passive voice.'' Instead of The report was read by Carl, say Carl
read the report. Write a program that reads a file of sentences (one per line), detects and eliminates
the passive voice, and prints the result. (Don't worry about irregular verbs or capitalization, though.)
Sample solution. Sample test sentences.
2. You have a list of phone numbers. The list is messy, and the only thing you know is that there are
either seven or 10 digits in each number (the area code is optional), and if there's an extension, it will
show up after an ``x'' somewhere on the line. ``416 555-1212,'' ``5551300X40'' and ``(306) 555.5000
ext 40'' are all possible. Write a fix_phone() sub that will turn all of these numbers into the standard
format ``(123) 555-1234'' or ``(123) 555-1234 Ext 100,'' if there is an extension. Assume that the
default area code is ``123.''
What is CGI?
The Web is based on a client-server model: your browser (the client) making requests to a Web
server. Most of these are simple requests for documents or images, which the server delivers to the
browser for display.
Of course, sometimes you want the server to do more than just dump the contents of a file. You'd like
to do something with a server-side program - whether that "something" is using Web-based e-mail,
looking up a phone number in a database or ordering a copy of Evil Geniuses in a Nutshell for your
favorite techie. This means the browser must be able to send information (an e-mail address, a name
to look up, shipping information for a book) to the server, and the server must be able to use that
information and return the results to the user.
The standard for communication between a user's Web browser and a server-side program running
on the Web server is called CGI, or Common Gateway Interface. It is supported by all popular Web
server software. To get the most out of this article, you will need to have a server that supports CGI.
This may be a server running on your desktop machine or an account with your ISP (though probably
not a free Web-page service). If you don't know whether you have CGI capabilities, ask your ISP or a
local sysadmin how to set things up.
Notice that I haven't described how CGI works; that's because you don't need to know. There's a
standard Perl module called CGI.pm that will handle the CGI protocol for you. CGI.pm is part of the
core Perl distribution, and any properly installed Perl should have it available.
Telling your CGI program that you want to use the CGI module is as simple as this:
use CGI ':standard';
The use CGI ':standard'; statement tells Perl that you want to use the CGI.pm module in your
program. This will load the module and make a set of CGI functions available for your code.
A Real CGI Program
Let's write our first real CGI program. Instead of doing something complex, we'll write something that
will simply throw back whatever we throw at it. We'll call this script backatcha.cgi:
print header();
print start_html();
for $i (param()) {
print "<b>", $i, "</b>: ", param($i), "<br>\n";
}
print end_html();
If you've never used HTML, the pair of <b> and </b> tags mean "begin bold" and "end bold",
respectively, and the <br> tag means "line break." (A good paper reference to HTML is O'Reilly's
HTML & XHTML: The Definitive Guide, and online, I like the Web Design Group.)
Install this program on your server and do a test run. (If you don't have a Web server of your own,
we've put a copy online for you here.) Here's a short list of what you do to install a CGI program:
1. Make sure the program is placed where your Web server will recognize it as a CGI script. This
may be a special cgi-bin directory or making sure the program's filename ends in .pl or .cgi. If
you don't know where to place the program, your ISP or sysadmin should.
2. Make sure the program can be run by the server. If you are using a Unix system, you may
have to give the Web-server user read and execute permission for the program. It's easiest to
give these permissions to everybody by using chmod filename 755.
3. Make a note of the program's URL, which will probably be something like http://server
name/cgi-bin/backatcha.cgi) and go to that URL in your browser. (Take a guess what you
should do if you don't know what the URL of the program is. Hint: It involves the words "ask,"
"your" and "ISP.")
If this works, you will see in your browser ... a blank page! Don't worry, this is what is supposed to
happen. The backatcha.cgi script throws back what you throw at it, and we haven't thrown anything at
it yet. We'll give it something to show us in a moment.
If it didn't work, you probably saw either an error message or the source code of the script. We'll try to
diagnose these problems in the next section.
Uh-Oh!
If you saw an error message, your Web server had a problem running the CGI program. This may be
a problem with the program or the file permissions.
First, are you sure the program has the correct file permissions? Did you set the file permissions on
your program to 755? If not, do it now. (Windows Web servers will have a different way of doing this.)
Try it again; if you see a blank page now, you're good.
Second, are you sure the program actually works? (Don't worry, it happens to the best of us.) Change
the use CGI line in the program to read:
use CGI ':standard', '-debug';
Now run the program from the command line. You should see the following:
(offline mode: enter name=value pairs on standard input)
This message indicates that you're testing the script. You can now press Ctrl-D to tell the script to
continue running without telling it any form items.
If Perl reports any errors in the script, you can fix them now.
(The -debug option is incredibly useful. Use it whenever you have problems with a CGI program, and
ignore it at your peril.)
The other common problem is that you're seeing the source code of your program, not the result of
running your program. There are two simple problems that can cause this.
First, are you sure you're going through your Web server? If you use your browser's "load local file"
option (to look at something like /etc/httpd/cgi-bin/backatcha.cgi instead of something like
http://localhost/cgi-bin/backatcha.cgi), you aren't even touching the Web server! Your browser is doing
what you "wanted" to do: loading the contents of a local file and displaying them.
Second, are you sure the Web server knows it's a CGI program? Most Web server software will have
a special way of designating a file as a CGI program, whether it's a special cgi-bin directory, the .cgi or
.pl extension on a file, or something else. Unless you live up to these expectations, the Web server
will think the program is a text file, and serve up your program's source code in plain-text form. Ask
your ISP for help.
CGI programs are unruly beasts at the best of times; don't worry if it takes a bit of work to make them
run properly.
Making the Form Talk Back
At this point, you should have a working copy of backatcha.cgi spitting out blank pages from a Web
server. Let's make it actually tell us something. Take the following HTML code and put it in a file:
<FORM ACTION="putyourURLhere" METHOD=GET>
<P>What is your favorite color? <INPUT NAME="favcolor"></P>
<INPUT TYPE=submit VALUE="Send form">
lt;/FORM>
Be sure to replace putyourURLhere with the actual URL of your copy of backatcha.cgi! If you want,
you can use the copy installed here at Perl.com.
This is a simple form. It will show a text box where you can enter your favorite color and a "submit"
button that sends your information to the server. Load this form in your browser and submit a favorite
color. You should see this returned from the server:
favcolor: green
CGI functions
The CGI.pm module loads several special CGI functions for you. What are these functions?
The first one, header(), is used to output any necessary HTTP headers before the script can display
HTML output. Try taking this line out; you'll get an error from the Web server when you try to run it.
This is another common source of bugs!
The start_html() function is there for convenience. It returns a simple HTML header for you. You can
pass parameters to it by using a hash, like this:
print $cgi->start_html( -title => "My document" );
(The end_html() method is similar, but outputs the footers for your page.)
Finally, the most important CGI function is param(). Call it with the name of a form item, and a list of
all the values of that form item will be returned. (If you ask for a scalar, you'll only get the first value,
no matter how many there are in the list.)
$yourname = param("firstname");
print "<P>Hi, $yourname!</P>\n";
If you call param() without giving it the name of a form item, it will return a list of all the form items that
are available. This form of param() is the core of our backatcha script:
for $i (param()) {
print "<b>$i</b>: ", param($i), "<br>\n";
}
Remember, a single form item can have more than one value. You might encounter code like this on
the Web site of a pizza place that takes orders over the Web:
<P>Pick your toppings!<BR>
<INPUT TYPE=checkbox NAME=top VALUE=pepperoni> Pepperoni <BR>
<INPUT TYPE=checkbox NAME=top VALUE=mushrooms> Mushrooms <BR>
<INPUT TYPE=checkbox NAME=top VALUE=ham> Ham <BR>
</P>
Someone who wants all three toppings would submit a form where the form item top has three values:
"pepperoni," "mushrooms" and "ham." The server-side code might include this:
print "<P>You asked for the following pizza toppings: ";
@top = param("top");
for $i (@top) {
print $i, ". ";
}
print "</P>";
Now, here's something to watch out for. Take another look at the pizza-topping HTML code. Try
pasting that little fragment into the backatcha form, just above the <INPUT TYPE=submit...> tag.
Enter a favorite color, and check all three toppings. You'll see this:
favcolor: burnt sienna
top: pepperonimushroomsham
Why did this happens? When you call param('name'), you get back a list of all of the values for that
form item. This could be considered a bug in the backatcha.cgi script, but it's easily fixed - use join() to
separate the item values:
print "<b>$i</b>: ", join(', ', param($i)), "<br>\n";
or call C<param()> in a scalar context first to get only the first value:
$j = param($i);
print "<b>$i</b>: $j
\n";
Always keep in mind that form items can have more than one value!
Our Second Script
So now we know how to build a CGI program, and we've seen a simple example. Let's write
something useful. In the last article, we wrote a pretty good HTTP log analyzer. Why not Web-enable
it? This will allow you to look at your usage figures from anywhere you can get to a browser.
Download the source code for the HTTP log analyzer
First, let's decide what we want to do with our analyzer. Instead of showing all of the reports we
generate at once, we'll show only those the user selects. Second, we'll let the user choose whether
each report shows the entire list of items, or the top 10, 20 or 50 sorted by access count.
We'll use a form such as this for our user interface:
<FORM ACTION="/cgi-bin/http-report.pl" METHOD=POST>
<P>Select the reports you want to see:</P>
<P><INPUT TYPE=checkbox NAME=report VALUE=url>URLs requested<BR>

<INPUT TYPE=checkbox NAME=report VALUE=status>Status codes<BR>
<INPUT TYPE=checkbox NAME=report VALUE=hour>Requests by hour<BR>
<INPUT TYPE=checkbox NAME=report VALUE=type>File types
</P>
<P><SELECT NAME="number">
<OPTION VALUE="ALL">Show all
<OPTION VALUE="10">Show top 10
</SELECT></P>
<INPUT TYPE=submit VALUE="Show report">

</FORM>
(Remember that you may need to change the URL!)
We're sending two different types of form item in this HTML page. One is a series of checkbox
widgets, which set values for the form item report. The other is a single drop-down list which will
assign a single value to number: either ALL, 10, 20 or 50.
Take a look at the original HTTP log analyzer. We'll start with two simple changes. First, the original
program gets the filename of the usage log from a command-line argument:
# We will use a command line argument to determine the log filename.
$logfile = $ARGV[0];
We obviously can't do that now, since the Web server won't allow us to enter a command line for our
CGI program! Instead, we'll hard-code the value of $logfile. I'll use "/var/log/httpd/access_log" as a
sample value.
$logfile = "/var/log/httpd/access_log";
Second, we must make sure that we output all the necessary headers to our Web server before we
print anything else:
print header();
print start_html( -title => "HTTP Log report" );
Now look at the report() sub from our original program. It has one problem, relative to our new goals:
It outputs all the reports instead of only the ones we've selected. We'll rewrite report() so that it will
cycle through all the values of the report form item and show the appropriate report for each.
sub report {
for $i (param('report')) {
if ($i eq 'url') {
report_section("URL requests", %url_requests);
} elsif ($i eq 'status') {
report_section("Status code requests", %status_requests);
} elsif ($i eq 'hour') {
report_section("Requests by hour", %hour_requests);
} elsif ($i eq 'type') {
report_section("Requests by file type", %type_requests);
}
}
}
Finally, we rewrite the report_section() sub to output HTML instead of plain text. (We'll discuss the
new way we're using sort in a moment.)
sub report_section {
my ($header, %type) = @_;
my (@type_keys);
# Are we sorting by the KEY, or by the NUMBER of accesses?

if (param('number') ne 'ALL') {
@type_keys = sort { $type{$b} <=> $type{$a}; } keys %type;
# Chop the list if we have too many results

if ($#type_keys > param('number') - 1) {
$#type_keys = param('number') - 1;
}
} else {
@type_keys = sort keys %type;
}
# Begin a HTML table

print "<TABLE>\n";
# Print a table row containing a header for the table

print "<TR><TH COLSPAN=2>", $header, "</TH></TR>\n";
# Print a table row containing each item and its value

for $i (@type_keys) {
print "<TR><TD>", $i, "</TD><TD>", $type{$i}, "</TD></TR>\n";
}
# Finish the table

print "</TABLE>\n";
}
Sorting
Perl allows you to sort lists with the sort keyword. By default, the sort will happen alphanumerically:
numbers before letters, uppercase before lowercase. This is sufficient 99 percent of the time. The
other 1 percent of the time, you can write a custom sorting routine for Perl to use.
This sorting routine is just like a small sub. In it, you compare two special variables, $a and $b, and
return one of three values depending on how you want them to show up in the list. Returning -1
means "$a should come before $b in the sorted list," 1 means "$b should come before $a in the
sorted list" and 0 means "they're equal, so I don't care which comes first." Perl will run this routine to
compare each pair of items in your list and produce the sorted result.
For example, if you have a hash called %type, here's how you might sort its keys in descending order
of their values in the hash.
sort {
if ($type{$b} > $type{$a}) { return 1; }
if ($type{$b} < $type{$a}) { return -1; }
return 0;
} keys %type;
In fact, numeric sorting happens so often, Perl gives you a convenient shorthand for it: the <=>
operator. This operator will perform the above comparison between two values for you and return the
appropriate value. That means we can rewrite that test as:
sort { $type{$b} <=> $type{$a}; } keys %type
(And this, in fact, is what we use in our log analyzer.)
You can also compare strings with sort. The lt and gt operators are the string equivalents of < and >,
and cmp will perform the same test as <=>. (Remember, string comparisons will sort numbers before
letters and uppercase before lowercase.)
For example, you have a list of names and phone numbers in the format "John Doe 555-1212." You
want to sort this list by the person's last name, and sort by first name when the last names are the
same. This is a job made for cmp!
@sorted = sort {
($c) = ($a =~ / (\w+)/);
($d) = ($b =~ / (\w+)/);
if ($c eq$d) { # Last names are the same, sort on first name
($c) = ($a =~ /^(\w+)/);
($d) = ($b =~ /^(\w+)/);
return $c cmp $d;
} else {
return $c cmp $d;
}
} @phone_numbers;
for $i (@sorted) { print $i, "\n"; }
Trust No One
Now that we know how CGI programs can do what you want, let's make sure they won't do what you
don't want. This is harder than it looks, because you can't trust anyone to do what you expect.
Here's a simple example: You want to make sure the HTTP log analyzer will never show more than 50
items per report, because it takes too long to send larger reports to the user. The easy thing to do
would be to eliminate the "ALL" line from our HTML form, so that the only remaining options are 10,
20 and 50. It would be very easy - and wrong.
Download the source code for the HTTP analyzer with security enhancements.
We saw that you can modify HTML forms when we pasted the pizza-topping sample code into our
backatcha page. You can also use the URL to pass form items to a script - try going to
http://www.perl.com/2000/12/backatcha.cgi?itemsource=URL&typedby=you in your browser.
Obviously, if someone can do this with the backatcha script, they can also do it with your log analyzer
and stick any value for number in that they want: "ALL" or "25000", or "four score and seven years
ago."
Your form doesn't allow this, you say. Who cares? People will write custom HTML forms to exploit
weaknesses in your programs, or will just pass bad form items to your script directly. You cannot trust
anything users or their browsers tell you.
You eliminate these problems by knowing what you expect from the user, and disallowing everything
else. Whatever you do not expressly permit is totally forbidden. Secure CGI programs consider
everything guilty until it is made innocent.
For example, we want to limit the size of reports from our HTTP log analyzer. We decide that means
the number form item must have a value that is between 10 and 50. We'll verify it like this:
# Make sure that the "number" form item has a reasonable value
($number) = (param('number') =~ /(\d+)/);
if ($number < 10) {
$number = 10;
} elsif ($number > 50) {
$number = 50;
}
Of course, we also have to change the report_section() sub so it uses the $number variable. Now,
whether your user tries to tell your log analyzer that the value of number is "10," "200," "432023,"
"ALL" or "redrum," your program will restrict it to a reasonable value.
We don't need to do anything with report, because we only act when one of its values is something we
expected. If the user tries to enter something other than our expressly permitted values ("url," "status,"
"hour" or "type"), we just ignore it.
Use this sort of logic everywhere you know what the user should enter. You might use s/\D//g to
remove non-numeric characters from items that should be numbers (and then test to make sure
what's left is within your range of allowable numbers!), or /^\w+$/ to make sure that the user entered a
single word.
All of this has two significant benefits. First, you simplify your error-handling code, because you make
sure as early in your program as possible that you're working with valid data. Second, you increase
security by reducing the number of "impossible" values that might help an attacker compromise your
system or mess with other users of your Web server.
Don't just take my word for it, though. The CGI Security FAQ has more information about safe CGI
programming in Perl than you ever thought could possibly exist, including a section listing some
security holes in real CGI programs.
Play Around!
You should now know enough about CGI programming to write a useful Web application. (Oh, and
you learned a little bit more about sorting and comparison.)
1. Write the quintessential CGI program: a guestbook. Users enter their name, e-mail address
and a short message, which is appended to an HTML file for all to see.
Be careful! Never trust the user! A good beginning precaution is to disallow all HTML by either
removing < and > characters from all of the user's information or replacing them with the <
and > character entities.
Use substr(), too, to cut anything the user enters down to a reasonable size. Asking for a
"short" message will do nothing to prevent the user dumping a 500k file into the message
field!
2. Write a program that plays tic-tac-toe against the user. Be sure that the computer AI is in a
sub so it can be easily upgraded. (You'll probably need to study HTML a bit to see how to
output the tic-tac-toe board.)
What Is an Object?
Think back to the first article in this series, when we discussed the two basic data types in Perl, strings
and numbers. There's a third basic data type: the object.
Objects are a convenient way of packaging information with the things you actually do with that
information. The information an object contains is called its properties, and the things you can do with
that information are called methods.
For example, you might have an AddressEntry object for an address book program - this object would
contain properties that store a person's name, mailing address, phone number and e-mail address;
and methods that print a nicely formatted mailing label or allow you to change the person's phone
number.
During the course of this article, we'll build a small, but useful, class: a container for configuration file
information.
Our Goal
So far, we've put the code for setting various options in our programs directly in the program's source
code. This isn't a good approach. You may want to install a program and allow multiple users to run it,
each with their own preferences, or you may want to store common sets of options for later. What you
need is a configuration file to store these options.
We'll use a simple plain-text format, where name and value pairs are grouped in sections, and
sections are indicated by a header name in brackets. When we want to refer to the value of a specific
key in our configuration file, we call the key section.name. For instance, the value of author.firstname
in this simple file is ``Doug:''
[author]
firstname=Doug
lastname=Sheppard
[site]
name=Perl.com
url=http://www.perl.com/
(If you used Windows in the ancient days when versions had numbers, not years, you'll recognize this
as being similar to the format of INI files.)
Now that we know the real-world purpose of our module, we need to think about what properties and
methods it will have: What do TutorialConfig objects store, and what can we do with them?
The first part is simple: We want the object's properties to be the values in our configuration file.
The second part is a little more complex. Let's start by doing the two things we need to do: read a
configuration file, and get a value from it. We'll call these two methods read and get. Finally, we'll add
another method that will allow us to set or change a value from within our program, which we'll call
set. These three methods will cover nearly everything we want to do.
Starting Off
We'll use the name TutorialConfig for our configuration file class. (Class names are normally named in
this InterCapitalized style.) Since Perl looks for a module by its filename, this means we'll call our
module file TutorialConfig.pm.
Put the following into a file called TutorialConfig.pm:
package TutorialConfig;
warn "TutorialConfig is successfully loaded!\n";

1;
(I'll be sprinkling debugging statements throughout the code. You can take them out in practice. The
warn keyword is useful for warnings - things that you want to bring to the user's attention without
ending the program the way die would.)
The package keyword tells Perl the name of the class you're defining. This is generally the same as
the module name. (It doesn't have to be, but it's a good idea!) The 1; will return a true value to Perl,
which indicates that the module was loaded successfully.
You now have a simple module called TutorialConfig, which you can use in your code with the use
keyword. Put the following into a very simple, one-line program:
use TutorialConfig;
When we run this program, we see the following:
TutorialConfig is successfully loaded!
What Does Our Object Do?
Before we can create an object, we need to know how to create it. That means we must write a
method called new that will set up an object and return it to us. This is also where you put any special
initialization code that you might need to run for each object when it is created.
The new method for our TutorialConfig class looks like this, and goes into TutorialConfig.pm right after
the package declaration:
sub new {
my ($class_name) = @_;
my ($self) = {};
warn "We just created our new variable...\n ";
bless ($self, $class_name);

warn "and now it's a $class_name object!\n";
$self->{'_created'} = 1;
return $self;
}
(Again, you won't need those warn statements in actual practice.)
Let's break this down line by line.
First, notice that we define methods by using sub. (All methods are really just a special sort of sub.)
When we call new, we pass it one parameter: the type of object we want to create. We store this in a
private variable called $class_name. (You can also pass extra parameters to new if you want. Some
modules use this for special initialization routines.)
Next, we tell Perl that $self is a hash. The syntax my ($self) = {}; is a special idiom that's used mostly
in Perl object programming, and we'll see how it works in some of our methods. (The technical term is
that $self is an anonymous hash, if you want to read more about it elsewhere.)
Third, we use the bless function. You give this function two parameters: a variable that you want to
make into an object, and the type of object you want it to be. This is the line that makes the magic
happen!
Fourth, we'll set a property called ``_created''. This property isn't really that useful, but it does show
the syntax for accessing the contents of an object: $object_name->{property_name}.
Finally, now that we've made $self into a new TutorialConfig object, we return it.
Our program to create a TutorialConfig object looks like this:
use TutorialConfig;
$tut = new TutorialConfig;
(You don't need to use parentheses here, unless your object's new method takes any extra
parameters. But if you feel more comfortable writing $tut = new TutorialConfig();, it'll work just as well.)
When you run this code, you'll see:
TutorialConfig is successfully loaded!
We just created the variable ...
and now it's a TutorialConfig object!
Now that we have a class and we can create objects with it, let's make our class do something!
Our Goal, Part 2
Look at our goals again. We need to write three methods for our TutorialConfig module: read, get and
set.
The first method, read, obviously requires that we tell it what file we want to read. Notice that when we
write the source code for this method, we must give it two parameters. The first parameter is the
object we're using, and the second is the filename we want to use. We'll use return to indicate
whether the file was successfully read.
sub read {
my ($self, $file) = @_;
my ($line, $section);
open (CONFIGFILE, $file) or return 0;
# We'll set a special property

# that tells what filename we just read.
$self->{'_filename'} = $file;
while ($line = <CONFIGFILE>) {
# Are we entering a new section?

if ($line =~ /^\[(.*)\]/) {
$section = $1;
} elsif ($line =~ /^([^=]+)=(.*)/) {
my ($config_name, $config_val) = ($1, $2);
if ($section) {
$self->{"$section.$config_name"} = $config_val;
} else {
$self->{$config_name} = $config_val;
}
}
}
close CONFIGFILE;
return 1;
}
Now that we've read a configuration file, we need to look at the values we just read. We'll call this
method get, and it doesn't have to be complex:
sub get {
my ($self, $key) = @_;
return $self->{$key};
}
These two methods are really all we need to begin experimenting with our TutorialConfig object. Take
the module and sample configuration file from above (or download the configuration file here and the
module here), put it in a file called tutc.txt, and then run this simple program:
use TutorialConfig;

$tut->read('tutc.txt') or die "Couldn't read config file: $!";
print "The author's first name is ",

$tut->get('author.firstname'),
".\n";
(Notice the syntax for calling an object's methods: $object->method(parameters).)
When you run this program, you'll see something like this:
TutorialConfig has been successfully loaded!

We just created the variable...
and now it's a TutorialConfig object!
The author's first name is Doug.
We now have an object that will read configuration files and show us values inside those files. This is
good enough, but we've decided to make it better by writing a set method that allows us to add or
change configuration values from within our program:
sub set {
my ($self, $key, $value) = @_;
$self->{$key} = $value;
}
Now let's test it out:
use TutorialConfig;
$tut->read('tutc.txt') or die "Can't read config file: $!";

$tut->set('author.country', 'Canada');
print $tut->get('author.firstname'), " lives in ",

$tut->get('author.country'), ".\n";
These three methods (read, get and set) are everything we'll need for our TutorialConfig.pm module.
More complex modules might have dozens of methods!
Encapsulation
You may be wondering why we have get and set methods at all. Why are we using $tut-
>set('author.country', 'Canada') when we could use $tut->{'author.country'} = 'Canada' instead? There
are two reasons to use methods instead of playing directly with an object's properties.
First, you can generally trust that a module won't change its methods, no matter how much their
implementation changes. Someday, we might want to switch from using text files to hold configuration
information to using a database like MySQL or Postgres. Our new TutorialConfig.pm module might
have new, read, get and set methods that look like this:
sub new {
my ($class) = @_;
my ($self) = {};
bless $self, $class;
return $self;
}
sub read {
my ($self, $file) = @_;
my ($db) = database_connect($file);
if ($db) {
$self->{_db} = $db;
return $db;
}
return 0;
}
sub get {
my ($self, $key) = @_;
my ($db) = $self->{_db};
my ($value) = database_lookup($db, $key);

return $value;
}
sub set {
my ($self, $key, $value) = @_;
my ($db) = $self->{_db};
my ($status) = database_set($db, $key, $value);

return $status;
}
(Our module would define the database_connect, database_lookup and database_set routines
elsewhere.)
Even though the entire module's source code has changed, all of our methods still have the same
names and syntax. Code that uses these methods will continue working just fine, but code that
directly manipulates properties will break!
For instance, let's say you have some code that contains this line to set a configuration value:
$tut->{'author.country'} = 'Canada';
This works fine with the original TutorialConfig.pm module, because when you call $tut-
>get('author.country'), it looks in the object's properties and returns ``Canada'' just like you expected.
So far, so good. However, when you upgrade to the new version that uses databases, the code will no
longer return the correct result. Instead of get() looking in the object's properties, it'll go to the
database, which won't contain the correct value for `àuthor.country''! If you'd used $tut-
>set('author.country', 'Canada') all along, things would work fine.
As a module author, writing methods will let you make changes (bug fixes, enhancements, or even
complete rewrites) without requiring your module's users to rewrite any of their code.
Second, using methods lets you avoid impossible values. You might have an object that takes a
person's age as a property. A person's age must be a positive number (you can't be -2 years old!), so
the age() method for this object will reject negative numbers. If you bypass the method and directly
manipulate $obj->{'age'}, you may cause problems elsewhere in the code (a routine to calculate the
person's birth year, for example, might fail or produce an odd result).
As a module author, you can use methods to help programmers who use your module write better
software. You can write a good error-checking routine once, and it will be used many times.
(Some languages, by the way, enforce encapsulation, by giving you the ability to make certain
properties private. Perl doesn't do this. In Perl, encapsulation isn't the law, it's just a very good idea.)
Play Around!
1. Our TutorialConfig.pm module could use a method that will write a new configuration file to any
filename you desire. Write your own write() method (use keys %$self to get the keys of the object's
properties). Be sure to use or to warn if the file couldn't be opened!
2. Write a BankAccount.pm module. Your BankAccount object should have deposit, withdraw, and
balance methods. Make the withdraw method fail if you try to withdraw more money than you have, or
deposit or withdraw a negative amount of money.
3. CGI.pm also lets you use objects if you want. (Each object represents a single CGI query.) The
method names are the same as the CGI functions we used in the last article:
use CGI;
$cgi = new CGI;
print $cgi->header(), $cgi->start_html();

print "The 'name' parameter is ", $cgi->param('name'), ".\n";
print $cgi->end_html();
Try rewriting one of your CGI programs to use CGI objects instead of the CGI functions.
4. A big advantage of using CGI objects is that you can store and retrieve queries on disk. Take a look
in the CGI.pm documentation to learn how to use the save() method to store queries, and how to pass
a filehandle to new to read them from disk. Try writing a CGI program that saves recently used
queries for easy retrieval.
Doing It Right the First Time
Perl is a useful tool, which many people use to write some good software. But like all programming
languages, Perl can also be used to create bad software. Bad software contains bugs, has security
holes and is hard to fix or extend.
Fortunately, Perl offers you many ways to increase the quality of the programs you write. In this last
installment in the Beginner's Intro series, we'll take a look at a few of them.
Comments
In the first part of this series, we looked at the lowly #, which indicates a comment. Comments are
your first line of defense against bad software, because they help answer the two questions that
people always have when they look at source code: What does this program do and how does it do it?
Comments should always be part of any software you write. Complex code with no comments is not
automatically evil, but bring some holy water just in case.
Good comments are short, but instructive. They tell you things that aren't clear from reading the code.
For example, here's some obscure code that could use a comment or two:
for $i (@q) {
my ($j) = fix($i);
transmit($j);
}
Bad comments would look like this:
for $i (@q) { # @q is list from last sub
my ($j) = fix($i); # Gotta fix $j...
transmit($j); # ...and then it goes over the wire
}
Notice that you don't learn anything from these comments. my ($j) = fix($i); # Gotta fix $j... is
meaningless, the equivalent of a dictionary that contains a definition like widget (n.): A widget. What is
@q? Why do you have to fix its values? That may be clear from the larger context of the program, but
you don't want to skip all around a program to find out what one little line does!
Here's something a little clearer. Notice that we actually have fewer comments, but they're more
instructive:
# Now that we've got prices from database, let's send them to the buyer
for $i (@q) {
my ($j) = fix($i); # Add local taxes, perform currency exchange
transmit($j);
}
Now it's obvious where @q comes from, and what fix() does.
Warnings
Comments are good, but the most important tool for writing good Perl is the ``warnings'' flag, the -w
command line switch. You can turn on warnings by placing -w on the first line of your programs like
so:
Or, if you're running a program from the command line, you can use -w there, as in perl -w
myprogram.pl.
Turning on warnings will make Perl yelp and complain at a huge variety of things that are almost
always sources of bugs in your programs. Perl normally takes a relaxed attitude toward things that
may be problems; it assumes that you know what you're doing, even when you don't.
Here's an example of a program that Perl will be perfectly happy to run without blinking, even though
it has an error on almost every line! (See how many you can spot.)
$filename = "./logfile.txt";
open (LOG, $fn);
print LOG "Test\n";
close LOGFILE;
Now, add the -w switch to the first line, and run it again. You should see something like this:
Name ``main::filename'' used only once: possible typo at ./a6-warn.pl line 3. Name ``main::LOGFILE''
used only once: possible typo at ./a6-warn.pl line 6. Name ``main::fn'' used only once: possible typo at
./a6-warn.pl line 4. Use of uninitialized value at ./a6-warn.pl line 4. print on closed filehandle
main::LOG at ./a6-warn.pl line 5.
Here's what each of these errors means:
1. Name ``main::filename'' used only once: possible typo at ./a6-warn.pl line 3. and Name ``main::fn''
used only once: possible typo at ./a6-warn.pl line 4. Perl notices that $filename and $fn both only get
used once, and guesses that you've misspelled or misnamed one or the other. This is because this
almost always happens because of typos or bugs in your code, like using $filenmae instead of
$filename, or using $filename throughout your program except for one place where you use $fn (like
in this program).
2. Name ``main::LOGFILE'' used only once: possible typo at ./a6-warn.pl line 6. In the same way that
we made our $filename typo, we mixed up the names of our filehandles: We use LOG for the
filehandle while we're writing the log entry, but we try to close LOGFILE instead.
3. Use of uninitialized value at ./a6-warn.pl line 4. This is one of Perl's more cryptic complaints, but it's
not difficult to fix. This means that you're trying to use a variable before you've assigned a value to it,
and that is almost always an error. When we first mentioned $fn in our program, it hadn't been given a
value yet. You can avoid this type of warning by always setting a default value for a variable before
you first use it.
4. print on closed filehandle main::LOG at ./a6-warn.pl line 5. We didn't successfully open LOG,
because $fn was empty. When Perl sees that we are trying to print something to the LOG filehandle, it
would normally just ignore it and assume that we know what we're doing. But when -w is enabled,
Perl warns us that it suspects there's something afoot.
So, how do we fix these warnings? The first step, obviously, is to fix these problems in our script. (And
while we're at it, I deliberately violated our rule of always checking if open() succeeded! Let's fix that,
too.) This turns it into:
$filename = "./logfile.txt";
open (LOG, $filename) or die "Couldn't open $filename: $!";
print LOG "Test\n";
close LOG;
Now, we run our corrected program, and get this back from it:
Filehandle main::LOG opened only for input at ./a6-warn2.pl line 5.
Where did this error come from? Look at our open(). Since we're not preceding the filename with > or
>>, Perl opens the file for reading, but in the next line we're trying to write to it with a print. Perl will
normally let this pass, but when warnings are in place, it alerts you to possible problems. Change line
4 to this instead and everything will be great:
open (LOG, ">>$filename") or die "Couldn't open $filename: $!";
The <-w> flag is your friend. Keep it on at all times. You may also want to read the <perldiag> man
page, which contains a listing of all the various messages (including warnings) Perl will spit out when
it encounters a problem. Each message is accompanied by a detailed description of what the
message means and how to fix it.
Taint
Using -w will help make your Perl programs correct, but it won't help make them secure. It's possible
to write a program that doesn't emit a single warning, but is totally insecure!
For example, let's say that you are writing a CGI program that needs to write a user's comment to a
user-specified file. You might use something like this:
$file = param('file');
$comment = param('comment');
unless ($file) { $file = 'file.txt'; }
unless ($comment) { $comment = 'No comment'; }
open (OUTPUT, ">>/etc/webstuff/storage/" . $file) or die "$!";
print OUTPUT $comment . "\n";
close OUTPUT;
print header, start_html;
print"<P>Thanks!</P>\n";
print end_html;
If you read the CGI programming installment, alarm bells are already ringing loud enough to deafen
you. This program trusts the user to specify only a ``correct'' filename, and you know better than to
trust the user. But nothing in this program will cause -w to bat an eye; as far as warnings are
concerned, this program is completely correct.
Fortunately, there's a way to block these types of bugs before they become a problem. Perl offers a
mechanism called taint that marks any variable that the user can possibly control as being insecure.
This includes user input, file input and environment variables. Anything that you set within your own
program is considered safe:
$taint = <STDIN>; # This came from user input, so it's tainted
$taint2 = $ARGV[1]; # The @ARGV array is considered tainted too.
$notaint= "Hi"; # But this is in your program... it's untainted
You enable taint checking with the -T flag, which you can combine with -w like so:
#!/usr/local/bin/perl -Tw
-T will prevent Perl from running most code that may be insecure. If you try to do various dangerous
things with tainted variables, like open a file for writing or use the system() or exec() functions to run
external commands, Perl will stop right away and complain.
You untaint a variable by running it through a regex with matching subexpressions, and using the
results from the subexpressions. Perl will consider $1, $2 and so forth to be safe for your program.
For example, our file-writing CGI program may expect that ``sane'' filenames contain only the
alphanumeric characters that are matched by the \w metacharacter (this would prevent a malicious
user from passing a filename like ~/.bashrc, or even ../test). We'd use a filter like so:
$file = param('file');
if ($file) {
$file =~ /^(\w+)$/;
$file = $1;
}
unless ($file) { $file = "file.txt"; }
Now, $file is guaranteed to be untainted. If the user passed us a filename, we don't use it until we've
made sure it matches only \w+. If there was no filename, then we specify a default in our program. As
for $comment, we never actually do anything that would cause Perl's taint checking to worry, so it
doesn't need to be checked to pass -T.
Stuff Taint Doesn't Catch
Be careful! Even when you've turned on taint checking, you can still write an insecure program.
Remember that taint only gets looked at when you try to modify the system, by opening a file or
running a program. Reading from a file will not trigger taintedness! A very common breed of security
hole exploits code that doesn't look very different from this small program:
$file = param('filename');
unless ($file) { $file = 'file.txt'; }
open (FILE, "</etc/webstuff/storage/" . $file) or die "$!";
print header();
while ($line = <FILE>) {
print $line;
}
close FILE;
Just imagine the joy when the ``filename'' parameter contains ../../../../../../etc/passwd. (If you don't see
the problem: On a Unix system, the /etc/passwd file contains a list of all the usernames on the
system, and may also contain an encrypted list of their passwords. This is great information for
crackers who want to get into a machine for further mischief.) Since you are only reading the file,
Perl's taint checking doesn't kick in. Similarly, print doesn't trigger taint checking, so you'll have to
write your own value-checking code when you write any user input to a file!
Taint is a good first step in security, but it's not the last.
use strict
Warnings and taint are two excellent tools for preventing your programs from doing bad things. If you
want to go further, Perl offers use strict. These two simple words can be put at the beginning of any
program:
#!/usr/local/bin/perl -wT
use strict;
A command like use strict is called a pragma. Pragmas are instructions to the Perl interpreter to do
something special when it runs your program. use strict does two things that make it harder to write
bad software: It makes you declare all your variables (``strict vars''), and it makes it harder for Perl to
mistake your intentions when you are using subs (``strict subs'').
If you only want to use one or two types of strictness in your program, you can list them in the use
strict pragma, or you can use a special no strict pragma to turn off any or all of the strictness you
enabled earlier.
use strict 'vars'; # We want to require variables to be declared
no strict 'vars'; # We'll go back to normal variable rules now
usestrict 'subs'; # We want Perl to distrust barewords (see below).
nostrict; # Turn it off. Turn it all off. Go away, strict.
(There's actually a third type of strictness - strict refs - which prevents you from using symbolic
references. Since we haven't really dealt with references, we'll concentrate on the other two types of
strictness.)
Strict vars
Perl is generally trusting about variables. It will alllow you to create them out of thin air, and that's what
we've been doing in our programs so far. One way to make your programs more correct is to use
strict vars, which means that you must always declare variables before you use them. You declare
variables by using the my keyword, either when you assign values to them or before you first mention
them:
my ($i, $j, @locations);
my $filename = "./logfile.txt";
$i = 5;
This use of my doesn't interfere with using it elsewhere, like in subs, and remember that a my variable
in a sub will be used instead of the one from the rest of your program:
my ($i, $j, @locations);
# ... stuff skipped ...
sub fix {
my ($q, $i) = @_; # This doesn't interfere with the program $i!
}
If you end up using a variable without declaring it, you'll see an error before your program runs:
use strict;
$i = 5;
print "The value is $i.\n";
When you try to run this program, you see an error message similar to Global symbol ``$i'' requires
explicit package name at a6-my.pl line 3. You fix this by declaring $i in your program:
use strict;
my $i= 5; # Or "my ($i); $i = 5;", if you prefer...
print "The value is $i.\n";
Keep in mind that some of what strict vars does will overlap with the -w flag, but not all of it. Using the
two together makes it much more difficult, but not impossible, to use an incorrect variable name. For
example, strict vars won't catch it if you accidentally use the wrong variable:
my ($i, $ii) = (1, 2);
print 'The value of $ii is ', $i, "\n";
This code has a bug, but neither strict vars nor the -w flag will catch it.
Strict subs
During the course of this series, I've deliberately avoided mentioning all sorts of tricks that allow you
to write more compact Perl. This is because of a simple rule: readability always wins. Not only can
compactness make it difficult to read code, it can sometimes have weird side effects! The way Perl
looks up subs in your program is an example. Take a look at this pair of three-line programs:
$a = test_value;
print "First program: ", $a, "\n";
sub test_value { return "test passed"; }
$a = test_value;
print "Second program: ", $a, "\n";
The same program with one little, insignificant line moved, right? In both cases we have a test_value()
sub and we want to put its result into $a. And yet, when we run the two programs, we get two different
results:
First program's result: test_value
Second program's result: test passed
The reason why we get two different results is a little convoluted.
In the first program, at the point we get to $a = test_value;, Perl doesn't know of any test_value() sub,
because it hasn't gotten that far yet. This means that test_value is interpreted as if it were the string
'test_value'.
In the second program, the definition of test_value() comes before the $a = test_value; line. Since
Perl has a test_value() sub to call, that's what it thinks test_value means.
The technical term for isolated words like test_value that might be subs and might be strings
depending on context, by the way, is bareword. Perl's handling of barewords can be confusing, and it
can cause two different types of bug.
Want a Sub, Get a String
The first type of bug is what we encountered in our first program, which I'll repeat here:
$a = test_value;
print "First program: ", $a, "\n";
Remember that Perl won't look forward to find test_value(), so since it hasn't already seen
test_value(), it assumes that you want a string. Strict subs will cause this program to die with an error:
use strict;
my $a = test_value;
print "Third program: ", $a, "\n";
sub test_value { "test passed"; }
(Notice the my put in to make sure that strict vars won't complain about $a.)
Now you get an error message like Bareword ``test_value'' not allowed while ``strict subs'' in use at
./a6-strictsubs.pl line 3. This is easy to fix, and there are two ways to do it:
1. Use parentheses to make it clear you're calling a sub. If Perl sees $a = test_value();, it will assume
that even if it hasn't seen test_value() defined yet, it will sometime between now and the end of the
program. (If there isn't any test_value() in your program, Perl will die while it's running.) This is the
easiest thing to do, and often the most readable.
2. Declare your sub before you first use it, like this:
use strict;
sub test_value; # Declares that there's a test_value() coming later ...
my $a = test_value; # ...so Perl will know this line is okay.
print "Fourth program: ", $a, "\n";
sub test_value { return "test_passed"; }
Declaring your subs has the advantage of allowing you to maintain the $a = test_value; syntax if that's
what you find more readable, but it's also a little obscure. Other programmers may not see why you
have sub test_value; in your code.
Of course, you could always move the definition of your sub before the line where you want to call it.
This isn't quite as good as either of the other two methods, because now you are moving code around
instead of making your existing code clearer. Also, it can cause other problems, which we'll discuss
now ...
Want a String, Get a Sub
We've seen how use strict can help prevent an error where you intend to call a sub, but instead get a
string value. It also helps prevent the opposite error: wanting a string value, but calling a sub instead.
This is a more dangerous class of bug, because it can be very hard to trace, and it often pops up in
the most unexpected places. Take a look at this excerpt from a long program:
use strict;
use SomeModule;
use SomeOtherModule;
use YetAnotherModule;
# ... (and then there's hundreds of lines of code) ...
# Now we get to line 400 of the program, which tests if we got an "OK"
# before we act on a request from the user.
if ($response_code eq OK) {
act_on($request);
} else {
throw_away($request);
}
This program works without a hitch for a long time, because Perl sees the bareword OK and
considers it to be a literal string. Then, two years later someone needs to add code to make this
program understand HTTP status codes. They stick this in at line 2, or line 180, or line 399 (it doesn't
matter exactly where, just that it comes before line 400):
sub OK { return 200; } # HTTP "request ok, response follows" code
sub NOT_FOUND { return 404; } # "URL not found" code
sub SERVER_ERROR { return 500; } # "Server can't handle request"
Take a moment to guess what happens to our program now. Try to work the word ``disaster'' into it.
Thanks to this tiny change, our program now throws away every request that comes in to it. The if
($response eq OK) test now calls the OK() sub, which returns a value of 200. The if now fails every
time! The programmer, if they still have a job after this fiasco, must hunt through the entire program to
find out exactly when the behavior of if ($response eq OK) changed, and why.
By the way, if the programmer is really unlucky, that new OK() sub wouldn't even be in their code at
all, but defined somewhere in a new version of SomeOtherModule.pm that just got installed!
Barewords are dangerous because of this unpredictable behavior. use strict (or use strict 'subs')
makes them predictable, because barewords that might cause strange behavior in the future will
make your program die before they can wreak havoc.
The One Exception
There's one place where it's OK to use barewords even when you've turned on strict subs: when you
are assigning hash keys.
$hash{sample} = 6; # Same as $hash{'sample'} = 6
%other_hash = ( pie => 'apple' );
Barewords in hash keys are always interpreted as strings, so there is no ambiguity.
Is This Overkill?
There are times when using all of the quality enforcement functionality (or ``correctness police,'' if you
like to anthropmorphize) Perl offers seems like overkill. If you're just putting together a quick, three-
line tool that you'll use once and then never touch again, you probably don't care about whether it'll
run properly under use strict. When you're the only person who will run a program, you generally don't
care if the -T flag will show that you're trying to do something unsafe with a piece of user input.
Still, it's a good idea to use every tool at your disposal to write good software. Here are three reasons
to be concerned about correctness when you write just about anything:
1. One-off programs aren't. There are few programs worth writing that only get run once. Software
tools tend to accumulate, and get used. You'll find that the more you use a program, the more you
want it to do.
2. Other people will read your code. Whenever programmers write something really good, they tend to
keep it around, and give it to friends who have the same problem. More importantly, most projects
aren't one-person jobs; there are teams of programmers who need to work together, reading, fixing
and extennding one another's code. Unless your plans for the future include always working alone
and having no friends, you should expect that other people will someday read and modify your code.
3. You will read your code. Don't think you have a special advantage in understanding your code just
because you wrote it! Often you'll need to go back to software you wrote months or even years earlier
to fix it or extend it. During that time you'll have forgotten all those clever little tricks you came up with
during that caffeine-fueled all-nighter and all the little gotchas that you noticed but thought you would
fix later.
These three points all have one thing in common: Your programs will be rewritten and enhanced by
people who will appreciate every effort you make to make their job easier. When you make sure your
code is readable and correct, it tends to start out much more secure and bug-free, and it tends to stay
that way, too!
Play Around!
During the course of this series, we've only scratched the surface of what Perl can do. Don't take
these articles as being definitive - they're just an introduction! Read the perlfunc page to learn about
all of Perl's built-in functions and see what ideas they inspire. My biography page tells you how to get
in touch with me if you have any questions.

A Word About Operating Systems

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Word About Operating Systems

Uploaded by

Copyright:

Available Formats

Welcome to Perl.

print "$marx is my favorite Marx brother.\n";

unless ($bank_account > 0) {

unless ($pie eq 'apple') {

open (APPEND, ">>append.txt") or die "$! error trying to append";

# compound_interest_file.pl - the miracle of compound interest, part 2

# First, we'll set up the variables we want to use.

return int (($apr / 100) * $nest_egg * 100) / 100;

# We have two local variables, so we'll use my to declare them here.

# Calculate interest for each year.

$line = join("\t", $year, $nest_egg, $interest,

print REPORT $line;

for $check_line (@email_lines) {

$name = "$2 $1";

use CGI ':standard';

<P><INPUT TYPE=checkbox NAME=report VALUE=url>URLs requested<BR>

<INPUT TYPE=submit VALUE="Show report">

# Are we sorting by the KEY, or by the NUMBER of accesses?

# Chop the list if we have too many results

# Begin a HTML table

# Print a table row containing a header for the table

# Print a table row containing each item and its value

# Finish the table

warn "TutorialConfig is successfully loaded!\n";

bless ($self, $class_name);

open (CONFIGFILE, $file) or return 0;

# We'll set a special property

while ($line = <CONFIGFILE>) {

# Are we entering a new section?

$tut = new TutorialConfig;

print "The author's first name is ",

TutorialConfig has been successfully loaded!

$tut->read('tutc.txt') or die "Can't read config file: $!";

print $tut->get('author.firstname'), " lives in ",

my ($value) = database_lookup($db, $key);

my ($status) = database_set($db, $key, $value);

print $cgi->header(), $cgi->start_html();

Doing It Right the First Time

You might also like