Software Carpentry: Automated Builds (Draft Version 348)

file:///C:/home/swc/build/www/make.html

Automated Builds
prev Draft Version 348 (Tue Aug 23 11:52:49 2005) next

How Do You Rebuild A Program?

Easy enough to create overlap.class from overlap.java by running javac overlap.java But what if you have many source files?
javac *.java doesn't work with subdirectories And will be very slow for large programs Especially if you're recompiling repeatedly during a debugging session

Worse, what if some files rely on others?

Suppose Overlap.java uses Rect.java If you change both, but only recompile Overlap.java, you'll be running an out-of-date version of Rect.class Comment on this slide

Automate, Automate, Automate

Remember rule #2: anything worth repeating is worth automating
Computers are good at details, and don't get bored, so have them do repetitive tasks We need: A way to describe the tasks (what things to do) A way to specify the dependencies between them (when to do things)

Most widely used tool for this is called Make

Invented in 1975 by Stuart Feldman at Bell Labs [Feldman 1979]

The good news is that Make will:

Figure out what has changed Work out what is affected by those changes Execute commands to bring things up to date (e.g., by recompiling) Commands are handed to the shell for execution, so they're the same ones you would run yourself

1 of 8

8/23/2005 11:55 AM

Software Carpentry: Automated Builds (Draft Version 348)

file:///C:/home/swc/build/www/make.html

The bad news is Make's syntax

Initially very simple, but has grown more complex over time Has turned into a little programming language in its own right (rule #11) We will ignore the complex bits for now, and take a look at a better way to solve these problems in Backward, Forward, and Sideways

Note: examples below are written in terms of Java, but Make doesn't care what language you use
Course material includes examples in C and Fortran Comment on this slide

Hello, Make
Put program in hello.java Put the following into a file called hello.mk:
# $Id: hello.mk 111 2005-06-10 21:13:26Z gvwilson $ hello.class : hello.java javac hello.java Note: that indentation must be a tab character, not eight spaces, or a mix of spaces and tabs Yeah, it's a wart, but we're stuck with it

Some terminology:

Figure 5.1: Structure of a Make Rule hello.class is the target of the rule hello.java is its prerequisite We'll see in a moment that a target may have many prerequisites The compilation command is the rule's action A rule may also have many actions

Run make -f hello.mk

Make sees that hello.class depends on hello.java Since hello.class doesn't exist, Make executes the rule's action (i.e., compiles hello.java)

Run make -f hello.mk again

hello.class is newer than its prerequisite, so the action is not executed Comment on this slide

Multiple Targets
2 of 8

8/23/2005 11:55 AM

Software Carpentry: Automated Builds (Draft Version 348)

file:///C:/home/swc/build/www/make.html

Very few programs consist of a single file

This Makefile describes two files that are independent of one another # $Id: double.mk 111 2005-06-10 21:13:26Z gvwilson $ Left.class : Left.java javac Left.java Right.class : Right.java javac Right.java

When you run make -f double.mk, only Left.java is compiled

The first target in the Makefile is the default target

Run make -f double.mk Right.class to build Right.class

Comment on this slide

Phony Targets
Obviously don't want to have to run Make separately for each possible target
That would hardly count as automation

Solution: create a phony target that:

Depends on all the things you want to recompile, but Doesn't correspond to any files, and is never up to date So making it always executes its actions # $Id: phony.mk 111 2005-06-10 21:13:26Z gvwilson $ all : Left.class Right.class Left.class : Left.java javac Left.java Right.class : Right.java javac Right.java Now, make -f phony.mk all compiles both Java files

Typical phony targets

"all": recompile everything "clean": delete all temporary files, and everything produced by compilation FILE NOT FOUND: src\make\all.mk "install": copy files to system directories Many open source projects are installed by typing make, and then make install Comment on this slide

3 of 8

8/23/2005 11:55 AM

Software Carpentry: Automated Builds (Draft Version 348)

file:///C:/home/swc/build/www/make.html

Dependencies
If Overlap depends on Rect, then Overlap.java must be recompiled if:
It's newer than Overlap.class, or It's newer than Rect.class Which must be re-created if it's older than Rect.java # $Id: depend.mk 111 2005-06-10 21:13:26Z gvwilson $ Overlap.class : Overlap.java Rect.class javac Overlap.java Rect.class : Rect.java javac Rect.java

Often visualize dependencies as a tree

Figure 5.2: Visualizing Dependencies Mark all the files that are out of date, and follow arrows to find out what will happen Note: Make can execute actions in any order it wants to, as long as it doesn't violate dependency ordering

Dependencies can have dependencies, and so on

# $Id: multi.mk 111 2005-06-10 21:13:26Z gvwilson $ Test.class : Test.java Overlap.class javac Test.java Overlap.class : Overlap.java Rect.class javac Overlap.java Rect.class : Rect.java javac Rect.java

Figure 5.3: Visualizing Deeper Dependencies Comment on this slide

Avoiding Redundancy
Often want to set options when compiling
"-d" to specify output directory "-source 1.4" to specify Java language version

Do not want to set these options over and over again

4 of 8

8/23/2005 11:55 AM

Software Carpentry: Automated Builds (Draft Version 348)

file:///C:/home/swc/build/www/make.html

Rule #3: anything repeated in two or more places will eventually be wrong in at least one

Solution: define variables (usually called macros)

Closely related to the shell's environment variables # $Id: macro.mk 111 2005-06-10 21:13:26Z gvwilson $ JC = javac -source 1.4 Test.class : Test.java Overlap.class ${JC} Test.java Overlap.class : Overlap.java Rect.class ${JC} Overlap.java Rect.class : Rect.java ${JC} Rect.java Define by specifying name and value To get value, put a "$" in front of the name, and parentheses or braces around it Without the parentheses, Make will interpret "$XYZ" as the value of the variable "X", followed by the characters "YZ" Yeah, it's a wart, but Comment on this slide

Automatic Variables
Make defines automatic variables to represent parts of rules
These variables are automatically redefined for each rule Names are unfortunately very cryptic Keep a reference card handy
"$@" "$<" "$?" "$^"

The rule's target The rule's first prerequisite All of the rule's out-of-date prerequisites All prerequisites
Table 5.1: Automatic Variables in Make

Example
# $Id: auto.mk 111 2005-06-10 21:13:26Z gvwilson $ JC = javac -source 1.4 Test.class : Test.java Overlap.class @echo "Building" $@ ${JC} $<

5 of 8

8/23/2005 11:55 AM

Software Carpentry: Automated Builds (Draft Version 348)

file:///C:/home/swc/build/www/make.html

Overlap.class : Overlap.java Rect.class @echo "Building" $@ ${JC} $< Rect.class : Rect.java @echo "Building" $@ ${JC} $< Note: by default, Make echoes actions before executing them Putting "@" at the start of the action line prevents this Comment on this slide

Pattern Rules
Most files of similar type (Java, C++, etc.) are compiled the same way
Rule #3 again: write a pattern rule specifying what to do in the general case

Use the wildcard "%" to represent the stem of the file's name in the target and prerequisites
Must use automatic variables in the actions This is why they were invented # $Id: rule.mk 111 2005-06-10 21:13:26Z gvwilson $ JC = javac -source 1.4 all : Left.class Right.class %.class : %.java ${JC} $< Comment on this slide

Being More Precise

Two rules:
If both a pattern and an explicit rule exist, the explicit rule is used Make collects all of the dependencies for a target before figuring out if the target is out of date

Example:
# $Id: accumulate.mk 111 2005-06-10 21:13:26Z gvwilson $ JC = javac -source 1.4 Test.class : Overlap.class Overlap.class : Rect.class Rect.class : Rect.java javac -source 1.4 -nowarn $<

6 of 8

8/23/2005 11:55 AM

Software Carpentry: Automated Builds (Draft Version 348)

file:///C:/home/swc/build/www/make.html

%.class : %.java ${JC} $< Tell Make that Test depends on Overlap, and Overlap depends on Rect Turn off warnings when compiling Rect.java, but use one pattern for everything else Comment on this slide

Passing In Data
Sometimes useful to pass values into Make when invoking it
E.g., change the directory that the output is put in

Rather than rewriting the Makefile each time, you can specify name=value pairs on the command line
Define a macro with the default value Override it when you want to

Example:
VAL = original echo : @echo "VAL is" ${VAL} $ make -f env.mk echo VAL is original $ make -f env.mk VAR=changed echo VAL is changed

Make checks the following in order:

Values specified on the command line Environment variables (like $HOME) Values set in the Makefile Comment on this slide

Analysis
Pro
Simple things are simple to do and not too difficult to read especially compared to the alternatives

Con
The syntax is unpleasant Complex things are difficult to read and even more difficult to debug

7 of 8

8/23/2005 11:55 AM

Software Carpentry: Automated Builds (Draft Version 348)

file:///C:/home/swc/build/www/make.html

Best you can do is use echo to print things as Make executes Not really very portable Hands commands to the shell for execution But commands use different flags on different operating systems Do you use del or rm to delete files?

Alternatives
Ant primarily for Java Less platform-dependent but just as hard to read and debug Integrated development environments Most are proprietary and platform-specific SCons Give users a library to manage dependencies and actions Makefile is actually a small program More powerful, and debuggable, but steeper learning curve Comment on this slide

prev

Copyright 2005, Python Software Foundation. See License for details.

next

8 of 8

8/23/2005 11:55 AM