The Makefile utility

Motivation

 Small programs single file

 “Not so small” programs :

– Many lines of code
– Multiple components
– More than one programmer
Mozilla
 Mozilla Directory Structure

One very large directory tree

26,000 files
2,500 subdirectories
240 file types
Mozilla Directory References
1,521 directory references
Avg. fan-in/fan-out 12
Max fan-in=100
Max fan-out=42
Median fan-in=5
Median fan-out=12
 Build-level
File (atomic entity)
–Source
–Documentation

Directory tree (container) = source tree

Build process
–To build/install software
–Driven by make/ANT/…

Configuration process
–To control build process
–Driven by configure/configuration files/…
 Mozilla Build Level

1,350 Makefiles
40,000 LOC build instructions
16,000 LOC configuration
Cyclic dependencies
Two-phase build process
Centralized build/configuration knowledge
Code duplication
Component implementations scattered
 Motivation – continued

 Problems:

– Long files are harder to manage

(for both programmers and machines)
– Every change requires long compilation
– Many programmers can not modify the
same file simultaneously
– Division to components is desired
 Motivation – continued

 Solution : divide project to multiple files

 Targets:

– Good division to components

– Minimum compilation when something is
changed
– Easy maintenance of project structure,
dependencies and creation
 Project maintenance

 Done in Unix by the Makefile mechanism

 A makefile is a file (script) containing :
– Project structure (files, dependencies)
– Instructions for files creation
 The make command reads a makefile,
understands the project structure and
makes up the executable
 Note that the Makefile mechanism is not
limited to C programs
 Project structure

 Project structure and dependencies can be

represented as a DAG (= Directed Acyclic
Graph)
 Example :
– Program contains 3 files
– main.c., sum.c, sum.h
– sum.h included in both .c files
– Executable should be the file sum
 sum (exe)

main.o sum.o

main.c sum.h sum.c sum.h

 Make: Header Dependencies
 What if …
– Sensor.cc changes?
Sensor.h Sensor.cc
– Sensor.h changes?
– Robot.h changes?
 Pattern rule ignores header
dependencies!
Robot.h Robot.cc
 Requires unnecessary “make
clean; make”
 Let gcc figure out the header
dependencies for us! robotest.cc
 The –MM option produces make
rules in a .d file which we can
then include in the Makefile
 makefile

sum: main.o sum.o

gcc –o sum main.o sum.o

main.o: main.c sum.h

gcc –c main.c

sum.o: sum.c sum.h

gcc –c sum.c
 Rule syntax

main.o: main.c sum.h

Rule
gcc –c main.c

tab

dependency action
 Equivalent makefiles

 .o depends (by default) on corresponding

.c file. Therefore, equivalent makefile is:
sum: main.o sum.o
gcc –o sum main.o sum.o

main.o: sum.h
gcc –c main.c

sum.o: sum.h
gcc –c sum.c
 Equivalent makefiles - continued

 We can compress identical dependencies

and use built-in macros to get another
(shorter) equivalent makefile :
sum: main.o sum.o
gcc –o $@ main.o sum.o
main.o sum.o: sum.h
gcc –c $*.c
 make operation

 Project dependencies tree is constructed

 Target of first rule should be created
 We go down the tree to see if there is a target
that should be recreated. This is the case when
the target file is older than one of its
dependencies
 In this case we recreate the target file according
to the action specified, on our way up the tree.
Consequently, more files may need to be
recreated
 If something is changed, linking is usually
necessary
 make operation - continued

 make operation ensures minimum

compilation, when the project structure is
written properly
 Do not write something like:
prog: main.c sum1.c sum2.c
gcc –o prog main.c sum1.c sum2.c
which requires compilation of all project when
something is changed
 Make operation - example

File Last Modified

sum 10:03
main.o 09:56
sum.o 09:35
main.c 10:45
sum.c 09:14
sum.h 08:39
 Make operation - example

 Operations performed:
gcc –c main.c
gcc –o sum main.o sum.o
 main.o should be recompiled (main.c is
newer).
 Consequently, main.o is newer than sum and
therefore sum should be recreated (by re-
linking).
 Another makefile example
# Makefile to compare sorting routines

BASE = /home/blufox/base
CC = gcc
CFLAGS = -O –Wall
EFILE = $(BASE)/bin/compare_sorts
INCLS = -I$(LOC)/include
LIBS = $(LOC)/lib/g_lib.a \
$(LOC)/lib/h_lib.a
LOC = /usr/local

OBJS = main.o another_qsort.o chk_order.o \

compare.o quicksort.o

$(EFILE): $(OBJS)
@echo “linking …”
@$(CC) $(CFLAGS) –o $@ $(OBJS) $(LIBS)

$(OBJS): compare_sorts.h
$(CC) $(CFLAGS) $(INCLS) –c $*.c

# Clean intermediate files

clean:
rm *~ $(OBJS)
 Example - continued

 We can define multiple targets in a makefile

 Target clean – has an empty set of
dependencies. Used to clean intermediate
files.
 make
– Will create the compare_sorts executable
 make clean
– Will remove intermediate files
 Passing parameters to makefile
 We can pass parameters to a makefile by
specifying them along with their values in
the command line.
 For example:
make PAR1=1 PAR2=soft1
will call the makefile with 2 parameters:
PAR1 is assigned the value “1” and PAR2
is assigned the value “soft1”. The same
names should be used within the makefile
to access these variables (using the usual
“$(VAR_NAME)” syntax)
 Passing parameters - continued
 Note that assigning a value to a variable
within the makefile overrides any value
passed from the command line.
 For example:
command line : make PAR=1
in the makefile:
PAR = 2
 PAR value within the makefile will be 2,
overriding the value sent from the command
line
 Conditional statements

 Simple conditional statements can be

included in a makefile.
 Usual syntax is:
ifeq (value1, value2)
body of if
else
body of else
endif
 Conditional statements - example
sum: main.o sum.o
gcc –o sum main.o sum.o

main.o: main.c sum.h

gcc –c main.c

#deciding which file to compile to create sum.o

ifeq ($(USE_SUM), 1)

sum.o: sum1.c sum.h

gcc –c sum1.c –o $@

else
sum.o: sum2.c sum.h
gcc –c sum2.c –o $@

endif
 Make: Advanced Options
 Text manipulation functions
– $(patsubst pattern,replacement,text)
– $(patsubst %.o,%.cc,<list of objfiles>)
 Pattern rules
– Uses a pattern in the target with % as wildcard
– Matched % can be used in dependencies as well
– Simple Example:
%.o : %.cc
<tab>command …
 Pattern rules with automatic variables
– $@ full target name
– $< first dependency
– $* string which matched % wildcard
– Advance Example:
%.o : %.cc
<tab>$(CC) $(CCFLAGS) –c $< $(INCPATHS)
 Make: A Simple Example
CC=g++ # Compiler to use
FLAGS=-g # Compile flags
MASLAB_ROOT=maslab-software # Maslab software root directory
LIB_DIR=$(MASLAB_ROOT)/liborc # orc-related library directory
INC_DIR=$(MASLAB_ROOT)/liborc # orc-related include directory
LIBS=-lm –lpthread -lorc # Library files

all : helloworld

helloworld.o : helloworld.cc
$(CC) $(FLAGS) –c $*.cc –o $@

helloworld: helloworld.o
$(CC) -o helloworld helloworld.o $(LIBS)

clean:
rm -f *.o helloworld
 Make: Example Makefile
MASLABROOT = /mnt/maslab/software/maslab-software-current

#This is the list of places to look for include files

INCPATHS = -I$(MASLABROOT)/libs/liborc \
-I$(MASLABROOT)/libs/libim \
-I$(MASLABROOT)/libs/libbotserver \
-I/opt/intel/ipp/include

#This is the list of places to look for libraries

LIBPATHS = -L$(MASLABROOT)/libs/liborc \
-L$(MASLABROOT)/libs/libim \
-L$(MASLABROOT)/libs/libbotserver \
-L/opt/intel/ipp/sharedlib

#This is the names of the libraries

LIBS = -lippipx -lippcvpx -lim -lorc -lbotserver -lm -lpthread -ljpeg -lpng

#This is the compiler to use

CC = g++

# This is your c++ file extension (usually cc or cpp)

CCEXT = cc
 Make: Example Makefile (cont)
CCFLAGS = -g -Wall

# This rule builds everything. You should put the names of all your
# programs in this list.
all : robotest

# This rule says how to turn any .cpp file into a .o file.
%.o : %.$(CCEXT)
$(CC) $(CCFLAGS) -c $< $(INCPATHS)

#----------------------------------------------------------------------
# robotest

PROGRAM_NAME = robotest
PROGRAM_OBJS = robotest.o BumpSensor.o Robot.o

GLOBAL_OBJS += $(PROGRAM_OBJS)
JUNK += $(PROGRAM_NAME)

$(PROGRAM_NAME): $(PROGRAM_OBJS)
$(CC) $(PROGRAM_OBJS) -o $(PROGRAM_NAME) $(LIBS) $(LIBPATHS)
chmod a+x $(PROGRAM_NAME)
 Extending Example Makefile
So to add a new executable program, simply cut and
paste the robotest section and enter your own
PROGRAM_NAME and PROGRAM_OBJS

#----------------------------------------------------------------------
# Sensor Incremental Test

PROGRAM_NAME = sensor-test
PROGRAM_OBJS = sensor.t.o BumpSensor.o

GLOBAL_OBJS += $(PROGRAM_OBJS)
JUNK += $(PROGRAM_NAME)

$(PROGRAM_NAME): $(PROGRAM_OBJS)
$(CC) $(PROGRAM_OBJS) -o $(PROGRAM_NAME) $(LIBS) $(LIBPATHS)
chmod a+x $(PROGRAM_NAME)
 Extending example Makefile
DEPENDS += $(patsubst %.o, %.d, $(GLOBAL_OBJS))
JUNK += $(DEPENDS)
include $(DEPENDS)
depend : $(DEPENDS)

depend.$(CCEXT) = set -e; $(CC) $(CCFLAGS) -MM $(DEFS) $(INCPATHS)

depend.filt = sed 's/\($*\)\.o[ :]*/\1.o $@ : /g' > $@; [ -s $@ ] || rm -f $@

%.d: %.$(CCEXT)
$(depend.cc) $< | $(depend.filt)

#======================================================================
# Miscellaneous Rules
#----------------------------------------------------------------------

# This rule cleans your directory. You could also add commands here
# to remove program files and any other files that should be cleaned
clean:
rm -f $(JUNK) *~ *.o core.[0-9]* core
 ANT:
Another Nice Tool
 What is Ant?
 Java-based Build tool from Apache
 De facto standard for building, packaging, and
installing Java applications
 Accomplishes same objectives that make does on
Unix based systems
 Files are written in XML
 Why Ant?
 Many claim.. That
– Unlike makefiles, Ant files work cross platform
- No need for multiple, complex makefiles
depending on the operating system.
- Tasks declared in platform independent way; Ant
engine translates to OS specific commands.
But still need to know (potentially) complex
path/install data. So not as general as they claim
 Easy to create own Ant “tasks”, in addition to core
tasks
 Installing Ant
 Download Ant binary distribution from:
http://ant.apache.org/bindownload.cgi
 Set ANT_HOME to where you installed Ant
 Include $ANT_HOME/bin in PATH
 Make sure JAVA_HOME is set to point to JDK
 Running Ant
 Type “ant” at the command line
 Automatically looks for build.xml file in current
directory to run
 Type “ant –buildfile buildfile.xml” to specify
another build file to run.
Ant Output
 Sample build.xml
<project name="MyProject" default="dist" basedir=".">
<property name="src" location="src"/>
<property name="build" location="build"/>
<property name="dist" location="dist"/>

<target name="init">
<tstamp/>
<mkdir dir="${build}"/>
</target>

<target name="compile" depends="init" >

<javac srcdir="${src}" destdir="${build}"/>
</target>

<target name="dist" depends="compile" >

<mkdir dir="${dist}/lib"/>
<jar jarfile="${dist}/lib/MyProject-${DSTAMP}.jar" basedir="${build}"/>
</target>

<target name="clean" >

<delete dir="${build}"/>
<delete dir="${dist}"/>
</target>
</project>
 Ant Overview: Project
 Each build file contains exactly one project and at
least one target
 Project tags specify the basic project attributes
and have 3 properties:
- name
- default target
- basedir
 Example: <project name=“MyProject” default=“build” basedir=“.”>
 Ant Overview: Targets
 Target is a build module in Ant
 Each target contains task(s) for Ant to do
 One must be a project default
 Overall structure of targets:
<target name="A"/>
<target name="B" depends="A"/>
<target name="C" depends="B"/>

<target name="D" depends="C,B,A"/>

 Ant Overview: Tasks
 Each target comprises one or more tasks
 Task is a piece of executable Java code (e.g.
javac, jar, etc)
 Tasks do the actual “build” work in Ant
 Ant has core (built in) tasks and the ability to
create own tasks
 Ant Overview: Tasks
Example:
<target name="prepare" depends="init“ >

<mkdir dir="${build}" />

</target>

<target name="build" depends="copy" >

<javac srcdir="src" destdir="${build}">

<include name="**/*.java" />
</javac>
</target>
 Ant Overview: Core Tasks
 javac – Runs the Java Compiler
 java – Runs the Java Virtual Machine
 jar (and war) – Create JAR files
 mkdir – Makes a directory
 copy – Copies files to specified location
 delete – Deletes specified files
 cvs – Invokes CVS commands from Ant
 Ant Overview: Writing Own
Task
 Create a Java class that extends org.apache.tools.ant.Task
 For each attribute, write a setter method that is
public void and takes a single argument
 Write a public void execute() method, with no
arguments, that throws a BuildException -- this
method implements the task itself
 Ant Overview: Properties
 Special task for setting up build file properties:
 Example:
<property name=“src” value=“/home/src”/>
 Can use ${src} anywhere in build file to denote
/home/src
 Ant provides access to all system properties as if
defined by the <property> task
 Ant Overview: Path Structures
 Ant provides means to set various environment
variables like PATH and CLASSPATH.
 Example of setting CLASSPATH:
<classpath>
<pathelement path="${classpath}"/>
<pathelement location="lib/helper.jar"/>
</classpath>
 Command Line Arguments
 -buildfile buildfile – specify build file to use
 targetname – specify target to run (instead of
running default)
 -verbose, -quiet, -debug – Allows control over
the logging information Ant outputs
 -logger classname – Allows user to specify their
own classes for logging Ant events
 IDE Integration
 Eclipse, NetBeans, JBuilder, VisualAge, and
almost any other Java IDE has Ant integration
built-in to the system
 Refer to each IDE’s documentation for how to use
Ant with that IDE
 Web Development with Ant
 Tomcat comes with special Ant tasks to ease Web
application development and deployment
 Copy $TOMCAT_HOME/server/lib/catalina-ant.jar to
$ANT_HOME/lib
 Ant tasks for Tomcat:
- install
- reload
- deploy
- remove
 Documentation/References

 Good tutorial for makefiles

http://www.gnu.org/manual/make-3.80/make.html
 Good tutorial for cmakef
http://www.cmake.org/HTML/Documentation.html
 ANT User Manual:
http://ant.apache.org/manual/index.html
 Sun’s Web development tutorial (Ant and JSPs):
http://java.sun.com/webservices/docs/1.2/tutorial/doc/GettingStarted3.html
 Standardized Build Interface
Different build systems exist, e.g.,
–make, ANT, shell scripts, IDE

Different software systems require different build

actions, e.g.,
–make make clean
–make bootstrap, make, make install make all
make check
make (un)install
make dist
 The real Problem
 How do we handle platform specific issues?
– Providing a different Makefile for each architecture
– Using Autoconf, Automake and Libtool

 The installer needs only

– Bourne shell
– C compilers
– Make program
 Standardized Configuration
Interface
Different mechanisms exist to control software
construction, e.g.,
–configuration files, configuration tools, Makefile
editing
Standardized configuration interface to enable
uniform compile-time configuration:
configure --help
configure --prefix=/usr
configure --with-aterm=/usr/lib
configure --with-optimization=true
configure --with-debug=false
 Explicit Context Dependencies
Dependencies on build-level components are
declared in configuration interfaces
> configure --help
…
--with-aterm=DIR use ATerm Library at DIR
--with-sglr=DIR use SGLR Parser at DIR
…
 Independent Deployment
Build-level components are deployed as packages

A package is a versioned release of a build-level

component, e.g.,

firefox-1.0.tar.gz

Packages are published on web/ftp sites

 Third-party Composition
A configuration interface enables late-binding of
dependencies

> configure --with-aterm=/usr/local/aterm

--with-sglr=/usr/local/sglr
--with-…=

Compositions are thus not predefined

and can be defined by a third party

 Some advantages when using GNU
autotools
 The installation of a program is straightforward:
./configure; make; make install

 This procedure checks for system parameters, libraries,

location of programs, availability of functions and writes
a Makefile

 ./configure supports many options to overwrite

defaults settings
 GNU autoconf

Source Code configure.ac

(configure.in)
aclocal

autoscan autoconf

configure.scan configure
 GNU automake

Makefile.am Makefile.in

automake configure

Makefile
 configure.ac
 dnl Comment … …

 AC_INIT(main.c)

 AM_INIT_AUTOMAKE(project_name, 1.2.8)

 AM_PROG_LIBTOOL it supports libtool and shared libraries

 AC_PROG_CC (or AC_PROG_CXX) it locates the C (C++) compiler

 AC_HEADER_STDC it checks for standard headers

 AC_CHECK_HEADERS(sys/time.h /header.h) it checks for headers availability

 AC_CHECK_LIB(crypto SSLeay_version) it checks for libraries availability

 AC_CHECK_FUNCS(ctime) it checks for functions availability

 AC_PROG_INSTALL it checks for BSD compatible install utility

 AC_OUTPUT([Makefile])
 Makefile.am
 bin_PROGRAMS = foo
/configure --prefix=… (default /usr/local)

 foo_PROGRAMS=foo.c foo.h

 noist_PROGRAMS=test
(make compiles, make install does nothing)

 EXTRA_DIST=disclaimer.txt
 Example
 foo.c :
#include <stdio.h>
main()
{
printf(“Cum grano salis\n");
}

 Makefile.am :
bin_PROGRAMS = foo
foo_SOURCES = foo.c

 configure.ac :
AC_INIT(foo.c)
AM_INIT_AUTOMAKE(latin_words, 0.9)
AC_PROG_CC
AC_HEADER_STDC
AC_PROG_INSTALL
AC_OUTPUT([Makefile])
 Summary
 Source Code, configure.ac, Makefile.am
 autoscan; aclocal; autoconf
 Create NEWS README AUTHORS ChangeLog
 automake –add-missing
 ./configure; make; make dist
 Result: project_name-2.10.tar.gz
aclocal.m4 autom4te-2.53.cache ChangeLog config.status
configure.in COPYING install-sh Makefile.am missing NEWS
README AUTHORS autoscan.log config.log configure
configure.scan INSTALL Makefile.in mkinstalldirs code.c
 References
 GNU Autoconf, Automake, and Libtool
http://sources.redhat.com/autobook/autobook/autobook_toc.html

 GNU Autoconf Manual

http://www.gnu.org/manual/autoconf

 GNU Automake Manual

http://www.gnu.org/manual/automake

 GNU Libtool Manual

http://www.gnu.org/manual/libtool

 Learning the GNU development tools

http://autotoolset.sourceforge.net/tutorial.html

 The GNU configure and build system

http://www.airs.com/ian/configure/configure_toc.html

 GNU macro processor (GNU m4)

http://www.gnu.org/manual/m4-1.4/m4.html