Computer Aided Geometric Design 21 (2004) 4364

www.elsevier.com/locate/cagd
Shape-preserving approximation of multiscale univariate data
by cubic L
1
spline ts
John E. Lavery
Mathematics Division, Army Research Ofce, Army Research Laboratory, P.O. Box 12211,
Research Triangle Park, NC 27709-2211, USA
Received 24 June 2003; received in revised form 24 June 2003; accepted 11 July 2003
Abstract
Spline ts are calculated by minimizing a data tting functional over a manifold of splines. Smoothing splines
are calculated by minimizing a linear combination of a data tting functional and an interpolating spline functional.
For multiscale data, that is, data with abrupt changes in magnitude and/or spacing, currently available spline ts
and smoothing splines typically have extraneous oscillation. In this paper, we introduce a class of cubic L
1
spline
ts that do not have extraneous oscillation. In contrast to conventional cubic spline ts, which are based on L
2
functionals (that is, on quadratic functionals, which consist of sums and integrals of squares), cubic L
1
splines are
based on minimizing L
1
functionals (sums and integrals of absolute values). Cubic L
1
spline ts are calculated
by a Lagrange-multiplier-based primal afne (interior point) algorithm. Cubic L
1
spline ts are compared with
conventional L
2
cubic spline ts and with cubic L
1
and L
2
smoothing splines. Computational results indicate that
L
1
spline ts preserve shape well, nearly as well as L
1
smoothing splines with good balance parameters and better
than conventional L
2
spline ts and L
2
smoothing splines. L
1
spline ts are computationally more expensive than
L
1
smoothing splines and considerably more expensive than L
2
spline ts and L
2
smoothing splines. However, L
1
spline ts require only half the storage required by smoothing splines and do not involve a balance parameter that
is required by smoothing splines.
2003 Elsevier B.V. All rights reserved.
MSC: 65D07; 65K10
Keywords: Approximation; Cubic spline; L
1
spline; L
2
spline; Multiscale; Shape preservation; Smoothing spline; Spline t;
Univariate
E-mail address: john.lavery2@us.army.mil (J.E. Lavery).
0167-8396/$ see front matter 2003 Elsevier B.V. All rights reserved.
doi:10.1016/j.cagd.2003.07.002
44 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
1. Introduction
In approximation of univariate data sets by smooth curves, splines have played a large role. Smoothing
splines, which are calculated by minimizing a linear combination of a data tting functional and an
interpolating spline functional, were introduced by Schoenberg (1964). Mehlum (1964), Reinch (1967,
1971), Anselone and Laurent (1968), Greville (1969), Schumaker (1969), Woodford (1971), Lyche and
Schumaker (1973), Wahba (1975, 1990), Wahba and Wold (1975), (de Boor, 1978, Chapter XIV) and
Utreras (1981) developed theory and algorithms for smoothing splines based on quadratic data tting
functionals and quadratic spline functionals. Because of the theoretical and computational successes in
these and other research papers, smoothing splines became a predominant choice in research and in
practice. A second, less common choice in approximating univariate data sets has been spline ts or,
as they are also called, least square (spline) approximants (de Boor, 1978, Chapter XIV; Farin, 1997,
Section 10.10), which are calculated by minimizing over a space of splines a data tting functional,
typically a weighted sum of the squares of the differences between the spline and the data at the locations
of the data.
Both spline ts and smoothing splines have enjoyed considerable success for many types of data.
However, for multiscale data, that is, data with abrupt changes in magnitude and/or spacing, they
typically have extraneous oscillation, just as do interpolating splines. Many of the procedures used to
reduce extraneous oscillation in interpolating splinesadjusting the number or position of the spline
nodes, locally varying the degree of the spline, replacing conventional polynomial splines by rational
or exponential splines, adding constraints or penalties, a posteriori ltering, fairing, etc.have been
successful for various types of spline ts and smoothing splines, but none of them completely eliminate
the extraneous oscillation for arbitrary multiscale data.
The use of data-tting functionals that do not consist of sums of squares has been investigated for over
two decades. Huber (1979) replaced the square in the sum of squares in the data-tting functional by
a function that is quadratic for small arguments and linear for large arguments. He noted that doing so
produced smoothing splines with greater robustness (resistance to outliers) than conventional smoothing
splines. Bloomeld and Steiger (1983) introduced LAD [Least Absolute Distance] smoothing splines,
which are piecewise cubic (or higher-degree) functions that minimize a linear combination of an
1
data
tting functional (the sum of the absolute values of the differences between the smoothing spline and the
data at the data locations ) and a conventional spline functional (the integral of the square of the second
or higher derivative). Bloomeld and Steiger noted that LAD smoothing splines preserve shape better
(have less extraneous oscillation) than conventional smoothing splines based on quadratic data tting
functionals. Cox and Jones (1987) investigated the shape-preserving properties of conventional splines
(spline ts) that minimize an
1
data-tting functional and noted that nding a suitable set of spline nodes
is important.
In all of these prior investigations, the spline functional was the integral of the square of the second (or
higher) derivative of the spline. Recently, cubic L
1
interpolating splines calculated by minimizing the L
1
norm (integral of the absolute value) of the second derivative have been developed and have been shown
to provide shape-preserving interpolation of multiscale data without the extraneous oscillation commonly
seen in conventional splines (Lavery, 2000a). Cubic L
1
smoothing splines, which minimize a linear
combination of an
1
data tting functional and the cubic L
1
spline functional (the integral of the absolute
value of the second derivative), were introduced in (Lavery, 2000b). The shape-preserving performance of
cubic L
1
smoothing splines is excellent. However, cubic L
1
smoothing splines like all other smoothing
J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364 45
splines depend on a balance parameter that xes the relative weight of tting the data vs. that of
smoothing out local variations in the data. Many researchers have investigated the balance parameter
for smoothing splines (see (Wahba, 1990) and references therein) but optimal balance parameters for
large classes of data, especially multiscale data, are still not available. Progress in determining optimal
balance parameters for cubic L
1
smoothing splines is not expected to be rapid (Lavery, 2000b, p. 720).
For this reason, investigation of spline ts, an alternative that does not require the user to choose a balance
parameter, is of considerable interest. The goal of the present paper is to investigate cubic L
1
spline ts
and to compare the properties of cubic L
1
spline ts with the properties of conventional cubic spline ts,
which we will call cubic L
2
spline ts, and with the properties of cubic L
1
and L
2
smoothing splines.
2. Cubic L
1
spline ts
We consider tting the data ( x
m
, z
m
), m = 1, 2, . . . , M, where the x
m
are in the nite real interval
[x
0
, x
I
]. The weight of the mth data point is a positive real number w
m
. The cubic splines with which we
will t the data have strictly monotonic but otherwise arbitrary nodes x
i
, i = 0, 1, . . . , I ,
<x
0
<x
1
<x
2
< <x
I1
<x
I
<. (1)
At each node x
i
, i = 0, 1, . . . , I , we seek the value z
i
=z(x
i
) of the cubic L
1
spline z =z(x).
A cubic L
1
spline t of the data is a piecewise cubic function that minimizes
M

m=1
w
m

z( x
m
) z
m

(2)
over the manifold of cubic L
1
splines, that is, C
1
-smooth piecewise cubic polynomials z with nodes x
i
,
i = 0, 1, . . . , I , that minimize
x
I
_
x
0

d
2
z
dx
2

dx (3)
for given nodal values z
i
= z(x
i
), i = 0, 1, . . . , I . (In what follows, this denition will be adjusted by
regularization and discretization.) Functional (3) is the functional that denes a cubic L
1
interpolating
spline (Lavery, 2000a). Boundary conditions or other conditions could be imposed with no signicant
change in the theory or computational methods. For simplicity, we consider here only the case with no
boundary conditions.
Calculating the coefcients of an L
1
spline t is a two-level optimization problem, namely, that of
minimizing the
1
data-tting functional (2) subject to a constraint that the solution occur at the minimum
of the L
1
spline functional (3). Functionals (2) and (3) are convex and therefore have minima. However,
they are not strictly convex and their minima need not be unique. For this reason, we add regularization
terms and redene a cubic L
1
spline t to be the piecewise cubic function that minimizes
M

m=1
w
m

z( x
m
) z
m

+
I

i=0

0,i
|z
i
| (4)
46 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
over the manifold of cubic L
1
splines that minimize
x
I
_
x
0

d
2
z
dx
2

dx +
I

i=0

1,i

dz
dx
(x
i
)

(5)
for given nodal values z
i
= z(x
i
), i = 0, 1, . . . , I . Here, the
0,i
and
1,i
are given small, positive
regularization parameters. Both the data-tting functional (4) and the spline functional (5) are still
only convex, not strictly convex. However, these two functionals have unique minima as long as the
0,i
and
1,i
are sufciently small (specically, smaller than any of the coefcients of the z
i
in (2) and than
any of the coefcients of the dz(x
i
)/dx in (3), respectively).
Conventional spline data tting technology has been based predominantly on the
2
and L
2
norms.
For this reason, we will compare each cubic L
1
spline t presented in this paper with a corresponding
cubic L
2
spline t dened to be the piecewise cubic function that minimizes
M

m=1
_
w
m
_
z( x
m
) z
m
__
2
(6)
over the manifold of cubic L
2
splines, that is, C
1
piecewise cubic polynomials that minimize
x
I
_
x
0
_
d
2
z
dx
2
_
2
dx. (7)
We will further compare each cubic L
1
spline t and each cubic L
2
spline t presented in this paper
with corresponding cubic L
1
and L
2
smoothing splines. For completeness, we present here the denitions
of cubic L
1
and L
2
smoothing splines (Lavery, 2000b). Let , 0 < < 1, be given. Cubic L
1
and L
2
smoothing splines are calculated by minimizing the functionals

m=1
w
m

z( x
m
) z
m

+(1 )
x
I
_
x
0

d
2
z
dx
2

dx +
I

i=0

1,i

dz
dx
(x
i
)

(8)
and

2
M

m=1
_
w
m
_
z( x
m
) z
m
__
2
+(1 )
2
x
I
_
x
0
_
d
2
z
dx
2
_
2
dx, (9)
respectively, over the set of C
1
-smooth piecewise cubic functions z with nodes x
i
, i = 0, 1, . . . , I .
3. Algorithms
Calculation of L
2
spline ts is a classical quadratic two-level optimization problem. For the
computational experiments reported in this paper, we solved this optimization problem by Lagrange
multipliers. This procedure involves one solution of a linear system with a 3(I +1) 3(I +1) symmetric
positive denite matrix that has 11 nonzero bands. The unknowns of this system are the values of the
spline and of its rst derivative at the I +1 spline nodes and the values of I +1 Lagrange multipliers.
J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364 47
Calculation of L
1
spline ts is also a two-level optimization problem. The lower level, that of
minimizing (4), is a linear program while the higher level, that of minimizing (5), is a nonlinear program
if the integral in (5) is in continuum form. Efcient algorithms do not yet exist for this nonlinear
program (see comments at the end of Section 4 of (Lavery, 2000a)). For the computational experiments
reported in this paper, we changed the nonlinear program of minimizing (5) into a linear program by
discretizing the integral in (5) using a midpoint rule with K equal subintervals in each interval (x
i
, x
i+1
),
i = 0, 1, . . . , I 1.
The two-level optimization problem of minimizing (4) subject to the spline parameters minimizing (5)
(with the integral discretized) can be solved by a variant of the basic primal afne method (Vanderbei
et al., 1986; Vanderbei, 1989; Adler and Monteiro, 1991; Tsuchiya and Muramatsu, 1992; Hall
and Vanderbei, 1993) previously used to calculate univariate and bivariate cubic L
1
splines (Lavery,
2000a, 2000b, 2001). The basic primal afne method is described in detail in (Lavery, 2001). The primal
afne method used to calculate the L
1
spline ts of the present paper, called the Lagrange-multiplier
primal afne method, is described in the following paragraphs.
Let z be the (I +1)-dimensional vector of the values z
i
, i = 0, 1, . . . , I , of the L
1
spline t at the nodes
and b be the (I + 1)-dimensional vector of the values dz(x
i
)/dx, i = 0, 1, . . . , I , of the rst derivative
of the L
1
spline t at the nodes. Minimizing (4) over the manifold of cubic L
1
splines that minimize (5)
consists of nding the z that solves an overdetermined system
(A
0
A
1
)
_
z
b
_
= c (10)
of linear algebraic equations representing (4) in the
1
norm subject to the condition that, for given z, the
vector b solves an overdetermined system
(B
0
B
1
)
_
z
b
_
= 0 (11)
of linear algebraic equations representing (5) in the
1
norm. The equations in (10) are created by setting
the summands in (4) equal to zero. A
0
and A
1
are known matrices with M +(I + 1) rows and (I + 1)
columns. The equations in system (11) are the expressions inside the absolute value symbols in the rst
term of (5) evaluated at the midpoints of the subintervals and the expressions inside the absolute value
signs in the second term of (5). B
0
and B
1
are known matrices with IK+(I +1) rows and I +1 columns.
(Recall that K is the number of subintervals in the midpoint rule by which each integral on (x
i
, x
i+1
) in
expression (5) is discretized.)
In what follows, z
new
and b
new
(the new vectors z and b on each iteration), are vectors of dimension
I +1. The vectors w (weights), r (residuals) and p (temporary storage) are of dimension M +(I +1).
(Note: The vector of weights w of the Lagrange-multiplier primal afne method is not related to the
weights w
m
of the data points z
m
.) The vectors v (weights), s (residuals) and q (temporary storage)
are of dimension IK + (I + 1). D and E are square diagonal matrices of dimension M + (I + 1)
and IK + (I + 1), respectively. The quantity
conv
is a prescribed small positive scalar that occurs in
the convergence criterion. With this notation, the Lagrange-multiplier primal afne algorithm can be
described as follows.
48 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
Lagrange-multiplier primal afne algorithm.
Step 0. Set the iteration number k = 0. Set z = 0, b = 0, w = 0 and v = 0.
Step 1.0. Increase the iteration number k by 1.
Step 1.1. Let D be the diagonal matrix whose ith diagonal element is 1|w
i
|, where w
i
is the ith element
of w. Let E be the diagonal matrix whose ith diagonal element is 1 |v
i
|, where v
i
is the ith element
of v.
Step 1.2. (This step is the computationally most expensive step of the Lagrange-multiplier primal afne
algorithm.) Compute the least squares solution (z
T
new
, b
T
new
)
T
of the Lagrange-multiplier weighted least
squares system
_
_
A
T
0
D
2
A
0
A
T
0
D
2
A
1
B
T
0
E
2
B
1
A
T
1
D
2
A
0
A
T
1
D
2
A
1
B
T
1
E
2
B
1
B
T
1
E
2
B
0
B
T
1
E
2
B
1
0
_
_
_
_
z
new
b
new

_
_
=
_
_
A
T
0
D
2
c
A
T
1
D
2
c
0
_
_
. (12)
Here, is the (I + 1)-dimensional vector of Lagrange multipliers. By appropriate ordering of the
unknowns, one can store the 3(I + 1) 3(I +1) matrix of this system as a symmetric positive denite
matrix with 11 nonzero bands.
Step 1.3. If the normalized
1
norm, that is, the
1
norm divided by 2(I +1), of (z
T
new
z
T
, b
T
new
b
T
)
T
is less than
conv
, then set ag = 1. Otherwise, set ag = 0.
Step 1.4. Set z =z
new
and b = b
new
.
Step 1.5. If ag = 1 or if a maximum number of iterations has been reached, stop. Otherwise, go to
Step 2.1.
Step 2.1. Compute the vectors of residuals r = c A
0
zA
1
b and s = B
0
zB
1
b. Compute the vectors
p =D
2
r and q =E
2
s.
Step 2.2. Compute = max
i
(max(

i
1
i
,

i
1+
i
)), where
i
is the ith element of (p
T
, q
T
)
T
and
i
is the ith
element of (w
T
, v
T
)
T
.
Step 2.3. Compute new w = old w +(0.666666/)p and new v = old v +(0.666666/)q.
Step 2.4. Go to Step 1.0.
The standard primal afne algorithm for one-level problems without Lagrange multipliers is known to
converge (Vanderbei et al., 1986; Vanderbei, 1989; Adler and Monteiro, 1991; Tsuchiya and Muramatsu,
1992; Hall and Vanderbei, 1993). The Lagrange-multiplier primal afne method for two-level L
1
spline
tting problems has, in all of the computational experiments by the author, converged to solutions that
are consistent among themselves and consistent with the results for L
1
smoothing splines. However,
convergence characteristics of the Lagrange-multiplier primal afne method are not known theoretically.
We calculated cubic L
2
spline ts with the integral in (7) discretized by the same K-subinterval
midpoint rule used to discretize the integral for L
1
spline ts in (5). L
2
spline ts could, of course, be
calculated with the integral in (7) evaluated analytically. However, L
2
spline ts based on a discretized
integral were chosen for comparison with L
1
spline ts because the main goal of this paper is to
demonstrate that the fundamental solution of the shape-preservation problem is a proper choice of the
function spaces. The most relevant comparisons are therefore those in which only the function spaces
(and not, for example, the methods of evaluating the integrals) differ.
J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364 49
The L
1
and L
2
smoothing splines presented in this paper were calculated by the standard primal afne
method and by direct solution of a linear system, respectively, as described in (Lavery, 2000b). The
integrals in the smoothing spline functionals (8) and (9) were discretized by the K-subinterval midpoint
rule that was used for L
1
and L
2
spline ts.
4. Computational results
Computational experiments were carried out for the four sets of data and spline nodes presented in the
appendix. The rst three of these sets are the same as the sets used in (Lavery, 2000b). The data of the
fourth set is the same as the data of the third set but there are ve additional spline nodes in the fourth
set. The integrals in all of the spline and smoothing spline minimization principles were discretized by a
midpoint rule with K = 10 equal subintervals in each interval (x
i
, x
i+1
). The regularization parameters

0,i
and
1,i
were set equal to 10
4
for all i. The convergence parameter
conv
was chosen to be 10
6
.
The smoothing spline balance parameter of (8) and (9) was chosen to be 0.75 for data and node sets 1
and 2. For data and node sets 3 and 4, the smoothing spline balance parameter was chosen to be 0.1.
These choices were made because they resulted in the visually best shape-preserving performance of the
smoothing splines presented in (Lavery, 2000b). L
1
spline ts are presented in Figs. 1, 5, 9 and 13. L
2
spline ts are in Figs. 2, 6, 10 and 14. L
1
smoothing splines are in Figs. 3, 7, 11 and 15 and L
2
smoothing
splines are in Figs. 4, 8, 12 and 16. In all of these gures, the data points are represented by the symbol
+ and the spline nodes are represented by the symbol .
The L
1
spline ts of Figs. 1 and 5 are comparable in visual shape preservation to the L
1
smoothing
splines in Figs. 3 and 7 and superior to the L
2
spline ts in Figs. 2 and 6 and to the L
2
smoothing splines
in Figs. 4 and 8. The L
1
spline t of Fig. 9 seems to preserve shape better than the L
2
spline t of Fig. 10
but not as well as the L
1
smoothing spline of Fig. 11. It is challenging to say whether the L
1
spline t of
Fig. 9 preserves shape better than the L
2
smoothing spline of Fig. 12, but the L
1
spline t does have less
oscillation. If one adds nodes, the situation changes. The L
1
spline t of Fig. 13 preserves shape better
than the L
2
spline t of Fig. 14 and the L
2
smoothing spline of Fig. 16 but still not quite as well as the
L
1
smoothing spline of Fig. 15.
Since smoothing splines have an additional parameter, namely, the balance parameter, it is not
surprising that they are able to achieve better performance if the parameter is suitably chosen. However,
as is mentioned above at the end of the Introduction and below in Section 5.1, guidelines for choosing
the balance parameter are not known. Overall, the computational results show that L
1
spline ts preserve
shape well. In particular, L
1
spline ts represent discontinuities without oscillation and without much
overshoot.
In Table 1, the
1
,
2
and

norms of the differences between the curves and the data of Figs. 116
are given. The
1
and
2
norms have been normalized by dividing the sum of the absolute values and the
sum of the squares, respectively, by the number of data points. Since the L
1
curves of Figs. 1, 3, 5 and 7
are visually signicantly better than the L
2
curves of Figs. 2, 4, 6 and 8, one would expect that the error
norms for the L
1
curves would be smaller than those for the L
2
curves. That is, however, not always the
case. The entries for Figs. 916 in Table 1 are also only partially correlated with the statements about
shape preservation in the second paragraph above. The results presented in Table 1 correspond only in
a loose, general way to human visual preferences for shape preservation and conrm the widespread
50 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
Fig. 1. Cubic L
1
spline t for data and node set 1.
Fig. 2. Cubic L
2
spline t for data and node set 1.
J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364 51
Fig. 3. Cubic L
1
smoothing spline with = 0.75 for data and node set 1.
Fig. 4. Cubic L
2
smoothing spline with = 0.75 for data and node set 1.
52 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
Fig. 5. Cubic L
1
spline t for data and node set 2.
Fig. 6. Cubic L
2
spline t for data and node set 2.
J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364 53
Fig. 7. Cubic L
1
smoothing spline with = 0.75 for data and node set 2.
Fig. 8. Cubic L
2
smoothing spline with = 0.75 for data and node set 2.
54 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
Fig. 9. Cubic L
1
spline t for data and node set 3.
Fig. 10. Cubic L
2
spline t for data and node set 3.
J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364 55
Fig. 11. Cubic L
1
smoothing spline with = 0.1 for data and node set 3.
Fig. 12. Cubic L
2
smoothing spline with = 0.1 for data and node set 3.
56 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
Fig. 13. Cubic L
1
spline t for data and node set 4.
Fig. 14. Cubic L
2
spline t for data and node set 4.
J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364 57
Fig. 15. Cubic L
1
smoothing spline with = 0.1 for data and node set 4.
Fig. 16. Cubic L
2
smoothing spline with = 0.1 for data and node set 4.
58 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
Table 1

1
,
2
and

norms of the difference between the curves and the data of Figs. 116
L
1
spline t L
2
spline t L
1
smoothing spline L
2
smoothing spline

1
error 0.0501 (Fig. 1) 0.106 (Fig. 2) 0.0417 (Fig. 3) 0.0454 (Fig. 4)
0.102 (Fig. 5) 0.133 (Fig. 6) 0.0848 (Fig. 7) 0.0844 (Fig. 8)
1.97 (Fig. 9) 2.03 (Fig. 10) 0.604 (Fig. 11) 0.679 (Fig. 12)
1.36 (Fig. 13) 1.39 (Fig. 14) 0.488 (Fig. 15) 0.564 (Fig. 16)

2
error 0.140 (Fig. 1) 0.159 (Fig. 2) 0.116 (Fig. 3) 0.108 (Fig. 4)
0.202 (Fig. 5) 0.212 (Fig. 6) 0.220 (Fig. 7) 0.184 (Fig. 8)
3.23 (Fig. 9) 3.14 (Fig. 10) 1.50 (Fig. 11) 1.38 (Fig. 12)
2.68 (Fig. 13) 2.58 (Fig. 14) 1.52 (Fig. 15) 1.30 (Fig. 16)

error 0.744 (Fig. 1) 0.557 (Fig. 2) 0.507 (Fig. 3) 0.430 (Fig. 4)

1.00 (Fig. 5) 0.875 (Fig. 6) 1.14 (Fig. 7) 0.879 (Fig. 8)
13.2 (Fig. 9) 11.6 (Fig. 10) 8.05 (Fig. 11) 7.60 (Fig. 12)
12.4 (Fig. 13) 11.6 (Fig. 14) 10.8 (Fig. 15) 7.61 (Fig. 16)
Table 2
CPU times for the curves of Figs. 116
L
1
spline t L
2
spline t L
1
smoothing spline L
2
smoothing spline
0.0407 sec (Fig. 1) 0.0008 sec (Fig. 2) 0.0148 sec (Fig. 3) 0.0009 sec (Fig. 4)
0.0208 sec (Fig. 5) 0.0008 sec (Fig. 6) 0.0165 sec (Fig. 7) 0.0010 sec (Fig. 8)
0.0171 sec (Fig. 9) 0.0003 sec (Fig. 10) 0.0143 sec (Fig. 11) 0.0006 sec (Fig. 12)
0.0230 sec (Fig. 13) 0.0005 sec (Fig. 14) 0.0142 sec (Fig. 15) 0.0008 sec (Fig. 16)
impression that traditional
1
,
2
and

norms are not ideal norms for quantifying shape preservation.

Unfortunately, norms that better quantify shape preservation are not yet known.
It is premature to make conclusive statements about computing times of L
1
and L
2
spline ts and
L
1
and L
2
smoothing splines. Nevertheless, the reader may wish to have some information on relative
computing times even if it is only preliminary. We provide that information here with the caveat that
the computer codes used to generate the computational results were research codes written for proof of
principle, not for operational efciency and thus that the computing (CPU) times of the different methods
given here should be considered as indicating only relative orders of magnitude. The CPU times given
in Table 2 are for computations on a Gateway personal computer running under Microsoft Visual C++
Version 6.0.
The Lagrange-multiplier primal afne method for the L
1
spline ts of Figs. 1, 5, 9 and 13 converged
in 66, 36, 49 and 49 iterations, respectively. The basic primal afne method for the L
1
smoothing
splines of Figs. 3, 7, 11 and 15 converged in 16, 21, 20 and 22 iterations, respectively. L
1
spline ts
and smoothing splines are more expensive than corresponding L
2
spline ts and smoothing splines by
factors approximately equal to the number of iterations required to calculate the L
1
curves.
L
1
spline ts have been calculated for data sets much larger than those presented above. The CPU
times for larger data sets scale roughly linearly with the size of the data set. Representative results for
a 1000-point cross section of a set of 1-meter-posting digital elevation data for downtown Baltimore,
Maryland are given in Table 3.
J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364 59
Table 3
CPU times and numbers of iterations for 1000-point data set for Baltimore, Maryland
L
1
spline t L
2
spline t L
1
smoothing spline L
2
smoothing spline
0.4807 sec, 87 iter 0.0061 sec, 1 iter 0.1207 sec, 35 iter 0.0040 sec, 1 iter
5. Comments on L
1
spline tting
5.1. Advantages and disadvantages of L
1
spline ts
In the authors experience, the computational cost of calculating the coefcients of L
1
spline ts has
generally been higher than the cost of calculating the coefcients of L
1
smoothing splines. This situation
is due to the larger number of iterations typically required by L
1
spline ts and to the fact that the linear
systems for L
1
spline ts have 3(I + 1) 3(I + 1) matrices with 11 nonzero bands while the linear
systems for L
1
smoothing splines have 2(I + 1) 2(I + 1) matrices with 7 nonzero bands. However,
once the coefcients are known, it costs no more to evaluate L
1
spline ts than to evaluate L
1
smoothing
splines. Moreover, L
1
spline ts have two signicant advantages over L
1
smoothing splines: less storage
and no balance parameter. To store an L
1
spline t, one need store only the values of the spline t at the
nodes. The values of the rst derivative of the spline t at the nodes can be reconstructed by interpolating
the values of the spline at the nodes by an L
1
spline. In contrast, to store an L
1
smoothing spline, one
must store not only the values of the smoothing spline at the nodes but also the values of its rst derivative
at the nodes. Spline ts thus require only half as much storage as smoothing splines.
A second advantage of spline ts over smoothing splines is the elimination of the need for choosing a
parameter that balances the weights of the data tting functional and the spline functional. The balance
parameter is the in the smoothing spline minimization principles (8) and (9). Methods for determining
optimal balance parameters for L
2
smoothing splines have been investigated in (Wahba and Wold, 1975;
de Boor, 1978, Chapter XIV; Huber, 1979; Wecker and Ansley, 1983; Wahba, 1990). An excellent survey,
including a description of cross validation, unbiased risk and maximum likelihood procedures can be
found in Chapter 4 of (Wahba, 1990). However, general methods (other than human decision making)
for determining balance parameters for L
1
and L
2
smoothing splines in a way that preserves shape for
multiscale data are not yet known. This is perhaps not surprising, since shape preservation has not yet
been quantitatively characterized. Until the concept of shape preservation is claried, perhaps in different
ways for different classes of geometric models, choosing balance parameters for smoothing splines will
be fundamentally ad hoc and spline ts, which require no balance parameter, will have an advantage.
5.2. Node placement
It is well known that appropriate placement of nodes for splines is important. Node placement for
conventional L
2
smoothing splines and spline ts has been investigated in (de Boor, 1978, Chapter XIV;
Cox and Jones, 1987; Farin, 1997, Chapter 10). The node-placement procedures proposed in these and
other publications have been successful in certain areas but they typically assume sufcient smoothness
of the function that the data approximate, an assumption that is inappropriate for data that represent
multiscale phenomena with discontinuities.
60 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
Appropriate placement of nodes is as important for L
1
spline ts as it is for conventional L
2
spline ts
and for smoothing splines. The shape-preserving performance of the L
1
spline t of Fig. 13 is signi-
cantly better than the performance of the L
1
spline t of Fig. 9 because of the insertion of ve additional
nodes. Designing procedures for node placement that take the multiscale, nonsmooth nature of the data
into account is an important topic for future research. In this research, one may have to make use of
challenging non-Gaussian heavy-tailed statistics that often accompany the
1
norm.
5.3. The data-tting minimization principle
The use of the L
1
norm rather than the square of the L
2
norm in the spline minimization principle
clearly and strongly improves shape preservation. What norm or metric is optimal for the data-tting
minimization principle is not as clear. The spline tting results presented in this paper and the smoothing
spline results presented in (Lavery, 2000b) indicate that use of the
1
norm in the data-tting minimiza-
tion principle (4) is consistent with preserving shape in many cases. Is it optimal? From a statistical point
of view, the
1
norm is appropriate for data, the error of which follows a heavy-tailed distribution. If the
error in the data follows a Gaussian distribution, it is statistically more appropriate to use the square of the

2
norm rather than the
1
norm in the data-tting minimization principle. If the error in the data follows
a uniform distribution, it is statistically best to use the

norm. However, the

2
norm tends to induce
oscillation and the

norm induces equioscillation, which are inconsistent with the shape-preserving

preferences of many users. It may turn out that options other than the
p
norms should be considered
for the data-tting minimization principle. Much further research will be needed to identify or construct
optimal norms for the data-tting minimization principle. These optimal norms may well be precisely
the elusive norms that express shape preservation.
5.4. Extension to bivariate spline tting
Bivariate L
1
interpolating and smoothing splines have been investigated in (Lavery, 2001) and (Gilsinn
and Lavery, 2002), respectively. In direct analogy to the univariate case, bivariate L
1
spline ts can be
calculated by minimizing a bivariate data-tting functional over a manifold of bivariate L
1
splines. It is
likely to be more expensive to calculate the coefcients of bivariate L
1
spline ts than to calculate the
coefcients of bivariate L
1
smoothing splines, in part because of larger matrix size and bandwidth (see
related comments on the univariate situation in Section 5.1). However, bivariate spline ts cost no more
to evaluate than bivariate smoothing splines. Bivariate spline ts have two advantages analogous to those
pointed out in Section 5.1 for univariate spline ts: Bivariate spline ts do not involve a smoothing spline
balance parameter and require only one third of the storage of bivariate smoothing splines (because only
the values of the spline and not the values of the two partial derivatives of the spline need be stored at the
nodes).
6. Conclusion
Replacing the squares of the
2
and L
2
norms by the
1
and L
1
norms in the data-tting and spline
minimization principles has surprisingly large positive inuence on the shape-preserving properties of
J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364 61
spline ts. The results presented above indicate that cubic L
1
spline ts preserve the shape of multiscale
data well, nearly as well as L
1
smoothing splines with good (but typically a priori unknown) balance
parameters and better than L
2
spline ts and L
2
smoothing splines. Cubic L
1
spline ts do not require
human intervention to choose a parameter that balances weights of the data-tting minimization principle
and the spline minimization principle. L
1
spline ts are capable of greater data compression than L
1
smoothing splines (by a factor of 2 for univariate data and a factor of 3 for bivariate data). The univariate
L
1
spline ts introduced in this paper are in a framework that leads directly to bivariate extensions.
Appendix A
The sets of data and spline nodes used to generate Figs. 116 are presented in this appendix. The
symbols x, z, w and x denote the vectors with elements x
m
, m=0, 1, . . . , M, z
m
, m= 0, 1, . . . , M, w
m
,
m= 0, 1, . . . , M, and x
i
, i = 0, 1, . . . , I , respectively.
A.1. Data and node set 1 for Figs. 14
M = 81, I = 16,
x =(0, 0.2, 0.4, 0.6, 0.8, 1, 1.2, 1.4, 1.6, 1.8, 2, 2.2, 2.4, 2.6, 2.8, 3, 3.2,
3.4, 3.6, 3.8, 4, 4.2, 4.4, 4.6, 4.8, 5, 5.04, 5.08, 5.12, 5.16, 5.2, 5.24, 5.28,
5.32, 5.36, 5.4, 5.44, 5.48, 5.52, 5.56, 5.6, 5.64, 5.68, 5.72, 5.76, 5.8, 5.84, 5.88, 5.92,
5.96, 6, 6.04, 6.08, 6.12, 6.16, 6.2, 6.24, 6.28, 6.32, 6.36, 6.4, 6.44, 6.48, 6.52, 6.56,
6.6, 6.64, 6.68, 6.72, 6.76, 6.8, 6.84, 6.88, 6.92, 6.96, 7, 7.04, 7.08, 7.12, 7.16, 7.2),
z =(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0),
w = 1,
x =(0, 1, 2, 3, 4, 5, 5.2, 5.4, 5.6, 5.8, 6, 6.2, 6.4, 6.6, 6.8, 7, 7.2).
A.2. Data and node set 2 for Figs. 58
M = 117, I = 16,
x =(0, 0.2, 0.4, 0.6, 0.8, 1, 1.2, 1.4, 1.6, 1.8, 2, 2.2, 2.4, 2.6, 2.8,
3, 3.05, 3.1, 3.15, 3.2, 3.25, 3.3, 3.35, 3.4, 3.45, 3.5, 3.55, 3.6, 3.65, 3.7,
3.75, 3.8, 3.85, 3.9, 3.95, 4, 4.05, 4.1, 4.15, 4.2, 4.25, 4.3, 4.35, 4.4, 4.45,
4.5, 4.55, 4.6, 4.65, 4.7, 4.75, 4.8, 4.85, 4.9, 4.95, 5, 5.05, 5.1, 5.15, 5.2,
5.25, 5.3, 5.35, 5.4, 5.45, 5.5, 5.55, 5.6, 5.65, 5.7, 5.75, 5.8, 5.85, 5.9, 5.95,
62 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
6, 6.5, 6.55, 6.6, 6.65, 6.7, 6.75, 6.8, 6.85, 6.9, 6.95, 7, 7.1, 7.2, 7.3,
7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 8, 8.2, 8.4, 8.6, 8.8, 9, 9.2, 9.4, 9.6,
9.8, 10, 10.2, 10.4, 10.6, 10.8, 11, 11.2, 11.4, 11.6, 11.8, 12),
z =(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.05, 0.1, 0.15, 0.2,
0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95,
1, 1, 1.025, 1.05, 1.075, 1.1, 1.125, 1.15, 1.175, 1.2, 1.225, 1.25, 1.275, 1.3, 1.325,
1.35, 1.375, 1.4, 1.425, 1.451.475, 1, 1, 1, 1, 1, 1, 1, 2, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1),
w = 1,
x =(0, 1, 2, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 8, 9, 10, 11, 12).
A.3. Data and node set 3 for Figs. 912
The data in this set are a cross section of a digital elevation map for Fort Hood near Killeen, Texas ob-
tained from the Terrain Resource Repository of the Terrain Modeling Project Ofce (TMPO) on the Na-
tional Imagery and Mapping Agencys WWW home page at http://www.nima.mil/geospatial/geospatial.
html.
M = 81, I = 8,
x =(0, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000,
1100, 1200, 1300, 1400, 1500, 1600, 1700, 1800, 1900, 2000,
2100, 2200, 2300, 2400, 2500, 2600, 2700, 2800, 2900, 3000,
3100, 3200, 3300, 3400, 3500, 3600, 3700, 3800, 3900, 4000,
4100, 4200, 4300, 4400, 4500, 4600, 4700, 4800, 4900, 5000,
5100, 5200, 5300, 5400, 5500, 5600, 5700, 5800, 5900, 6000,
6100, 6200, 6300, 6400, 6500, 6600, 6700, 6800, 6900, 7000,
7100, 7200, 7300, 7400, 7500, 7600, 7700, 7800, 7900, 8000),
z =(280, 280, 280, 280, 280, 280, 280, 281, 281, 281, 281,
280, 280, 280, 279, 279, 278, 278, 277, 277, 276,
276, 276, 276, 275, 276, 277, 278, 278, 279, 280,
300, 300, 300, 300, 300, 300, 301, 302, 302, 303,
303, 303, 303, 303, 303, 303, 305, 305, 305, 306,
306, 306, 308, 308, 308, 307, 306, 305, 305, 304,
303, 302, 302, 301, 300, 300, 300, 300, 300, 300,
300, 300, 300, 300, 300, 300, 300, 300, 300, 300),
J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364 63
w = 1,
x =(0, 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000).
A.4. Data and node set 4 for Figs. 1316
This data and node set is the same as set 3 except that there are ve additional spline nodes.
M = 81, I = 13,
x [same as for set 3],
z [same as for set 3],
w = 1,
x =(0, 1000, 2000, 3000, 3500, 4000, 4500, 5000, 5500, 6000, 6500, 7000, 7500, 8000).
References
Adler, I., Monteiro, R.D.C., 1991. Limiting behavior of the afne scaling continuous trajectories for linear programming. Math.
Programming 50, 2951.
Anselone, P.M., Laurent, P.J., 1968. A general method for the construction of interpolating and smoothing spline functions.
Numer. Math. 12, 6682.
Bloomeld, P., Steiger, W.L., 1983. Least Absolute Deviations: Theory, Applications, and Algorithms. Birkhuser, Boston.
Cox, M.G., Jones, H.M., 1987. Shape preserving spline approximation in the
1
-norm. In: Mason, J.C., Cox, M.G. (Eds.),
Algorithms for Approximation. Clarendon Press, Oxford, pp. 115129.
de Boor, C., 1978. A Practical Guide to Splines. In: Applied Mathematical Sciences, Vol. 27. Springer-Verlag, New York.
Farin, G., 1997. Curves and Surfaces for Computer-Aided Geometric Design: A Practical Guide, 4th Ed. Academic Press, San
Diego.
Gilsinn, D.E., Lavery, J.E., 2002. Shape-preserving, multiscale tting of bivariate data by cubic L
1
smoothing splines. In:
Chui, C.K., Schumaker, L.L., Stckler, J. (Eds.), Approximation Theory X: Wavelets, Splines, and Applications. Vanderbilt
University Press, Nashville, TN, pp. 283293.
Greville, T.N.E., 1969. Introduction to spline functions. In: Greville, T.N.E. (Ed.), Theory and Applications of Spline Functions.
Academic Press, New York, pp. 135.
Hall, L.A., Vanderbei, R.J., 1993. Two-thirds is sharp for afne scaling. OR Letters 13, 197201.
Huber, P.J., 1979. Robust smoothing. In: Launer, R.L., Wilkinson, G.N. (Eds.), Robustness in Statistics. Academic Press, New
York, pp. 3347.
Lavery, J.E., 2000a. Univariate cubic L
p
splines and shape-preserving, multiscale interpolation by univariate cubic L
1
splines.
Computer Aided Geometric Design 17, 319336.
Lavery, J.E., 2000b. Shape-preserving, multiscale tting of univariate data by cubic L
1
smoothing splines. Computer Aided
Geometric Design 17, 715727.
Lavery, J.E., 2001. Shape-preserving, multiscale interpolation by bi- and multivariate cubic L
1
splines. Computer Aided
Geometric Design 18, 321343.
Lyche, T., Schumaker, L.L., 1973. Computation of smoothing and interpolating natural splines via local bases. SIAM J. Numer.
Anal. 10, 10271038.
Mehlum, E., 1964. A curve tting method based on a variational criterion. BIT 4, 213223.
Reinsch, C.H., 1967. Smoothing by spline functions. Numer. Math. 10, 177183.
Reinsch, C.H., 1971. Smoothing by spline functions, II. Numer. Math. 16, 451454.
Schoenberg, I.J., 1964. Spline models and the problem of graduation. Proc. Nat. Acad. Sci. USA 52, 947950.
Schumaker, L.L., 1969. Some algorithms for the computation of interpolating and approximating spline functions. In: Greville,
T.N.E. (Ed.), Theory and Applications of Spline Functions. Academic Press, New York, pp. 87102.
64 J.E. Lavery / Computer Aided Geometric Design 21 (2004) 4364
Tsuchiya, T., Muramatsu, M., 1992. Global convergence of a long-step afne scaling algorithm for degenerate linear
programming problems. SIAM J. Optim. 5, 525551.
Utreras, F.I., 1981. Optimal smoothing of noisy data using spline functions. SIAM J. Sci. Stat. Comput. 2, 349362.
Vanderbei, R.J., 1989. Afne-scaling for linear programs with free variables. Math. Programming 43, 3144.
Vanderbei, R.J., Meketon, M.J., Freedman, B.A., 1986. A modication of Karmarkars linear programming algorithm.
Algorithmica 1, 395407.
Wahba, G., 1975. Smoothing noisy data with spline functions. Numer. Math. 24, 383393.
Wahba, G., 1990. Spline Models for Observational Data. SIAM, Philadelphia, PA.
Wahba, G., Wold, S., 1975. A completely automatic French curve. Comm. Statist. 4, 117.
Wecker, W., Ansley, C., 1983. The signal extraction approach to non-linear regression and spline smoothing. J. Amer. Statist.
Assoc. 78, 8189.
Woodford, C.H., 1971. An algorithm for data smoothing using spline functions. BIT 10, 501510.