Professional Documents
Culture Documents
o f B e s t M i n i m a x - A p p r o x i m a t i o n to a F u n c t i o n
Defined on a F i n i t e P o i n t Set*
PHILIP C. CURTIS, JR. AND WERNER I~. FRANK
Statement of Problem
Given a function f ( x ) defined on a finite point set S = {xl, . . . , XN}, it is
desired to approximate this function by the unique polynomial p ( x ) , of degree
less than or equal to n, which satisfies the minimax or Chebyshev criterion
The system (2) m a y always be solved for the unknowns a0, • • • , a~, & More-
over, there exists u in T such that the solution of (2) satisfies (3), and if q(x)
is the polynomial with coefficients a0, • • • , a~, then q(x) is the polynomial p ( x )
of best approximation to f ( x ) in the sense of (1). Conversely, if p ( x ) is a poly-
nomial satisfying (1), then there is a vector u in T such that the coefficients of
p ( x ) satisfy (2) and (3). The coefficients of p ( x ) are uniquely determined by
this property, although the vector u is not necessarily unique. For proofs of these
facts see [3].
A practical method for determining a set of n + 2 points with the desired
property is due to Remes [5]. This was extended to more general ciasses of func-
tion by Novodvorskii and Pinsker [4], and numerical applications have been
described by Shenitzer [6]. This algorithm is as follows:
1) Select an arbitrary vector u from T.
2) Solve the linear system (2). If the coefficient a0, al, • • • , am and the error
n j
~t satisfy (3), then the iteration ceases. If not, let p ( x ) = ~ j = o a~x and proceed
to step 3.
3) A new vector u' = (u0', ul', • .. , u~+~) from T is chosen such t h a t for each
• Received'August, 1958.
395
396 P.C. CURTIS, JR., AND W. L. FRANK
k = 0, 1, . - . , n -t- 1,
l f(u~') - / ~ ( u J ) I --> I~1
sgn [f(ud) - p(ud)] = (-1) sgn[f(uk_l)
' - p(uk-1)],
" ' k = 1, .-- n "b 1
and
max I f ( u d ) - p(uk') [ = max I f ( x 0 - /,(x,)I.
k = 0 , 1 , • • *, n-t-1 i--0,1,- • •,N
B~(u)
k~0
where
n+l
Bk(u) = ~ l u~ ~- uk 1-1. (4)
lm0
] / ( u ~ ) - p (u k ) ].
POLYNOMIAL OF BEST M I N I M A X A P P R O X I M A T I O N 397
Form u' by replacing one coordinate of u by x ' in such a way that f ( x ) - q(x)
still oscillates on u ~ (i.e.,
sgn I f ( u / ) - q ( u / ) ] = ( - 1 ) sgn [f(u:.-1) - q(u~-l)], j = 1,..-,n + 1).
Then by the reasoning at the end of the proof of theorem 1,
I I > In(u) I.
On the other hand, suppose for some j, 0 =< j =< n, uj < x' < u j + ~ , t h e n x '
replaces u; or u;+~. But this contradicts the fact t h a t 1~;(x') [ _-< I ~(u) [ and
I ~j+l(X') [ ~ ] ~(u) [- If u~+~ < x' ~ XN, then u' is either ( n o , u l , . . . , u ~ , x ' )
or (u~, u~, - . . , u~+~, x'). But
] ~(U0, U i , " ' " , U n , Xt) I = I ~n+l(x/) [ ~ I~(u) I
and
f (ul, ..., u +l, z') [ = [ I < is(u) [.
A similar verification can be made if x0 =< x' ~ u0. Hence I ~(u') [ =< [6(u) l,
which is a contradiction.
POLYNOMIAL OF BEST MINIMAX APPROXIM2~T[ON 399
In the case that the finite point set S is replaced by an interval [a, b], a maxi-
mization procedure and proof of convergence has been announced by Bratten
[1]. For applications to more generM families of approximating functions see [2].
Numerical Results
In order to test the second algorithm a subroutine was prepared for the UNIVAC
Scientific 1103A. The computation is performed in floating point arithmetic,
where the number representation consists of an 8 binary bit exponent and a 27
binary bit fractional part. The input is the tabular data (xi, f(x~)) and the de-
gree of the desired polynomial approximation. Normalization of the independent
variable is made to the interval [ - 1 , . 1] by the routine.
The program gives the following output:
A. Basic Output
1. Coefficients of the normalized approximating polynomial over [ - 1 , 1]
2. ~, as obtained from equation (1)
3. Transformation which carries [xl, x~] to [--1, 1]
B. Optional Output
1. Identification of the successive vector u and associated approximation
/t for each step of the iteration
2. Coefficients of polynomial over Ix1, x~]
3. Table of xi , f(xi), p(xi), f(x~) -- p(xi), where p(x) is computed over
[ - 1 , 1] a n d / o r [xl, x~]
4. Additional table of values zi, p(z~) where the zi are any desired argu-
ments in [xl, XN]. The computation of p(zi) is performed by first
normalizing zi to [ - 1, 1] and using the normalized polynomial.
A number of features of the code deserve special comment:
1. To start the iteration off it is desirable to pick a vector u for which the
magnitude of the corresponding error 5 is as large as possible. An a priori best
choice for the coordinates of such a vector are the n + 2 abscissae associated
with the extrema of the Chebyshev polynomial of degree n + 1 over the interval
[ - 1 , 1], i.e.,
k~r
~k = - c o s k = 0,1, . . - n--l- 1.
n+l'
If S does not contain these points one selects close approximations to them. The
reason why this choice is "best" can be seen if one assumes that f ( x ) is a poly-
nomial of degree n ~ 1. The error function E(x), associated with the minimax
solution of order less than or equal to n is proportional to the Chebyshev poly-
nomial of degree n -b 1 which attains its extrema at the points defined above.
2. To avoid difficulties which numerical round-off might contribute, it is prac-
tical in the maximizing process to compare ~i(x) to (1 --k e)~(u) for some small
> 0. This has the desired effect of eliminating unnecessary iterations which
may be required due to noise generated in computing ~1(x). I t has been found
that a value of e = 10-5 is satisfactory.
3. A substantial reduction in the amount of computation is realized b y avoid-
r-
400 P.C. CURTIS~ JR.~ A N D W . L. F R A N K
Ix -
B/ = Bk Ix' u~l"
I n t h e case t h a t k = j , t h e o r i g i n a l f o r m u l a for Bk is used. T h i s p r o c e d u r e has,
h o w e v e r , t h e d i s a d v a n t a g e of i n t r o d u c i n g some round-off errors d u e to t h e loss
of significance which results w h e n x o r x ' is close to u k .
T a b l e s 1 a n d 2 e x h i b i t n u m e r i c a l results in m a k i n g m i n i m a x a p p r o x i m a t i o n s
to a n u m b e r of functions. W e o b s e r v e t h e following:
1. A l t h o u g h t h e m a g n i t u d e of 6 decreases m o n o t o n i c a l l y as t h e degree of t h e
a p p r o x i m a t i n g p o l y n o m i a l increases, t h i s p r o p e r t y is v i o l a t e d w h e n 6 b e c o m e s of
t h e o r d e r of a few u n i t s in t h e 8 t h significant d i g i t of t h e i n p u t d a t a , f(x) I n t h i s
case, d u e to t h e effect of round-off error, one no longer a c h i e v e s p r o p e r con-
v e r g e n c e a n d I 6 I m a y b e g i n t o increase, a n d / o r t h e e x t r e m e s of t h e e r r o r func-
t i o n E(x) m a y be g r e a t e r t h a n [6 [. E v e n t h o u g h a m i n i m a x s o l u t i o n is t h e r e -
fore n o t o b t a i n e d , i t h a s b e e n o b s e r v e d , for t h e a p p r o x i m a t i o n s considered, t h a t
t h e fit o b t a i n e d is a t w o r s t o n l y a few u n i t s off in t h e 8 t h significant digit• W h i l e
t h e b e s t a p p r o x i m a t i n g p o l y n o m i a l is n o t d e t e r m i n e d one still h a s o b t a i n e d a
r e a s o n a b l e fit.
T a b l e 1, w h i c h is a s t u d y in a p p r o x i m a t i n g t h e a r c s i n e f u n c t i o n for t h e i n t e r v a l
[0, ~ / ~ / 2 ] , i l l u s t r a t e s t h e s e remarks• T h e h o r i z o n t a l lines in t h e t h r e e c o l u m n s
TABLE 1
ApproximationofArcsi, xover[O,~]
IS] max E(xO
Degree of i
Approximating
Polynomial x~ Arcsin
Arcsin • Arcsinx -- x -- ~- W ffi - -
x
xW
12 .46 X 10-s
POLYNOMIAL OF BEST MINIMAX APPROXIMATION 401
TABLE 2
as i n d i c a t e d in c o l u m n three. T h e r e m o v i n g of a m a j o r c o m p o n e n t of t h e func-
t i o n d e l a y s t h e effect of round-off error in t h i s case for t w o m o r e stages. T h e de-
sired fit to arcsin x is t h e n r e c o v e r e d b y s i m p l y a d d i n g x + x3/6 to t h e p o l y n o m i a l
w h i c h a p p r o x i m a t e s g(x). I n t h e p r o b l e m cited here t h i s p r o c e d u r e h a d t h e de-
sired effect of r e d u c i n g t h e m a x i m u m d e v i a t i o n of t h e e r r o r f u n c t i o n f r o m 8 to 3
u n i t s in t h e 8 t h significant figure for t h e case n = 10.
A second device, which also h a s t h e effect of r e m o v i n g e r r o r d u e t o round-off,
is to a p p r o x i m a t e t h e i n p u t f u n c t i o n f(x) b y a p o l y n o m i a l p(x) a n d t h e n per-
f o r m a f i t of t h e error f u n c t i o n E(x) = f(x) - p(x). W h i l e t h e s e c o n d p o l y -
n o m i a l should be zero in t h e o r y , one a c t u a l l y o b t a i n s t h e fit p~(x) b e c a u s e of
round-off errors. A n i m p r o v e d fit to f(x) is t h e n o b t a i n e d b y t a k i n g t h e p o l y -
n o m i a l p(x) + p~(x).
2. K n o w l e d g e of t h e p r o p e r t i e s of a f u n c t i o n can m a t e r i a l l y a i d in finding
m o r e efficient fits. F o r e x a m p l e , consider a n even f u n c t i o n f(x) defined o v e r
[--a, a]. One can apply the minimax theory and obtain the function of best
approximation to f(x) which in theory is an even polynomial of degree k. In
practice the actual fit is not exactly even. I t is possible, however, to fit ] ( % / ~ )
over [0, w/a] by a polynomial p(co) of degree k and then make the change of
variable co = x ~ in order to obtain a minimax fit which is an even polynomial p(x)
of degree 2k over [ - a , a]. B y applying this device one can obtain a better fit
for the same number of coefficients.
A similar consideration can be given odd functions. Assuming the existence of
limf('v/~) _ g(~/~),
V;
we approximate g(%/~) by a polynomial p(o~) of degree k over [0, w/a]. Making
the change in variable oo = x ~ we obtain the even polynomial p(x). The approxi-
mation to f(x) is then attained by forming xp(x), which is an odd polynomial.
This procedure does not, of course, produce the best approximating polynomial.
Nevertheless, as is exhibited in column four and five of table 1, the resulting
approximation is far better than the normal results of column 2. In fact, for
n = 4 the improvement is more than two orders of magnitude.
3. The slow rate of decrease of [6] for increasing n for the f u n c t i o n f ( x ) = I x5 I
is predictable since the fourth derivative of f fails to exist at the origin. In fact,
it is well known that for such a function there exists a constant C, such that
for each n the best approximation ~ __< C/n 3.
4. The graph of the acceleration data in the last column of table 2 is given by
figure 1. The behavior of the ~'s suggests that this function is essentially a fourth
degree polynomial, and no substantial improvements can be obtained by ap-
proximations of higher degree.
The error function, for the approximation n = 4, is given by figure 2. Unlike
most error functions which are smooth and have only n + 1 zeros, this particular
120
I10
ioo
2
90
80
IZ 1'3 14 15
T i m e in Seconds
F~G. I
POLYNOMIAL OF BEST MINIMAX APPROXIMATION 403
A
/
F I G . 2.
one exhibits many fluctuations and the convergence was slightly slower than
usual. Such problems offer a good test to study the convergence of the method
and efficiency of the machine program.
5. The acceleration data of table 2 is an example of data for which a minimax
solution is attained over the interval [ - 1 , 1] but the polynomial transformed to
Ix1, x~] does not satisfy the convergence criteria. An illustration of this difficulty
is given by this data for n = 81 for which the error function over the restored
interval [12, 18] had extremes of 6&
The transformation of the polynomial from coefficients as over [ - 1 , 1] to
coefficients bl over Ix1, xN] is given by:
i-- 0,11...,nl
where
and
2 Xl - ~ x N
O/ -- , ~ -- •
XN ~ Xl Xl ~ XN
second procedure the function ~ is always computed from the input data Ix, f(x)],
while in the first method the evaluation of the error function depends in sequence
upon the solution of a linear system, the computation of the value of a poly-
nomial, and finally the difference f ( x ) - p ( x ) . The latter computation results in
a loss of significant figures when p ( x ) is close to f ( x ) . Experimental evidence is
at hand which suggests t h a t the second algorithm can do better when the error
function E ( x ) approaches the order of the limits of the precision of the computer.
The successive vectors u chosen by each respective algorithm will in general
not be the same even though the initial sets are equal. As a result, one or the
other of the two methods m a y converge faster. Unfortunately, lack of experience
with the first algorithm prohibits any conclusions. However, one m a y still dis-
cuss the difference in the number of operations (multiplications and divisions)
associated with the basic computation for the two procedures.
A complete cycle r for either of the algorithms is defined to be respectively the
determination of E ( x ) over the entire interval [ - 1 , 1] or the successive deter-
mination of 8i(x), j = 0, . . . , n T 1, for a particular choice of a vector u.
The number of cycles r required for the first procedure must be integral and
greater than or equal to 2 unless the chosen initial set of n -t- 2 points happens
to be the solution. In [6], r is reported to be in the range 6-8.
On the other hand the number of cycles for the second method can be frac-
tional, and r has been observed to be approximately 2 for most problems.
Considering t h e cycle to be the unit which measures the convergence we ob-
• tain the following results, where n is the degree of the approximating polynomial
and N is the number of points in S:
Method Number of multiplications and divisions
I: r-V(n + 1 ) ( n W 2 ) ( n + 6) + 3nhr 1
3
II: r[(n + 1 ) ( n -k 2) + N ( 7 n -~ 9)] + n + 2 [ n ~..kl0n+ 6]
In the practical range of interest, say n =< 10 and N = 100, one can probably
do better with respect to computing time when using method I. However, in
either case the total computer time requirements are quite small.
REFERENCES
1. D. BRATTEN,New results in the theory and techniques of Chebyshev fitting. Abstract
No. 546-34, Notices Amer. Math. Soc. 5 (1958) 248.
2. P. C. CURTIS, JR., n-parameter families and best approximations. To appear; see also
Abstract No. 548-70, Notices Amer. Math. Soc. 5 (1958), 496.
3. C. DE LAVALL~EPOUSSIN,Lecons sur l'approximations des fonctions d'une variable r~ele.
Gauthier Villars, Paris, 1952.
4. E. N. NOVODVORSKIIAND I. SH. PINSKER, On a process of equalization of maxima.
Usp. ,Mat. Nauk 6 (1951), 174-181 [Russian]. (English translation by A. Shenitzer
available from New York University.)
5. YA L. RE~S, On a method of Chebyshev type approximation of functions. Ukr. A.h r.
1935.
6. A. SHENITZER,Chebyshev approximation of a continuous function by a class of func-
tions, J. Assoc. Comp. Mach. 4 (1957), 30-35.