Journal of Computational and Applied Mathematics 255 (2014) 74–85

Contents lists available at ScienceDirect

Journal of Computational and Applied

Mathematics
journal homepage: www.elsevier.com/locate/cam

Complexity of interior-point methods for linear optimization

based on a new trigonometric kernel function
M. Reza Peyghami a,c,∗ , S. Fathi Hafshejani a,c , L. Shirvani b
a
Department of Mathematics, K.N. Toosi University of Tech., P.O. Box 16315-1618, Tehran, Iran
b
Department of Mathematics, Isfahan University, P.O. Box 81745-163, Isfahan, Iran
c
Scientific Computations in OPtimization and Systems Engineering (SCOPE), K.N. Toosi University of Tech., P.O. Box 16315-1618,
Tehran, Iran

article info abstract

Article history: In this paper, we propose a new kernel function with trigonometric barrier term for primal–
Received 12 July 2012 dual interior point methods in linear optimization. Using an elegant and simple analysis and
Received in revised form 5 April 2013 under some easy to check conditions, we explore the worst case complexity result for the
large update primal–dual interior point methods. We obtain the  worst case
 iteration bound
Keywords: 2
Kernel function for the large update primal–dual interior point methods as O n 3 log nϵ which improves
Linear optimization the so far obtained complexity results for the trigonometric kernel function in [M. El Ghami,
Primal–dual interior-point methods Z.A. Guennoun, S. Boula, T. Steihaug, Interior-point methods for linear optimization based
Large-update methods on a kernel function with a trigonometric barrier term, Journal of Computational and
Applied Mathematics 236 (2012) 3613–3623] significantly.
© 2013 Elsevier B.V. All rights reserved.

1. Introduction

After the landmark paper of Karmarkar [1], the interior point methods became an active area of research. Nowadays,
introducing an efficient variant of the polynomial time interior point methods with low complexity results is the main
challenge in this area of research.
In this paper, we deal with primal–dual Interior-Point Methods (IPMs) for solving the standard Linear Optimization (LO)
problem:

min{c T x : Ax = b, x ≥ 0} (P)
where A ∈ R m×n
with rank(A) = m ≤ n, x, c ∈ R and b ∈ R . The dual problem of (P) is given by
n m

max{bT y : AT y + s = c , s ≥ 0} (D)
m n
where y ∈ R and s ∈ R . One knows that the necessary and sufficient conditions for the existence of optimal solutions of
(P) and (D) lead us to the following nonlinear system:
Ax = b, x≥0
A y + s = c,
T
s≥0 (1)
xs = 0.

∗ Corresponding author at: Department of Mathematics, K.N. Toosi University of Tech., P.O. Box 16315-1618, Tehran, Iran. Tel.: +98 21 23064316.
E-mail addresses: peyghami@kntu.ac.ir (M.R. Peyghami), sfathi@mail.kntu.ac.ir (S.F. Hafshejani), leila.shirvani@sci.ui.ac.ir (L. Shirvani).

0377-0427/$ – see front matter © 2013 Elsevier B.V. All rights reserved.
http://dx.doi.org/10.1016/j.cam.2013.04.039
 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85 75

The first and second equations in (1) require the feasibility of x and (y, s) for (P) and (D), respectively, whereas the third
equation is the so-called complementarity condition for (P) and (D). Note that xs indicates to the coordinate-wise (Hadamard)
product of the vectors x and s.
Without loss of generality, we may assume that the problems (P) and (D) satisfy the interior-point condition (IPC) [2],
i.e., there exist x0 and (y0 , s0 ) such that

Ax0 = b, x0 > 0
A y + s = c,
T 0 0
s0 > 0.

Let us briefly discuss about the idea behind generic primal–dual IPMs. In these methods, the third equation in (1)
is replaced by a parametric equation xs = µe, where µ is a positive parameter and e denotes the all-one vector, i.e.,
e = (1, 1, . . . , 1)T . It can be easily shown that, under IPC condition and the full row rankness of A, the system obtained
from (1) by replacing the complementarity condition with the parametric equation, i.e.,

Ax = b, x≥0
A y + s = c,
T
s≥0 (2)
xs = µe

has a unique solution, for each µ > 0. We denote such a solution as (x(µ), y(µ), s(µ)), and we call x(µ) as the µ-center of
(P) and (y(µ), s(µ)) as the µ-center of (D). The set of all µ-centers, with µ > 0, forms a homotopic path which is called
the central path of (P) and (D) [3,4]. The central path for linear optimization was first recognized by Sonnevend [4] and
Megiddo [5]. It has been proved that as µ → 0, the limit of the central path exists and converges to an analytic center of the
optimal solutions set of (P) and (D).
For fixed µ > 0, a direct application of the Newton method on the system (2) gives the following system for displacements
∆x, ∆y and ∆s in order to produce the new point:
A∆x = 0,
AT ∆y + ∆s = 0, (3)
x∆s + s∆x = µe − xs.

Therefore, the new iterate is computed as

x+ = x + α ∆x, y+ = y + α ∆y, s+ = s + α ∆s

where α ∈ (0, 1] is obtained by using some rules so that the new iterate satisfies (x+ , s+ ) > 0. Let us define the scaled
vector v as

xs
v := .
µ
Using the scaled vector v , the Newton system (3) can be rewritten as:

Ādx = 0
ĀT dy + ds = 0 (4)
d x + d s = v −1 − v

where
1
Ā := AV −1 X = AS −1 V
µ
V := diag(v), X := diag(x), S := diag(s) (5)
v ∆x v ∆s
dx = , ds = .
x s
Note that dx = ds = 0 if and only if v − v −1 = 0 if and only if x = x(µ), s = s(µ).
2
Assuming ψc (t ) = t 2−1 − log t, for t > 0, it can be easily seen that ψc (t ) is a strictly differentiable convex barrier (kernel)
function on Rn++ with ψc (1) = ψc′ (1) = 0, i.e. it attains its minimal value at t = 1. Note that the right hand side of the third
equation in (4) is the minus gradient of the proximity function
n

Ψc (v) = ψc (vi ).
i=1
76 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85

By replacing the proximity function Ψc (v) by a proximity function Ψ (v) = i=1 ψ(vi ), where ψ(t ) is any strictly
n
differentiable convex barrier function on Rn++ with ψ(1) = ψ ′ (1) = 0, the system (4) is converted to the following system:
Ādx = 0
ĀT dy + ds = 0 (6)
dx + ds = −∇ Ψ (v).
We reassert that in (6), dx = ds = 0 holds if and only if Ψ (v) = 0 if and only if v = e if and only if (x, s) = (x(µ), s(µ)), as
it should.
Now, we briefly describe one step of the primal–dual interior point method with the above mentioned attitude. Starting
with the initial interior point (x0 , y0 , s0 ), µ0 > 0 and the proximity function Ψ (v), assume that a good approximation of the
µ-center (x(µ), y(µ), s(µ)) is known for µ > 0 at the current iteration. We decrease the parameter µ by a factor 1 − θ , for
θ ∈ (0, 1), and set µ := (1 − θ )µ. Then, an approximate solution of the µ-center is obtained by frequently using Newton
method. In fact, we first solve the system (6) for dx and ds and then find the Newton directions ∆x, ∆y and ∆s using (5).
It is worth mentioning to note that while (x, y, s) ̸= (x(µ), y(µ), s(µ)), we have a nonzero (∆x, ∆s, ∆y) and therefore a
nonzero (dx , dy , ds ). This procedure is repeated until we get to the point in which xT s < ϵ , for given accuracy ϵ > 0. In this
case, we say that the current x and (y, s) are ϵ -approximate solutions of the primal and dual problems, respectively. Note
that, since A has full row rank, then system (6) uniquely defines a search direction (dx , dy , ds ) for any (x, s) > 0.
The above mentioned procedure can be outlined in the following primal–dual interior point scheme [6].
Algorithm 1. Generic Primal–dual Algorithm for LO
Input
a proximity function Ψ (v)
a threshold parameter τ > 0
an accuracy parameter ε > 0
a barrier update parameter θ , 0 < θ < 1
begin
x := e; s := e; µ := 1; v := e;
while nµ > ε do
begin
µ := (1 − θ )µ;
while Ψ (v) > τ do
begin
x := x + α △ x
s := s + α △ s
y=y+α△y

v := xs
µ
end
end
end
Algorithm 1 consists of inner and outer iterations. Each outer while loop (iteration) includes an update of parameter µ
and a sequence of (one or more) inner iterations. The total iteration number is described as a function of the dimension n
and ε . The choice of θ , the so-called barrier update parameter, plays an important role in theory and practice of IPMs. If θ is a
constant, let say θ = 12 , then Algorithm 1 is called as a large-update method, while for the case when the θ is depended on n,
let say θ = √1n , then Algorithm 1 is named as a small-update method. It is worth mentioning that the small-update methods
have the best iteration bound in theory while the large-update methods are practically efficient [2]. This phenomenon is
mentioned as ‘‘The gap between theory and practice’’ in the literature.
The interior point methods based on barrier functions have been widely studied in the literature. The so called self-
concordant barrier functions which were first introduced by Nesterov and Nemirovskii in [7], allowed the interior point
methods for LO to be extended to the more general class of convex optimization problems such as nonlinear convex pro-
gramming, nonlinear complementarity problems, Second Order Cone Programming (SOCP) and Semidefinite Optimization
(SDO) problems. For large neighborhood IPMs based on self-concordant function, the so far best known iteration bound is
O(n log nϵ ). This bound has been improved by Peng et al. in [6] by introducing the well known Self-Regular (SR) proximity
measure. They constructed primal–dual interior point methods based on SR kernel functions for linear optimization and
extended it to √the SDO, SOCP and complementary problems. They obtained the so far best known worst case iteration com-
plexity as O( n log n log nϵ ). Later, many attempts have been done to introduce a non-SR kernel functions for at least meeting
the above mentioned best known complexity [8–10]. A comparative study on the kernel function can be found in [11].
 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85 77

Recently, El Ghami et al. in [12] proposed a trigonometric kernel function for the first time and analyzed the interior
point methods based on this kernel function. They showed that primal–dual interior point methods for solving LO enjoys
3
O(n 4 log nϵ ) as the worst case iteration complexity.
In this paper, we propose a new trigonometric kernel function for primal–dual interior-point methods which improves
the complexity bound obtained by El Ghami et al. in [12] for LO. More precisely, under some mild conditions and simple
analysis, we show that the worst case iteration complexity for primal–dual IPMS based on the proposed kernel function
2
enjoys O(n 3 log nϵ ) which improves the complexity obtained by El Ghami et al. in [12] significantly.
The paper is organized as follows: In Section 2 we introduce the new kernel function and derive its properties. Section 3
is devoted to express the proximity reduction during an inner iteration. For this purpose, we obtain a default value for the
step size in this section. The complexity analysis of primal–dual IPMs based on the new kernel function is given in Section 4.
We finally end up the paper by providing some concluding remarks.
We use the following notational conventions: Throughout the paper, ∥ · ∥ denotes the Euclidian norm of a vector. The
nonnegative and positive orthants are denoted by Rn+ and Rn++ , respectively. For a vector x = (x1 , . . . , xn ) ∈ Rn , x∗ is the
minimum component of x. For given vectors x and s, the vectors xs and xs denote the coordinate-wise operations on the
vectors, i.e., whose components are xi si and si , respectively. We say that f (t ) = Θ (g (t )), if there exist positive constants ω1
x
i
and ω2 so that ω1 g (t ) ≤ f (t ) ≤ ω2 g (t ), for all t ∈ R++ . We also say that f (t ) = O (g (t )), if there exists a positive constant
ω so that f (t ) ≤ ωg (t ), for all t ∈ R++ .

2. The new kernel function and its properties

This section is devoted to introduce a new kernel function together with its properties that are used in the complexity
analysis of the Algorithm 1 based on this function. The goal of this paper is to focus on the new function

t2 − 1 1
ψ(t ) = − log(t ) + tan2 (h(t )) (7)
2 8
where
π (1 − t )
h( t ) = . (8)
2 + 4t
We first show that this function is indeed a kernel function and then study the complexity of primal–dual interior point
methods based on this kernel function. To do so, we need the first three derivatives of the function (7), which are:
1 1
ψ ′ (t ) = t − + h′ (t ) tan(h(t )) 1 + tan2 (h(t ))
 
(9)
t 4
1 1
ψ ′′ (t ) = 1 + 1 + tan2 (h(t )) h′′ (t ) tan(h(t )) + h′ (t )2 1 + 3 tan2 (h(t ))
  
+ (10)
t2 4
2 1
ψ ′′′ (t ) = − 1 + tan2 (h(t )) k(t )

+ (11)
t3 4
where
−6π
h′ (t ) = ,
(2 + 4t )2
48π
h′′ (t ) = ,
(2 + 4t )3 (12)
−576π
h′′′ (t ) = ,
(2 + 4t )4
k(t ) = 3h′ (t )h′′ (t ) 1 + 3 tan2 (h(t )) + 4h′ (t )3 tan(h(t )) 2 + 3 tan2 (h(t )) + h′′′ (t ) tan(h(t )).
   

It can be easily seen that the function ψ(t ), defined by (7), is a decreasing function with respect to t in the interval (0, 1]
and an increasing function in the interval [1, +∞). Now, we provide some technical lemmas about this function.

Lemma 2.1 (Lemma 2.1 in [12]). For the function h(t ), defined by (8), we have:
1 1
tan(h(t )) − > 0, 0<t ≤ .
3π t 2

Lemma 2.2. For the function ψ(t ), defined by (7), we have:

(i) ψ ′′ (t ) > 1, ∀t > 0,
(ii) t ψ ′′ (t ) − ψ ′ (t ) > 0, ∀t > 0,
78 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85

(iii) t ψ ′′ (t ) + ψ ′ (t ) > 0, ∀t > 0

(iv) ψ ′′′ (t ) < 0, ∀t > 0.
π π 2
Proof. We first prove that (i) is satisfied. To do so, if t ∈ (0, 1), then we have h′′ (t ) = (248 and tan(h(t )) > 0.
2
+4t )3
, h′ = (236
+4t )4
Thus, ψ (t ) > 1, for all t ∈ (0, 1). In order to prove the inequality for t ≥ 1, it is sufficient to prove that
′′

1 1 
2

η(t ) = + (1 + tan2 (h(t ))) h′′ (t ) tan(h(t )) + h′ (t )(1 + 3 tan2 (h(t ))) > 0.
t2 4
Using the fact that, for t ≥ 1, it yields tan(h(t )) ∈ [−1, 0], we have
 
1 4 ′2
η(t ) = 1 + tan (h(t )) 2
+ h (t ) tan(h(t )) + h (t )(1 + 3 tan (h(t )))
′′ 2

4 t 2 (1 + tan2 (h(t )))
48π (2 + 4t ) 36π 2
 
1  2
≥ 1 + tan2 (h(t )) − +
4 t2 (2 + 4t )4 (2 + 4t )4
 2(2 + 4t )4 − 48π t 2 (2 + 4t ) + 36t 2 π 2
 
1
= 1 + tan2 (h(t )) > 0,
4 t 2 (2 + 4t )4
where the last inequality is obtained by using simple calculus.
In order to prove (ii), one can see that, for t ∈ (0, 1), we have ψ ′′ (t ) > 0 and ψ ′ (t ) < 0. Therefore t ψ ′′ (t ) − ψ ′ (t ) > 0 holds
for this case. Now, assume that t ≥ 1. Then, using the fact that th′′ (t ) − h′ (t ) > 0 and tan(h(t )) ∈ [−1, 0], we have:
2 1
t ψ ′′ (t ) − ψ ′ (t ) = 1 + tan2 (h(t )) (t h′′ (t ) − h′ (t )) tan(h(t )) + t h′ (t )2 1 + 3 tan2 (h(t ))
  
+
t 4
2 1
1 + tan2 (h(t )) −t h′′ (t ) + h′ (t ) + t h′ (t )2
 
≥ +
t 4
 
1 8
1 + tan (h(t )) −t h (t ) + h (t ) + t h (t ) +
2 ′′ ′ ′ 2

=
4 t (1 + tan2 (h(t )))
 −6π t (12t 2 + 8t + 1) + 9π 2 t 2 + (2 + 4t )4
 
≥ 1 + tan (h(t )) 2
>0

t (2 + 4t )4
where the last inequality is obtained by using simple calculus.
Now, we prove that (iii) holds. For this purpose, we have:
1
t ψ ′′ (t ) + ψ ′ (t ) = 2t + 1 + tan2 (h(t )) (t h (t ) + h′ (t )) tan(h(t )) + h′ (t )2 1 + 3 tan2 (h(t )) .
  ′′  
4
If t ≥ 1, then ψ ′ (t ) ≥ 0 and ψ ′′ (t ) > 0, which imply that t ψ ′′ (t ) + ψ(t ) > 0 holds for this case.
24π(4t 2 −1)
For 1
2
< t < 1, we have t h′′ (t ) + h′ (t ) = (2+4t )4
> 0. Therefore, t ψ ′′ (t ) + ψ ′ (t ) > 0 holds for this case too.
If 0 < t ≤ 1
2
, using Lemma 2.1, we have

24π (4t 2 − 1)
 
2 1
t h′′ (t ) + h′ (t ) tan h(t ) + th′ (t ) 1 + 3 tan2 (h(t )) > tan(h(t )) + th′ (t )2 tan(h(t ))
   
1+
(2 + 4t )4 πt
24π (4t 2 − 1) + 36π
 
= tan(h(t )) + th′ (t )2 > 0
(2 + 4t )4
which shows that t ψ ′′ (t ) + ψ ′ (t ) > 0 holds for t ∈ (0, 12 ].
To complete the proof of the lemma, we show that (iv) holds for all t > 0. To do so, we have: If 0 < t ≤ 1, then h′′′ (t ) < 0,
h′ (t )h′′ (t ) < 0, h′ (t ) < 0 and tan(h(t )) > 0, which imply that ψ ′′′ (t ) < 0, for all t ∈ (0, 1].
3
√
If 1 < t ≤ 4, then −3 3 ≤ tan(h(t )) < 0. Therefore, using (13), we have:
2 1
ψ ′′′ (t ) = − 1 + tan2 (h(t )) k(t )

+
t3 4
 
1 8
1 + tan2 (h(t )) −   + k(t )

=
4 t 1 + tan2 (h(t ))
3

 
1 6 12 3 1
1 + tan2 (h(t )) − 3 + 3h′ (t )h′′ (t ) − √ h′ (t ) − √ h′′′ (t )

≤
4 t 3 3
 
1 384 12 ′3 1 ′′′
1 + tan2 (h(t )) − ′
( ) ′′
( ) ( ) ( )

≤ + 3h t h t − √ h t − √ h t
4 (2 + 4t )3 3 3
 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85 79

where the last inequality is obtained from the fact that 64t 3 < (2 + 4t )3 , for all t > 0. Now, using the definition of h(t ) and
some calculation, we obtain:
−96(2 + 4t )3 − 216π 2 (2 + 4t ) + 648
√ π3 + √π
144
(2 + 4t )2 
3 3
ψ ′′′ (t ) ≤ 1 + tan2 (h(t )) < 0.

(2 + 4t ) 6

It remains to verify the inequality (iv) for the case when t > 4. In this case, we have: −1 ≤ tan(h(t )) < − √1 . Thus, using
3
this relation and the fact that 64t 3 < (2 + 4t )3 , for all t > 0, we obtain,
2 1
ψ ′′′ (t ) = − 1 + tan2 (h(t )) k(t )

+
t3 4
 
1 8
1 + tan2 (h(t )) −   + k(t )

=
4 t 3 1 + tan2 (h(t ))
 
6h′ (t )h′′ (t ) − 20h′ (t ) − h′′′ (t )
3
1 4
1 + tan2 (h(t )) − 3 +

≤
4 t (2 + 4t )6
−64(2 + 4t )3 − 432π 2 (2 + 4t ) + 1080π 3 + 144π (2 + 4t )2 
1 + tan2 (h(t )) < 0.

≤
(2 + 4t ) 3

This completes the proof of the lemma. 

According to ψ(1) = ψ (1) = 0, one can completely describe the function ψ(t ) by its second derivative as follows:
′

 t ξ
ψ(t ) = ψ ′′ (ζ )dζ dξ . (13)
1 1

The following lemma provides equivalent forms of the e-convexity property for a kernel function [11].

Lemma 2.3 (Lemma 2.1.2 in [6]). Let ψ(t ) be a twice differentiable kernel function for t > 0. Then, the following three properties
are equivalent:
√
(i) ψ( t1 t2 ) ≤ 21 (ψ(t1 ) + ψ(t2 )), for t1 , t2 > 0
(ii) ψ ′ (t ) + t ψ ′′ (t ) ≥ 0 for t > 0
(iii) ψ(eξ ) is a convex function.
Using Lemma 2.2, one can find out that the new kernel function defined by (7) has the e-convexity property. This property
plays an important role in the analysis of primal–dual algorithms based on kernel functions.
In the sequel, we provide some other results related to the new kernel function which are important in the analysis of
Algorithm 1. For this purpose, we first define the norm-based proximity measure δ(v) according to

n

1 1 
δ(v) := ∥∇ Ψ (v)∥ =  (ψ ′ (vi ))2 v ∈ Rn++ . (14)
2 2 i=1

Based on this measure, we have:

Lemma 2.4. For the new kernel function ψ(t ) defined by (7), we have:
(i) 21 (t − 1)2 ≤ ψ(t ) ≤ 21 ψ ′ (t )2 , for all t > 0.
(ii) Ψ (v) ≤√2δ(v)√ 2
. √
(iii) ∥v∥ ≤ n + 2Ψ (v) ≤ n + 2δ(v).
Proof. Using (13) and Lemma 2.2, the proof line is straight as Lemma 3.4 in [10]. 

Corollary 2.1. If Ψ (v) ≥ 1, then we have:

1
δ := δ(v) ≥ . (15)
2

Proof. It is easily followed from the second item of Lemma 2.4. 

3. Analysis of the algorithm

This section is devoted to study about the effect of updating the barrier parameter µ on the value of the proximity
measure. To be exact, after updating the barrier parameter µ by a factor 1 − θ , for θ ∈ (0, 1), the proximity measure would
80 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85

be increased. The following results provide an upper bound for the growth of the barrier function after µ-update. A default
value for the step size is also introduced in this section.

3.1. Growth behavior for the barrier function

For given τ , at the start of any outer iteration of Algorithm

√1 and just before updating of µ, we have Ψ (v) ≤ τ . Due to
the updating strategy of µ, the vector v is divided by a factor (1 − θ ), for 0 < θ < 1. This leads to an increase in the value
of Ψ (v), in general. The subsequent inner iterations are performed in order to bring the values of Ψ (v) back to the situation
where we have: Ψ (v) ≤ τ . Therefore, the largest values of Ψ (v) occur just after the updating of the parameter µ. Here, we
study the growth behavior of Ψ (v) after µ-update. To do so, we have the following results which are important in deriving
the iteration complexity of Algorithm 1.

Lemma 3.1. Let β ≥ 1. Then, we have

1
ψ(β t ) ≤ ψ(t ) + (β 2 − 1)t 2 .
2

Proof. Let us define ψ(t ) as

1
ψ(t ) = (t 2 − 1) + q(t )
2
where the function q(t ) is defined as follows:
1
q(t ) = − log(t ) + tan2 (h(t )).
8
Now, we have
1
ψ(β t ) − ψ(t ) = (β 2 − 1)t 2 + q(β t ) − q(t ).
2
As β ≥ 1, to prove the lemma, it is sufficient to show that the function q(t ) is a decreasing function. For this purpose, we
have
1 3π
q′ (t ) = − tan(h(t )) 1 + tan2 (h(t )) .
 
−
t 2(2 + 4t ) 2

For 0 < t ≤ 1, using the fact that tan(h(t )) ≥ 0, we have q′ (t ) < 0.

Now, let t > 1. In this case, since −1 ≤ tan(h(t )) < 0, then we have
1 3π 1 3π
q′ (t ) ≤ − 1 + tan2 (h(t )) ≤ − < 0.
 
+ +
t 2(2 + 4t ) 2 t (2 + 4t )2
This completes the proof of the lemma. 

Lemma 3.2. Let 0 < θ < 1 and v+ = √ v . Then

1−θ

θ 
Ψ (v+ ) ≤ Ψ (v) + (2Ψ (v) + 2 2nΨ (v) + n).
2(1 − θ )

Proof. Using Lemma 3.1 with β = √ 1 , one obtains

1−θ

n
1 θ ∥v∥2
Ψ (βv) ≤ Ψ (v) + (β 2 − 1)vi2 = Ψ (v) + .
2 i =1 2(1 − θ )

Now, the lemma is easily followed by applying the third item of Lemma 2.4 on this inequality. 

3.2. An approximate value for the step size

After an inner iteration, the new point is computed as

x+ = x + α ∆ x, y+ = y + α ∆y, s+ = s + α ∆s
where α is a step size which is obtained by using a line search strategy. From third equation of (5), we have:
x s
x+ = (v + α dx ), y+ = y + α ∆y, s+ = (v + α ds ).
v v
 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85 81

Thus,
x+ s+
v+
2
= = (v + α dx )(v + α ds ).
µ
Since the e-convexity property holds for the function Ψ (v), then we have
  1
Ψ (v+ ) = Ψ (v + α dx )(v + α ds ) ≤ (Ψ (v + α dx ) + Ψ (v + α ds )).
2

Let
f (α) = Ψ (v+ ) − Ψ (v),
1
f1 (α) = (Ψ (v + α dx ) + Ψ (v + α ds )) − Ψ (v). (16)
2
Obviously, we have f (0) = f1 (0) = 0 and f (α) ≤ f1 (α). On the other hand, f1 (α) is a convex function due to the convexity
of Ψ . Moreover, in the first two terms on the right hand side, the argument of Ψ is linear with respect to α . Now, taking the
first and second derivatives of the function f1 (α) with respect to α , we get
n
1  ′
f1′ (α) = ψ (vi + α dxi )dxi + ψ ′ (vi + α dsi )dsi

2 i =1
n
1   ′′
f1′′ (α) = ψ (vi + α dxi )d2xi + ψ ′′ (vi + α dsi )d2si .

2 i=1

It is easily seen that

1 1
f1′ (0) = ∇ Ψ (v)T (dx + ds ) = − ∇ Ψ (v)T ∇ Ψ (v) = −2δ(v)2 ,
2 2
where the last equality is obtained from (14). For simplicity, in the sequel, we use the following notations:
v∗ = min(v), δ := δ(v).
Now, we have:

Lemma 3.3 (Lemma 4.1 in [11]). Let f1 be defined as in (16). Then, the second derivative of the function f1 w.r.t. α satisfies the
following inequality:
f1′′ (α) ≤ 2δ 2 ψ ′′ (v∗ − 2αδ).

Lemma 3.4 (Lemma 4.2 in [11]). The inequality f1′ (α) ≤ 0 holds if α satisfies the following inequality:

− ψ ′ (v∗ − 2αδ) + ψ ′ (v∗ ) ≤ 2δ. (17)

Lemma 3.5 (Lemma 4.3 in [11]). The largest possible step size α holding (17) is given by

1
α= (ρ(δ) − ρ(2δ)) (18)
2δ
where ρ : [0, ∞) → (0, 1] is the inverse function of restricted of − 21 ψ ′ (t ) to the interval (0, 1].
The following technical lemma is crucial in obtaining an upper bound for the variation of the function Ψ in an inner
iteration.

Lemma 3.6 (Lemma 1.3.3 in [6]). Let h be a twice differentiable convex function with h(0) = 0, h′ (0) < 0 which attains its
minimum at t ∗ > 0. If h′′ is increasing for t ∈ [0, t ∗ ], then
1
h( t ) ≤ th′ (0), 0 ≤ t ≤ t ∗.
2

Due to Lemma 3.6, we have:

Lemma 3.7. Let α be defined as in (18). Then, for α satisfying α ≤ α , we obtain

f (α) ≤ −αδ 2 . (19)

82 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85

Proof. The proof is similar to the Lemma 3.5 in [12]. We just restate it here. Let h(α) be defined by
1 1
h(α) := −2αδ 2 + αδψ ′ (v∗ ) − ψ(v∗ ) + ψ(v∗ − 2αδ).
2 2
Then, we have

h(0) = f1 (0) = 0, h′ (0) = f1′ (0) = −2δ 2 , h′′ (α) = 2δ 2 ψ ′′ (v∗ − 2αδ).

Due to Lemma 3.3, one has f1′′ (α) ≤ h′′ (α), and therefore, f1′ (α) ≤ h′ (α) and f1 (α) ≤ h(α). On the other hand, we may write
 α
h′ (α) = −2δ 2 + 2δ 2 ψ ′′ (v∗ − 2ξ δ)dξ = −2δ 2 − δ(ψ ′ (v∗ − 2αδ) − ψ ′ (v∗ )).
0

Now, Lemma 3.4 and the fact α ≤ α imply that h′ (α) ≤ 0. Since ψ ′′ (t ) is a decreasing function with respect to t, thus h′′ (α)
is an increasing function with respect to α . Therefore, using Lemma 3.6, we obtain
1
f (α) ≤ f1 (α) ≤ h(α) ≤ α h′ (0) = −αδ 2 .
2
This completes the proof of the lemma. 

Lemma 3.8 (Lemma 4.4 in [11]). Let α be defined as in (18). Then, it yields the following inequality:

1
α≥ . (20)
ψ ′′ (ρ(2δ))

Due to Lemmas 3.5 and 3.8, in what follows, we set the following value for the step size in Algorithm 1 as the default
value:
1
α̃ = . (21)
ψ ′′ (ρ(2δ))
It is easily seen that α̃ ≤ α .

Lemma 3.9. Let Ψ (v) ≥ 1, and ρ and α̃ be defined as in Lemma 3.5 and (21), respectively. Then, we have
2
δ2 δ3
f (α̃) ≤ − ≤− . (22)
ψ (ρ(2δ))
′′ 5020

Proof. Lemma 3.7 and the fact α̃ ≤ α imply that f (α̃) ≤ −α̃δ 2 , which follows the first inequality. In order to obtain the
inverse function of the restriction of − 12 ψ ′ (t ) in the interval (0, 1], we need to solve the equation − 12 ψ ′ (t ) = s for t. To do
so, we have
 
1 1
− t − + h′ (t ) tan(h(t ))(1 + tan2 (h(t ))) = 2s.
t 4
This implies that,

4(2 + 4t )2
   
−4 1 1 48s
tan(h(t )) 1 + tan (h(t )) = ′ 2
 
2s + t − = 2s + t − ≤
h (t ) t 6π t π
where the last inequality is obtained from the fact that t ≤ 1. Now, putting t = ρ(2δ), we get 4δ = −ψ ′ (t ). Thus, we have

96δ 10 1
tan3 (h(t )) ≤ tan(h(t )) 1 + tan2 (h(t )) ≤ H⇒ tan(h(t )) ≤ δ3
 
(23)
π 3
where the first inequality is followed from the fact that tan(h(t )) ≥ 0. Now, we obtain a lower bound for α . For this purpose,
we have
1 1
α̃ = = .
ψ ′′ (t ) 1 1
1 + tan (h(t )) h′′ (t ) tan(h(t )) + h′ 2 (t )(1 + 3 tan2 (h(t )))
 2

1+ t2
+ 4

From Lemma 2.1, it yields

1 1
tan(h(t )) > ∀0 < t ≤ ,
3π t 2
 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85 83

which implies that

1
(1 + tan(h(t ))) > ∀0 < t ≤ 1. (24)
3π t
Therefore, letting ζ = 1 + tan2 (h(t )) and using (23) and (24), we have
1
α̃ ≥ 1 1
h′′ (t ) tan(h(t ))ζ + h′ 2 (t )ζ + 3h′ 2 (t ) tan2 (h(t ))ζ
 
1+ t2
+ 4
1
≥  2   
1 2
96δ
1 + 9π 2 1 + 10
3
δ 3 + 1
4
h′′ (t ) π + h′ 2 (t ) 1 + 100
9
δ3 + 288
π
tan(h(t ))δ
1
≥  1 2
   2 4
 .
1 + 9π 2 1 + 10
3
δ3 + 1
4
h′′ (t ) 96πδ + h′ 2 (t ) 1 + 100
9
δ3 + 960
π
δ3

Now, for all t ∈ (0, 1], it holds h′′ (t ) = (2t6+π1)3 ≤ 6π and h′ (t ) = 4(2t +1)4 ≤ 9π4 . Therefore, using (15), we get
9π 2
2 2

1
α̃ ≥ 4
5020δ 3
which implies that
2
δ2 δ2 δ3
f (α̃) ≤ − ′′ ≤− =− .
ψ (ρ(2δ)) 4
5020δ 3 5020
This completes the proof. 

A direct consequence of applying the second item of Lemma 2.4 to (22) is:
2 1
δ3 Ψ3  1

f (α̃) ≤ − ≤− 2
≤ Θ −Ψ 3 . (25)
5020 5020 × 2 3

4. Iteration complexity

In this section, we derive the worst case total iteration complexity for Algorithm 1 based on the proximity measure Ψ
induced from the kernel function ψ defined by (7). It has to be mentioned that α̃ , defined by (21), is utilized as the default
value for the step size during the complexity analysis in this section.
Using Lemma 3.2, we have the following inequality right after updating the parameter µ to (1 − θ )µ, for θ ∈ (0, 1),

θ 
Ψ (v+ ) ≤ Ψ (v) + (2Ψ (v) + 2 2nΨ (v) + n). (26)
2(1 − θ )
At the start of an outer iteration and just before updating of the parameter µ, we have Ψ (v) ≤ τ . Due to (26), the value of
Ψ (v) exceeds from the threshold τ after updating of µ. Therefore, we need to compute the inner iterations that are required
to return to the situation where Ψ (v) ≤ τ after µ update. Let us denote the value of Ψ (v) after µ-update as Ψ0 , and the
subsequent values as Ψj , for j = 1, . . . , L − 1, where L stands for the total number of inner iterations in an outer iteration.
Now, using (26), we have

θ √
Ψ0 ≤ τ + (2τ + 2 2nτ + n). (27)
2(1 − θ )
Due to (25), the decrease of Ψ in each inner iteration is given by
1
Ψj + 1 ≤ Ψj − κ Ψj 3 j = 0, 1, . . . , L − 1, (28)
where κ is some positive constant. Now, we are in situation to state the inner iteration complexity result in an outer iteration.
To do so, we first need the following technical lemma. One can find an elementary proof for this lemma in [6].

Lemma 4.1. Given α ∈ [0, 1] and t ≥ 1, one has

(1 + t )α ≤ 1 + α t .
84 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85

The following theorem provides the worst case upper bound for the total number of inner iterations in an outer iteration.

Theorem 4.1. Considering (28), we have

2
3Ψ03
L≤1+ . (29)
2κ

Proof. Using (28), for all j = 0, 1, . . . , L − 1, we have

2
 1
 32
0 ≤ Ψj+1 ≤3
Ψj − κ Ψj 3

2
  23
− 23
= Ψj 3 1 − κ Ψj
 
2 2 − 23
≤ Ψj 3 1− κ Ψj
3
2 2
= Ψj − κ 3
(30)
3
where the last inequality is obtained from Lemma 4.1. By subsequently using the inequality (30), we obtain
2 2 2
Ψj+3 1 ≤ Ψ03 − jκ.
3
Taking j = L − 1, we get
2 2
0 ≤ ΨL ≤ Ψ03 − (L − 1)κ
3
2
3Ψ03
⇒ L≤1+
2κ
which completes the proof of the theorem. 

In order to work in large neighborhood of the central path, assume that τ = O(n). Then, using (27), for the large update
interior point methods, i.e. θ = Θ (1), we obtain Ψ0 = O(n). Therefore, from Theorem 4.1, we obtain the following upper
bound for the total number of inner iterations:
 2

3Ψ03
    
 = O n 23 .
 
L≤ 1+
 (31)
2κ

On the other hand, for given θ = Θ (1) and  an accuracy parameter ϵ > 0, the total number of outer iterations for getting
nµ ≤ ϵ are bounded away by O θ1 log nϵ , see Lemma I.36 in [2]. Moreover, the total number of iterations for Algorithm 1
is obtained by multiplying the total number of inner and outer iterations. Therefore, by omitting the integer bracket in (31)
which does not change the order of complexity, we derive the following total number of iterations for Algorithm 1 to get an
ϵ solution, i.e. a solution that satisfies xT s = nµ ≤ ϵ :
 2 n
O n 3 log .
ϵ
This bound significantly improves the so far iteration bound of large update primal–dual interior point methods based on
the trigonometric kernel functions which was obtained in [12]. The iteration complexity for the small-update methods is
straight and we rest it for the interested readers.

5. Concluding remarks

In this paper, we propose a new kernel function with trigonometric barrier term for primal–dual interior point methods in
linear optimization. Using some mild and easy to check conditions, we provide a simple analysis for large update primal–dual
IPMs based on kernel functions, as described in Algorithm 1. In our analysis, the e-convexity property of the kernel function
2
plays an important role. We finally derive the worst case iteration complexity of Algorithm 1 which is O(n 3 log nϵ ). This
bound improves the complexity of the IPMs based on trigonometric as well as log barrier kernel functions significantly,
but still
√ needs to be improved so that it meets at least the so far best known complexity results for large-update IPMs,
i.e. O( n log n log nϵ ).
 M.R. Peyghami et al. / Journal of Computational and Applied Mathematics 255 (2014) 74–85 85

References

[1] N.K. Karmarkar, A new polynomial-time algorithm for linear programming, Combinatorica 4 (1984) 373–395.
[2] C. Roos, T. Terlaky, J.-P. Vial, Theory and Algorithms for Linear Optimization: An Interior Point Approach, Springer, New York, 2005.
[3] R.D.C. Monteiro, I. Adler, Interior-point path following primal–dual algorithms: part I: linear programming, Mathematical Programming 44 (1989)
27–41.
[4] G. Sonnevend, An analytic center for polyhedrons and new classes of global algorithms for linear (smooth, convex) programming, in: A. Prakopa,
J. Szelezsan, B. Strazicky (Eds.), Lecture Notes in Control and Information Sciences, vol. 84, Springer-Verlag, Berlin, 1986, pp. 866–876.
[5] N. Megiddo, Pathways to the optimal set in linear programming, in: N. Megiddo (Ed.), Progress in Mathematical Programming: Interior Point and
Related Methods, Springer-Verlag, New York, 1989, pp. 131–158.
[6] J. Peng, C. Roos, T. Terlaky, Self-Regularity: A New Paradigm for Primal–Dual Interior-Point Algorithms, Princeton University Press, 2002.
[7] Y.E. Nesterov, A.S. Nemirovskii, Interior Point Polynomial Algorithms in Convex Programming, in: SIAM Studies in Applied Mathematics, vol. 13, SIAM,
Philadelphia, PA, 1994.
[8] Y.Q. Bai, M. El Ghami, C. Roos, A new efficient large-update primal–dual interior-point methods based on a finite barrier, SIAM Journal on Optimization
13 (3) (2003) 766–782 (electronic).
[9] M. El Ghami, C. Roos, Generic primal–dual interior point methods based on a new kernel function, International Journal of RAIRO-Operations Research
42 (2) (2008) 199–213.
[10] M. Peyghami, K. Amini, A kernel function based interior-point methods for solving P∗ (κ)-linear complementarity problem, Acta Mathematica Sinica
26 (9) (2010) 1761–1778.
[11] Y.Q. Bai, M. El Ghami, C. Roos, A comparative study of kernel functions for primal–dual interior-point algorithms in linear optimization, SIAM Journal
on Optimization 15 (1) (2004) 101–128.
[12] M. El Ghami, Z.A. Guennoun, S. Boula, T. Steihaug, Interior-point methods for linear optimization based on a kernel function with a trigonometric
barrier term, Journal of Computational and Applied Mathematics 236 (2012) 3613–3623.