Scientific African 9 (2020) e00536

Contents lists available at ScienceDirect

Scientific African
journal homepage: www.elsevier.com/locate/sciaf

Combining modified ridge-type and principal component

regression estimators
Adewale F. Lukman a,b,∗, Kayode Ayinde c, Olajumoke Oludoun a,
Clement A. Onate a
a
Department of Physical Sciences, Landmark University, Omu-Aran, Nigeria
b
Centre Emile Borel, Institut Henri Poincare, Paris, France
c
Department of Statistics, Federal University of Technology, Akure, Nigeria

a r t i c l e i n f o a b s t r a c t

Article history: The performance of ordinary least squares estimator (OLSE) when there is multicollinearity
Received 28 January 2020 (MC) in a linear regression model becomes inefficient. The principal components regression
Revised 4 August 2020
and the modified ridge-type estimator have been proposed at a different time to handle
Accepted 28 August 2020
the problem of MC. However, in this paper, we developed a new estimator by combining
these two estimators and derived the necessary and sufficient condition for its superiority
Keywords: over other competing estimators. Furthermore, we establish the dominance of this new es-
OLSE timator over other estimators through a simulation study, and numerical example in terms
Principal component of the estimated mean squared error.
Modified ridge-type
Competing estimators © 2020 The Author(s). Published by Elsevier B.V. on behalf of African Institute of
Mathematical Sciences / Next Einstein Initiative.
This is an open access article under the CC BY license
(http://creativecommons.org/licenses/by/4.0/)

Introduction

The linear regression model is described as:

y = X β + ε, (1.1)
where y is an n × 1 vector of response variable, X is a known n × p full rank matrix of predictor or explanatory variables,
β is an p × 1 vector of unknown regression parameters, ɛ is an n × 1 vector of errors such that E (ε ) = 0 and V (ε ) = σ 2 In ,
In is an n × n identity matrix. The ordinary least squares estimator (OLS) estimator of β in (1.1) is defined as:

βˆ = (S )−1 X  y, (1.2)
where S = X  X is the design matrix.
OLS estimator ranked best for a long time until literature proved that this technique is not valid when multicollinear-
ity is present. Various biased estimators came up as an alternative to this method. The estimators include Stein estimator
[25], principal component regression (PCR) estimator [19], ridge regression (RR) estimator [8], contraction estimator [20] and

∗
Corresponding author.
E-mail addresses: adewale.folaranmi@lmu.edu.ng (A.F. Lukman), kayinde@futa.edu.ng (K. Ayinde), olajumoke@lmu.edu.ng (O. Oludoun),
onate.clement@lmu.edu.ng (C.A. Onate).

https://doi.org/10.1016/j.sciaf.2020.e00536
2468-2276/© 2020 The Author(s). Published by Elsevier B.V. on behalf of African Institute of Mathematical Sciences / Next Einstein Initiative. This is an
open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)
2 A.F. Lukman, K. Ayinde and O. Oludoun et al. / Scientific African 9 (2020) e00536

modified ridge regression (MRR) estimator [26], Liu estimator [14]. The performance of some of these estimators has been
combined to mitigate multicollinearity. These include two-parameter estimator [22], k − d class estimator [24], new two-
parameter estimator [27], new Liu-type estimator [13], modified two-parameter estimator [6] and very recently modified
ridge-type estimator and modified new two-parameter [17, 18] among others. The expectation is that these estimators com-
bine the advantages of estimators pooled together.
PCR estimator is employed to reduce the dimensionality of the estimation problem. Baye and Parker [3] developed the r -
k class estimator by combining the PCR and the ridge regression estimator. This new estimator was shown to perform better
rather than considering the estimators individually. This article had motivated other researches in that line. Kaciranlar and
Sakallioglu [11] combined the PCR and Liu estimator and named it the r–d class estimator. Batah et al. [2] introduced the
modified r–k class ridge regression (MCRR) estimator by pooling the unbiased ridge regression (URR) estimator by Crouse
et al. [4] with the PCR estimator. Also, Ozkale [23] grafted the PCR estimator into the restricted least squares estimator
to developed a new estimator, namely, the Restricted principal components regression (RPCR) estimator. Recently, Chang
and Yang [5] developed the principal component two-parameter by pooling together the PCR and new two-parameter es-
timator. The focus of this paper is to introduce a new estimator that can be used to effectively handle multicollinearity in
the linear regression model. Thus, compare the performance of this new estimator with the OLS, PCR, r–k and modified
ridge-type estimators in the sense of mean squared error criteria. The article’s structure is as follows. In Sect. 2, the new
estimator is introduced, and some properties of the new estimator discussed in Sect. 3. A Monte Carlo simulation study and
a numerical example conducted in Sects. 4 and 5, respectively. Finally, some conclusions provided in Sect. 6.

The proposed estimator

The modified ridge-type estimator proposed by Lukman et al. [17] is defined as:

βˆ MRT (k, d ) = Akd βˆ = (X  X + k(1 + d ))−1X  y, (2.1)

where Akd = (X  X + k(1 + d )I )−1 X  X, k>0 and 0<d<1. This estimator can produce the same results with the OLS and the
ridge estimator, especially when k = 0 and d = 0, respectively. Following Ozkale [23] and Chang and Yang [5], model (1.1)
is transformed as follows:

y = X V V  β + ε = Zα + ε, (2.2)

where Z = XV , α = V  β and V = (v1 , v2 , ..., v p ) = (Vr , Vp−r )is a p × p orthogonal matrix with
 
˜r 0
(Vr , Vp−r ) X  X (Vr , Vp−r ) = 
˜ = ,
0 
˜ p−r

where 0 < r ≤ p,  ˜ = diag(v1 , v2 , ..., v p ), 

˜ r = diag(v1 , v2 , ..., vr ), 
˜ p−r = diag(vr+1 , vr+2 , ..., v p )and v1 ≥ v2 ≥ ... ≥ vp > 0are
the ordered eigenvalues of X X. By definition, Z = XV = (Zr , Z p−r ) is the n × p matrix of the principal components
(PCs), where zi = X vi is the ith PC. Let Z p−r contains PCs corresponding to near zero eigenvalues, Z can be portioned as
Zr andZ p−r ,such that Z p−r will be eliminated. Model (2.2) can further be written as

y = Zr αr + Z p−r α p−r + ε . (2.3)

The PCR estimator of β is

 −1
βˆPCR = Vr V  r X  X Vr Vr X  y. (2.4)

Baye and Parker [3] defined the r - k class estimator of β as

 −1
βˆr = Vr V  r X  X Vr + kIr Vr X  y = VrVr βˆ (k ), (2.5)

where βˆ (k ) = (X  X + kI p )−1 X  y.
Following this approach, the r - k class estimator (2.1) in canonical form is

αˆ rMRT (k, d ) = Vr (Zr Zr + k(1 + d )Ir )−1Zr y. (2.6)

Its corresponding estimator of β is

βˆrMRT (k, d ) = Vr (Vr X  X Vr + k(1 + d )Ir )−1Vr X  y = VrVr βˆ MRT (k, d ). (2.7)

This estimator has the following properties

βˆrMRT (0, d ) = Vr (Vr X  X Vr )−1Vr X  y; PCR estimator;
βˆ MRT (0, d ) = (X  X )−1 X  y;OLS estimator;
p
βˆrMRT (k, 0 ) = Vr (Vr X  X Vr + kIr )−1Vr X  y; the r - k class estimator.
 A.F. Lukman, K. Ayinde and O. Oludoun et al. / Scientific African 9 (2020) e00536 3

Properties of the new estimator

In this section, the superiority of the new estimator is established over some existing estimators by the MSEM crite-
rion. Some definitions and lemmas provided for the proof. For a matrix B, B , B− B+ , R(B) and N(B) denote the transpose,
generalized inverse, Moore–Penrose inverse, column space and null space of A.

Lemma 1. Let Fn × p be the set of n × pcomplex matrices and Gn × n be the subset of Fn × p consisting of Hermitian matrices.
Further, given H ∈ Fn × p , H∗ , R(H)and κ (H) represent the conjugate transpose, the range and the set of all generalized inverses
of H, respectively. Now, let D ∈ Gn × n , a1 and a2 ∈ Fn × 1 be linearly independent, fi j = ai ∗ D− a j , i, j = 1, 2 for D− ∈ κ (H )and if
∗ ∗
a1 ∈R(D), let s = [a∗1 (I − DD− ) (I − DD− )a2 ]/[a∗1 (I − DD− ) ) (I − DD− )a1 ].Then, D + a1 a∗1 − a2 a∗2 ≥ 0 if and only if (iff) one of the
following sets of conditions holds:
a D ≥ 0, ai ∈ R(D ), i = 1, 2, ( f11 + 1 )( f22 − 1 ) ≤ | f12 |2 ;
b D ≥ 0, a1 ∈/ R ( D ), a2 ∈ R ( D : a1 ), ( a2 − sa1 )∗ ( a2 − sa1 ) ≤ 1 − |s|2 ;
c D = U U ∗ − λvv∗ , ai ∈ R(D ), i = 1, 2, v∗ a1 = 0, f11 + 1 ≤ 0, f22 − 1 ≤ 0, ( f11 + 1 )( f22 − 1 ) ≥ | f12 |2 ;
where (Uv) is a subunitary matrix (U possibly absent), is a non-negative definite diagonal matrix (occur-
ing when U is present) and λ is a positive scalar. Further, the conditions (a), (b) and (c) are all independent of the choice
of D− ∈ κ (H ).

Proof. See Baksalary and Trenkler [1].

↔ ↔ ↔
Let β i = Ai y, i = 1, 2 be two linear estimators of β . The estimator β 2 is said to be superior to β 1 if and only if (iff)
↔ ↔ ↔ ↔ ↔ ↔ ↔ ↔ ↔ ↔
Cov(β 1 , β 2 ) = MSEM (β 1 ) − MSEM (β 2 ) ≥ 0, where MSEM (β i ) = E (β i − β )(β i − β ) = Cov(β i ) + Bias(β i )Bias(β i ) , i = 1, 2 is
↔ ↔ ↔
the mean square error matrix, Cov(β i ), i = 1, 2 denotes the covariance matrix of β i and Bias(β i ) = (Ai X − I )β , i = 1, 2
denotes the bias vector. The properties of the proposed estimator are obtained as follows:
   
Bias βˆrMRT (k, d ) = Vr  ˜ rV  r − Ip
˜ r (kd )−1  β, (3.1)

where 
˜ r (kd )−1 = ( ˜ r = Vr X  XV and
˜ r + k(1 + d )Ir )−1 ,
 
Cov βˆrMRT (k, d ) = σ 2Vr 
˜ r (kd )−1  ˜ r (kd )−1Vr .
˜ r

Hence,
     
MSEM βˆrMRT (k, d ) = σ 2Vr 
˜ r (kd )−1  ˜ r (kd )−1Vr + Vr 
˜ r ˜ rV  r − Ip
˜ r (kd )−1  ββ  Vr  ˜ rV  r − Ip .
˜ r (kd )−1  (3.2)

The Scalar Mean Squared error is


r
g2i 
p
SMSE = (gi − 1 )2 αi2 + σ 2 + αi2 , (3.3)
i=1
λi i=r+1

λ
where gi = λ +k(i1+d ) and αi = V  β .
i

Comparison of the proposed estimator with the PCR estimator

Recall that when k = 0, the proposed estimator becomes PCR. Applying this to (3.2) implies that the MSEM of PCR is:
     
MSEM βˆPCR = σ 2Vr 
˜ −1   ββ  VrV  r − Ip . ,
r Vr + Vr V r − I p (3.4)

Therefore,
   
MSEM βˆPCR − MSEM βˆrMRT (k, d ) = D1 + a1 a1 − a2 a2 , (3.5)

−1 ˜ ˜ −1   −1 ˜  
whereD1 = σ 2Vr ( r − r (kd ) r r (kd ) )Vr ,a1 = (Vr V r − I p )β ,and a2 = (Vr r (kd ) r V r − I p )β . By definition, Vr Vr =
˜ −1 ˜ ˜
 
Ir ,Vp−r Vp−r = I p−r ,Vp−r Vr = 0.Due to some algebraic simplification, D1 becomes

D1 = σ 2Vr  ˜ r (kd )−1Vr ,

˜ r (kd )−1 B∗  (3.6)
where B∗= 2k ( 1 + d ) + ˜ −1
+ d )2  k2 ( 1
r ≥ 0 for all k>0 and 0<d<1.
The Moore-Penrose inverse D+1
of D1 is given by
˜ r (kd )Vr .
1 = σ Vr 
D+ −2 ˜ r (kd )B∗−1  (3.7)
We can easily show that D1 D+
1
= VrVr ,and D1 D+ a = a1 ,which implies that a1 ∈R(D1 ).Therefore, part (a) of Lemma does
1 1
not hold. We then examine part (b) of Lemma 1.
4 A.F. Lukman, K. Ayinde and O. Oludoun et al. / Scientific African 9 (2020) e00536

˜ r (kd )−1 [k(1 + d )]Vr β = D1 η1 ,where η1 = −σ −2Vr 

a2 − a1 = −Vr  ˜ r (kd )B∗−1 k(1 + d )Vr β ,such that
a2 ∈ R(D1 a1 ). We observe that part (b) of Lemma 1 is applicable where s = 1 in this case. From Lemma 1, the new
estimator is superior to the PCR estimator iff

(a2 − a1 ) D−1 (a2 − a1 ) = η1 D1 D−1 D1 η1 = η1 D1 η1 ≤ 0.

1
We observed that η1 D1 η1 = σ −2 β Vr k(1 + d )B∗−1 k(1 + d )Vr β = β  T1 T1 β where T1 = σ −1 B∗− /2 k(1 + d )Vr ,the condition
turns out to be T1 β = 0,means that β ∈ N(T1 ), where N(T1 ) denotes the null space of T1 . Thus, we have the following
result.

Theorem 1. The new estimator is superior to the PCR estimator by the criterion of MSEM iff β ∈ N(T1 ), where N(T1 ) denotes the
1
null space of T1 = σ −1 B∗− /2 k(1 + d )Vr .

Comparison of the proposed estimator with the r-k estimator

Recall that when d = 0, the proposed estimator becomes r-k estimator. Applying this to (3.2) implies that the MSEM of
r-k estimator is:
     
MSEM βˆr = σ 2Vr 
˜ r (k )−1  ˜ r (k )−1Vr + Vr 
˜ r ˜ rV  r − Ip
˜ r (k )−1  ββ  Vr  ˜ rV  r − Ip
˜ r (k )−1  (3.8)

where 
˜ r (k )−1 = (
˜ r + kIr )−1 .
Therefore,
   
MSEM βˆr − MSEM βˆrMRT (k, d ) = D2 + a1 a1 − a2 a2 , (3.9)

where D2 = σ 2Vr 
˜ r (k )−1 
˜ r
˜ r (kd )−1 (2kd
˜ r + k2 d ( 2 + d ) )  ˜ r (k )−1Vr ,
˜ r (kd )−1 
   
a1 = Vr  ˜ rV  r − Ip
˜ r (k )−1  ˜ rV  r − Ip β .
˜ r (kd )−1 
β and a2 = Vr 
The Moore-Penrose inverse D+
2
of D2 is given by
 −1
˜ r (k )Vr ,
2 = σ Vr 
D+ −2 ˜ r ( k ) r r (kd ) 2kd r + k d (2 + d )
˜ −1 ˜ ˜ 2

˜ r (kd ) (3.10)

We can easily show that = VrVr ,and

D2 D+
2
D2 D+
= a1 ,which implies that a1 ∈R(D1 ).Therefore, part (a) of Lemma does
a
2 1
not hold. We then examine part (b) of Lemma 1.
a2 − a1 = −kdVr ˜ r (kd )−1  ˜ r (k )−1Vr β = D2 η2 ,where η1 = −σ −2 kdVr 
˜ r ˜ r ( k ) ˜ r + k2 d (2 + d ) ]−1Vr β , such that
˜ r (kd )[2kd
a2 ∈ R(D1 a1 ). We observe that part (b) of Lemma 1 is applicable where s = 1 in this case. From Lemma 1, the new
estimator is superior to the PCR estimator iff

(a2 − a1 ) D−2 (a2 − a1 ) = η2 D2 D−2 D2 η2 = η2 D2 η2 ≤ 0.

We observed that η1 D1 η1 = σ −2 β Vr 
˜ r (kd )[2kd ˜ r (kd )Vr β = β  T  T2 β where
˜ r + k2 d (2 + d ) ]−1 
2
T2 =
σ −1  
Vr ,the condition turns out to be T2 β = 0,means that β ∈ N(T2 ), where N(T2 ) denotes
r (kd )[2kd r + k d (2 + d )]
˜ ˜ 2 −1 / 2

the null space of T2 . Thus, we have the following result.

Theorem 1 The new estimator is superior to the PCR estimator by the criterion of MSEM iff β ∈ N(T2 ), where N(T2 )
denotes the null space of T2 = σ −1  ˜ r (kd )[2kd˜ r + k2 d (2 + d )]−1/2Vr .

Comparison of the proposed estimator with the modified ridge-type estimator

The MSEM of modified ridge-type estimator is obtained by replacing r with p in Eq. (3.2)
     
MSEM βˆ MRT (k, d ) = σ 2V (
˜ kd )−1  ˜ kd )−1V  + V (
˜ ( ˜ V  − Ip
˜ kd )−1  ββ  V ( ˜ V  − Ip ,
˜ kd )−1  (3.11)

where (
˜ kd )−1 = (
˜ + k(1 + d )I p )−1 .
Therefore,
   
MSEM βˆ MRT (k, d ) − MSEM βˆrMRT (k, d ) = D3 + a1 a1 − a2 a2 , (3.12)

where D3 = σ 2Vp−r 
˜ p−r (kd )−1 
˜ p−r   ,
˜ p−r (kd )−1Vp−r
   
a1 = V (
˜ kd ) −1 ˜ V  − I p β and a2 = Vr 
 ˜ rV  r − Ip β .
˜ r (kd )−1 

The Moore-Penrose inverse D+

4
of D4 is given by

D3 = σ −2Vp−r 
˜ p−r (kd ) 
p−r  p−r (kd )Vp−r ,
˜ −1 ˜ (3.13)
 A.F. Lukman, K. Ayinde and O. Oludoun et al. / Scientific African 9 (2020) e00536 5

Table 4.1
Estimated MSE when n = 50, k = 0.5, d = 0.1, p = 3.

Est. σ =5 σ =10
ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99 ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99
OLS 3.04922 4.17071 7.59642 69.7664 12.17681 16.6614 30.3504 279.066
PCR 0.51785 0.49983 0.48147 0.46871 1.23212 1.15184 1.07261 1.00714
r-k 0.51603 0.49853 0.48064 0.46828 1.22361 1.14497 1.06724 1.00299
mrt 2.88073 3.83837 6.46289 21.8865 11.5002 15.3251 25.8006 87.4009
r-mrt 0.51585 0.49839 0.48056 0.46824 1.22276 1.14429 1.06671 1.00258
CN 7.171 11.589 25.314 278.270 7.171 11.589 25.314 278.270

Table 4.2
Estimated MSE when n = 50, k = 0.8, d = 0.3, p = 3.

Est. σ =5 σ =10
ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99 ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99
OLS 3.04922 4.17071 7.59642 69.7664 12.17681 16.6614 30.3504 279.066
PCR 0.51785 0.49983 0.48147 0.46871 1.23212 1.15184 1.07261 1.00714
r-k 0.51496 0.49776 0.48016 0.46803 1.21857 1.14091 1.06405 1.00053
mrt 2.74364 3.57872 5.67491 11.97887 10.94873 14.2792 22.6318 47.63431
r-mrt 0.51412 0.49716 0.47978 0.46784 1.21458 1.13769 1.06153 0.99858
CN 7.171 11.589 25.314 278.270 7.171 11.589 25.314 278.270

We can easily show that D4 D+ 4

= VrVr ,and D4 D+ a = a1 ,which implies that a1 ∈R(D1 ).Therefore, part (a) of Lemma does
4 1
not hold. We then examine part (b) of Lemma 1.
a2 − a1 = −Vp−r ˜ p−r (kd )−1   β = D η ,where η = −σ −2V  β such that
˜ p−r Vp−r 4 4 4 p−r 
˜ p−r (kd )Vp−r
a2 ∈ R(D1 a1 ). We observe that part (b) of Lemma 1 is applicable where s = 1 in this case. From Lemma 1, the new
estimator is superior to the PCR estimator iff

(a2 − a1 ) D−4 (a2 − a1 ) = η4 D4 D−4 D4 η4 = η4 D4 η4 ≤ 0.

We observed that η4 D4 η4 = σ −2 β Vp−r   β = β  T  T β where T = σ −1 
˜ p−r Vp−r
3 3 3
/2 
˜ 1p−r Vp−r ,the condition turns out to be
η D  
η = β T T3 β = 0.Then, the new estimator is superior to modified ridge type estimator (MRTE) iff T3 β = 0which means
4 4 4
that β ∈ N(T3 ), where N(T3 ) denotes the null space of T3 . Thus, we have the following result.

Theorem 1. The new estimator is superior to the MRTE by the criterion of MSEM iff β ∈ N(T3 ), where N(T3 ) denotes the null
/2 
space of T3 = σ −1 
˜ 1p−r Vp−r .

Monte Carlo simulation

We conducted the simulation study of this paper using the R 3.4.1 programming language. The experiment is designed
to depend on certain factors that are expected to affect the properties of the estimators. Following Gibbon [7]; Kibria [12];
Lukman et al. [16] and Lukman et al. [16], we generated the explanatory variables using the following equation:
  12
xij = 1 − ρ 2 zij + ρ zip+1 , i = 1, 2, . . . , n, j = 1, 2, 3, . . . ., p (4.1)

where zij are independent standard normal pseudo-random numbers and ρ2

denotes the correlation between any two ex-
planatory variables. Four different sets of correlation are considered corresponding to ρ =0.7, 0.8, 0.9 and 0.99. The number
of explanatory variables, p, is taken to be 3 and 7, respectively. These variables are expressed in a standardized form. The
following equation will be used to generate the dependent variable y:
yi = β0 + β1 xi1 + β2 xi2 + β3 xi3 + . . . + β p xip + ei , i = 1, 2, . . . , n, (4.2)
where ei are i.i.d N(0,σ 2 )
and without loss of any generality will assume zero intercept for the model in (5.2). The values
of β are chosen to satisfy the constraint β  β =1 [21]. The simulation studies was repeated 10 0 0 times for the sample sizes
n = 50, 100, and σ 2 = 25 and 100. Following Kaciranlar and Sakallioglu [11]; Chang and Yang [5]; r is chosen to be the
eigenvalues higher than one. For each replicate, the estimated MSE for each of the estimators α ∗ is obtained as follows:

1  ∗
10 0 0
MSE (α ∗ ) = (α − α ) (α ∗ − α ), (4.3)
10 0 0
i=1

where α ∗ would be any of the estimators (OLS, PCR, r-k, MRT, r-MRT), the estimator with the smallest estimated MSE is
considered best. The simulation results is summarized in Table 4.1–4.6 and the degree of multicollinaerity is determined
using the condition number, CN=max(eigenvalue)/min(eigenvalue). We have the following results from Table 4.1–4.6.
6 A.F. Lukman, K. Ayinde and O. Oludoun et al. / Scientific African 9 (2020) e00536

Table 4.3
Estimated MSE when n = 100, k = 0.5, d = 0.1, p = 3.

Est. σ =5 σ =10
ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99 ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99
OLS 2.06314 3.24404 6.95757 76.7722 8.16319 12.9199 27.8089 307.0864
PCR 0.41455 0.38353 0.35973 0.34363 0.73028 0.66855 0.62053 0.58662
r-k 0.41416 0.38316 0.35935 0.34323 0.72845 0.66701 0.61916 0.58535
mrt 1.96808 2.99181 5.80260 19.8026 7.77517 11.9001 23.1701 79.0062
r-mrt 0.41412 0.38312 0.35931 0.34319 0.72827 0.66685 0.61902 0.58523
CN 13.038 23.842 58.155 712.311 13.038 23.842 58.155 712.311

Table 4.4
Estimated MSE when n = 100, k = 0.8, d = 0.3, p = 3.

Est. σ =5 σ =10
ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99 ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99
OLS 2.06314 3.24404 6.95757 76.7722 8.16319 12.9199 27.8089 307.0864
PCR 0.41455 0.38353 0.35973 0.34363 0.73028 0.66855 0.62053 0.58662
r-k 0.41392 0.38294 0.35912 0.34298 0.72736 0.66609 0.61834 0.58459
mrt 1.89024 2.79566 5.02883 10.73709 7.45641 11.1043 20.0518 42.5907
r-mrt 0.41374 0.38277 0.35894 0.34279 0.72650 0.66535 0.61769 0.58399
CN 13.038 23.842 58.155 712.311 13.038 23.842 58.155 712.311

Table 4.5
Estimated MSE when n = 50, k = 0.6, d = 0.8, p = 7.

Est. σ =5 σ =10
ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99 ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99
OLS 12.1950 19.2632 41.0398 441.8147 48.7574 77.0377 164.1557 1767.27
PCR 0.65508 0.61856 0.58976 0.57183 0.9900 0.90311 0.83414 0.78739
r-k 0.65442 0.61808 0.58939 0.57151 0.98753 0.90129 0.83278 0.78631
mrt 9.45002 12.8009 19.1767 29.0493 37.7689 51.1750 76.6727 115.8248
r-mrt 0.65391 0.61770 0.58910 0.57126 0.98554 0.89985 0.83171 0.78545
CN 50.5874 103.078 275.502 3550.99 50.5874 103.078 275.502 3550.99

Table 4.6
Estimated MSE when n = 100, k = 0.6, d = 0.8, p = 7.

σ =5 σ =10
ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99 ρ =0.7 ρ =0.8 ρ =0.9 ρ =0.99
OLS 5.56914 8.61958 17.8810 184.995 22.2490 34.4610 71.5193 739.993
PCR 0.56162 0.54814 0.54058 0.53668 0.74484 0.70228 0.67120 0.64972
r-k 0.56138 0.54796 0.54045 0.53659 0.74411 0.70174 0.67080 0.64944
mrt 4.86413 6.89342 11.4369 29.8394 19.4181 27.5427 45.7214 119.116
r-mrt 0.56120 0.54782 0.54034 0.53652 0.74352 0.70131 0.67049 0.64922
CN 48.0650 94.777 245.085 3028.18 48.0650 94.777 245.085 3028.18

1 The proposed estimator r-MRT estimator dominates every other estimator in this study, and this agrees with the theo-
retical findings.
2 The estimated MSE of the estimators’ increases as the correlation level of the explanatory variables increases for fixed n,
k and d.
3 The estimated MSEs increase as the number of explanatory variables increases while other factors remain constant.
4 The estimated MSEs increases as the error variance increases while other factors remain constant.
5 The estimated MSEs decrease as the sample size increases while other factors remain constant.

Numerical example

We consider the data that was initially generated by Hoerl and Kennard [10] and later adopted by Crouse et al. [4]; and
Batah et al. [2]. The values of β satisfy the constraint β  β =300, and the error follows a normal distribution. The eigenvalues
of X X matrix are 0.0138, 0.0583, 0.1549, 0.1941, and 4.5759 and the computed condition index is 331.6. Both test results
revealed the presence of multicollinearity. The OLS estimator of σ 2 is 1.4281 and we chose r = 1 because only one of the
eigenvalues of the standardized variable is greater than unity. The ridge parameter k and Liu parameter d are estimated
 A.F. Lukman, K. Ayinde and O. Oludoun et al. / Scientific African 9 (2020) e00536 7

Table 5.1
Estimated regression coefficients and SMSE for the estimators.

coef. Population βˆOLS βˆPCR βˆr βˆ MRT βˆrMRT

β1 9.0269 7.9567 6.7600 6.7403 7.3530 6.7319
β2 8.3384 16.6563 6.7035 6.6839 11.8062 6.6756
β3 3.0903 2.6446 6.5023 6.4834 1.3809 6.4753
β4 3.3411 −5.9090 6.8756 6.8557 1.1548 6.8471
β5 11.3258 12.3692 6.4507 6.4319 11.6437 6.4239
MSE 144.6341 144.644 100.3084 101.7127 94.7917

using the harmonic mean version suggested by Hoerl et al. [9] and the method suggested by Ozkale and Kaciranlar [22],
respectively. They are defined as follows:

pσˆ 2
kˆ = p . (5.1)
 2
αˆ i
i=1

αˆ i2
dˆ = min σˆ 2 + α
. (5.2)
λi ˆ i2

Therefore, k and d are estimated to be 0.0133 and 0.4313. The results are shown in the Table 5.1.
From Table 5.1, we observed that the mean square error of the proposed estimator is smaller than that of other esti-
mators. This result agrees with the theoretical findings in Section 3. We also observed that OLSE was the only estimator
that produces a wrong sign in the estimation of the parameter,β 4 . According to Lukman and Ayinde [15], one of the conse-
quences of multicollinearity on the OLSE is that the regression coefficients might exhibit a wrong sign.

Conclusion

In this article, we developed a new estimator that combines the principal component regression and the modified ridge-
type estimator to jointly mitigate the effect of multicollinearity in a linear regression model. We state the condition under
which this new estimator will produce the same results with OLS, PCR and r–k class estimator. Also, derived the necessary
and sufficient conditions for the superiority of the new estimator over the PCR, r–k class and the modified ridge-type esti-
mator. Finally, a numerical example and a simulation study show that the proposed estimator generally outperforms other
competing estimators in this study when the explanatory variables are correlated.

Declaration of Competing Interest

We declare that this article does not conflict with any other journal. No financial assistance was received for this study.

References

[1] J.K. Baksalary, G. Trenkler, Nonnegative and positive definiteness of matrices modified by two matrices of rank one, Linear Algebra Appl. 151 (1991)
169–184.
[2] F.M. Batah, M.R. Ozkale, S.D. Gore, Combining unbiased ridge and principal component regression estimators, Commun. Stat. Theory Methods 38
(2009) 2201–2209.
[3] M.R. Baye, D.F. Parker, Combining ridge and principal component regression: a money demand illustration, Commun. Stat. Theory Methods 13 (1984)
197–205.
[4] R. Crouse, C. Jin, R. Hanumara, Unbiased ridge estimation with prior information and ridge trace, Commun. Stat. Theory Methods 24 (1995) 2341–2354.
[5] X. Chang, H. Yang, Combining two-parameter and principal component regression estimators, Stat. Pap. 53 (2012) 549–562.
[6] A.V. Dorugade, Modified two parameter estimator in linear regression, J. Stat. Trans. New Ser. 15 (1) (2014) 23-36.
[7] D.G. Gibbons, A simulation study of some ridge estimators, J. Am. Stat. Assoc. 76 (1981) 131–139.
[8] A.E. Hoerl, R.W. Kennard, Ridge regression: biased estimation for non-orthogonal problems, Technometrics 12 (1970) 55–67.
[9] A.E. Hoerl, R.W. Kennard, K.F. Baldwin, Ridge regression: some simulations, Commun. Stat. 4 (2) (1975) 105–123.
[10] A.E. Hoerl, R.W. Kennard, Ridge regression - I980 advances, algorithms, and applications, Am. J. Math. Manag. Sci. 1 (1981) 5–83.
[11] S. Kaciranlar, S. Sakallioglu, Combining the Liu estimator and the principal component regression estimator, Commun. Stat. Theory Methods 30 (2001)
2699–2705.
[12] B.M.G. Kibria, Performance of some new ridge regression estimators, Commun. Stat. - Simulat. Comput. 32 (2003) 419–435.
[13] Y. Li, H. Yang, A new Liu-type estimator in linear regression model, Stat. Pap. 53 (2) (2012) 427-437.
[14] K. Liu, A new class of biased estimate in linear regression, Commun. Stat. - Theory Methods 22 (2) (1993) 393–402.
[15] A.F. Lukman, K. Ayinde, Review and Classifications of the ridge parameter estimation techniques, Hacettepe J. Math. Stat. 46 (5) (2017) 953–967.
[16] A.F. Lukman, K. Ayinde, S.A. Ajiboye, Monte-Carlo study of some classification-based ridge parameter estimators, J. Modern Appl. Stat. Methods 16 (1)
(2017) 428–451.
[17] A.F. Lukman, K. Ayinde, S. Binuomote, A.C. Onate, Modified ridge-type estimator to combat multicollinearity: application to chemical data, J. Chemom.
(2019) e3125.
[18] A.F. Lukman, K. Ayinde, S.K. Sek, E. Adewuyi, A modified new two-parameter estimator in a linear regression model, Modell. Simul. Eng. (2019)
https://doi.org/10.1155/2019/6342702.
[19] W.F. Massey, Principal Components Regression in exploratory statistical research, J. Am. Stat. Assoc. 60 (1965) 234–256.
8 A.F. Lukman, K. Ayinde and O. Oludoun et al. / Scientific African 9 (2020) e00536

[20] L. Mayer, T. Willke, On biased estimation in linear models, Technometrics 15 (1973) 497–508.
[21] J.P. Newhouse, S.D. Oman, An Evaluation of Ridge Estimators, A report prepared for United States Air Force project RAND, 1971.
[22] M.R. Ozkale, S. Kaciranlar, The restricted and unrestricted two-parameter estimators, Commun. Statist. Theor. Meth. 36 (2007) 2707–2725.
[23] M.R. Ozkale, Principal components regression estimator and a test for the restrictions, Statistics (Ber) 43 (2009) 541–551.
[24] S. Sakallıoglu, S. Kaçıranlar, A new biased estimator based on ridge estimation, Statist. Papers 49 (4) (2008) 669–689.
[25] C. Stein, Inadmissibility of the usual estimator for mean of multivariate normal distribution, Proceed. Third Berkeley Symp. Math. Stat. Probab. 1 (1956)
197–206.
[26] B.F. Swindel, Good ridge estimators based on prior information, Commun. Stat.-Theory Methods 5 (1976) 1065–1075.
[27] H. Yang, X. Chang, A new two-parameter estimator in linear regression, Cmmun. Stat. - Theory Methods 39 (6) (2010) 923–934.