Multiuser Detection and Statistical Mechanics: Dongning Guo

Chapter 13
MULTIUSER DETECTION AND

STATISTICAL MECHANICS
Dongning Guo
Dept. of Electrical Engineering
Princeton University
Princeton, NJ 08544, U.S.A.
Sergio Verdu
Dept. of Electrical Engineering
Princeton University
Princeton, NJ 08544, U.S.A.
Abstract A framework for analyzing multiuser detectors in the context of sta-

tistical mechanics is presented. A multiuser detector is shown to be
equivalent to a conditional mean estimator which finds the mean value
of the stochastic output of a so-called Bayes retrochannel. The Bayes
retrochannel is equivalent to a spin glass in the sense that the distri-
bution of its stochastic output conditioned on the received signal is
exactly the distribution of the spin glass at thermal equilibrium. In the
large-system limit, the performance of the multiuser detector finds its
counterpart as a certain macroscopic property of the spin glass, which
can be solved using powerful tools developed in statistical mechanics.
In particular, the large-system uncoded bit-error-rate of the matched
filter, the MMSE detector, the decorrelator and the optimal detectors is
solved, as well as the spectral efficiency of the Gaussian CDMA channel.
A universal interpretation of multiuser detection relates the multiuser
efficiency to the mean-square error of the conditional mean estimator
output in the many-user limit.
1. Introduction
Code-division multiple access (CDMA) is becoming the ubiquitous
air-interface in future generation communication systems. In a CDMA
V. K. Bhargava et al. (eds.), Communications, Information and Network Security
© Springer Science+Business Media New York 2003
230 COMMUNICATIONS, INFORMATION AND NETWORK SECURITY
system, all frequency and time resources are allocated to all users simul-
taneously. To distinguish between users, each user is assigned a user-
specific spreading code on which the user's information symbol is mod-
ulated before transmission. By selecting mutually orthogonal spreading
codes for all users, each user can be separated completely by matched
filtering to one's spreading code. It is not very realistic to maintain
orthogonality in a mobile environment and hence multiple access inter-
ference (MAl) arises. The problem of demodulating in the presence of
the MAl becomes vital for a CDMA system.
A variety of multiuser detectors [1] have been proposed to mitigate the
MAL The simplest one is the single-user matched filter, which totally
ignores the existence of the MAL Its performance is not very satisfac-
tory and is particularly limited by the near-far problem. In the other
extreme, the individually optimal (10) and the jointly optimal (JO) de-
tectors achieve the minimum probability of error but entail prohibitive
complexity which is exponential in the number of users. A wide spec-
trum of multiuser detectors offer performance in between the matched
filter and the optimal detectors with substantially reduced complexity.
The most popular ones include the MMSE detector and the decorrelat-
ing detector. The performance of multiuser detectors have been stud-
ied extensively in the literature. A collection of results is found in [1].
In general, the performance is dependent on the spreading factor, the
number of users, the transmit signal-to-noise ratios (SNR), and the in-
stantaneous spreading codes. The dependence on the large number of
parameters results in very complex expressions for all but the simplest
cases. Not only are these expressions hard to evaluate, but the com-
plication allows little useful insight into the problem. To eliminate the
dependency by averaging over all spreading codes (e.g. [2]) is plausible
but usually a prohibitive task.
Recently, it is found that performance analysis can be greatly sim-
plified for randomly spread systems the size of which tend to infin-
ity [1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]. Of special interest is the case
where the number of users and the spreading factor both tend to infin-
ity with their ratio fixed. This is referred to as the large-system limit
in the literature. As far as linear multiuser detector is concerned, an
immediate advantage of the large-system setting is that the multiple-
access interference, as a sum of contributions from all interfering users,
becomes Gaussian-like in distribution under mild conditions as the num-
ber of users goes to infinity [13]. This allows the signal-to-interference
ratio (SIR) and the uncoded bit-error-rate (BER) to be easily character-
ized for linear detectors such as the matched filter, the MMSE detector
and the decorrelator. The large-system treatment also finds its success
Multiuser Detection and Statistical Mechanics 231
in solving the capacity (or the spectral efficiency) of CDMA channels

when certain multiuser detectors are used. Somewhat surprisingly, in
all the above instances, the SIR, the BER and the spectral efficiency
are independent of the spreading code assignment with probability l.
The underlying theory is that the empirical eigenvalue distribution of a
large random code correlation matrix converges to a deterministic dis-
tribution with probability 1 [14, 15, 16]. One may compare this to the
concept of typical sequences in information theory [17]. We can say that
for a sufficiently large system, almost all spreading code assignments are
"typical" and lead to the (same) average performance.
Unlike in the above, a more recent but quite different view of large
CDMA systems is inspired by the successful analysis of certain error-
control codes using methods developed in statistical mechanics [18, 19,
20, 21, 22, 23, 24]. In [25] a CDMA system finds an equivalent spin
glass similar to the Hopfield model. A spin glass is a magnetic system
consisting of a large number of directional spins, in which the interaction
of the spins is determined by the so-called quenched random variables,
Le., whose realizations are determined by the realization of the spin
glass [26].1 In [27, 28], well-known multiuser detectors are expressed
as Marginal-Posterior-Mode detectors, which can be embedded in spin
glasses. The large-system performance of a multiuser detector is then
found as a thermodynamic limit of a corresponding spin glass, which
can be solved by powerful techniques sharpened in statistical mechanics.
In [29, 24], Tanaka analyzed the 10 and the JO detectors, the decor-
relator and the MMSE detector under the assumption that all users
are received at the same energy (perfect power control). In addition to
the rederivation of some previously known results, Tanaka found for the
first time, the large-system BER of the optimal detectors at finite SNRs,
assuming perfect power control.
This new statistical mechanics approach to large systems emerges to
be more fundamental. In fact, the convergence of the empirical eigen-
value distribution, which underlies many above-mentioned large-system
results, can be proved in statistical mechanics [30, Chapter 1]. Deeply
rooted in statistical physics, the new approach brings a fresh look into
the decades-old multiuser detection problem. In this chapter we present
a systematic treatment of multiuser detection in the context of statistical
mechanics. We introduce the concept of Bayes retrochannel, which takes
the multiaccess channel output as the input and generates a stochas-
tic estimate of the transmitted data. The characteristic of the Bayes
retrochannel is the posterior probability distribution under some postu-
lated prior and conditional probability distributions. A multiuser detec-
tor is equivalent to a conditional mean estimator which finds the mean
value of the stochastic output of the Bayes retrochannel. By carefully

choosing the postulated prior and conditional probability distributions
of the Bayes retrochannel, we can arrive at different multiuser detection
optimality criteria. Moreover, the Bayes retrochannel is found to be
equivalent to a spin glass with the spreading code assignment and the
noise as quenched randomness. The conditional output distribution of
the Bayes retrochannel is exactly the same as the distribution of the spin
glass system at thermal equilibrium. In the large-system limit, the per-
formance of the detector finds its counterpart as a certain macroscopic
property of the thermodynamic system, which can be solved using the
replica method developed in statistical mechanics.
In particular, we solve the large-system BER of the matched filter, the
decorrelating detector, the MMSE detector and the optimal detectors.
We also find the spectral efficiency of the Gaussian CDMA channel.
Unlike in [29], we do not assume equal-energy users. It is found that
the multiuser efficiency for all users is the same, which is the solution
to a fixed-point equation similar to the Tse-Hanly equation in [4]. A
universal interpretation of multiuser detection relates the multiuser ef-
ficiency to the mean-square error of the conditional mean estimators in
the many-user limit. We present a canonical interference canceller that
approximates a general multiuser detector.
This chapter is organized as follows. In Section 2 the CDMA channel
and multiuser detectors of interest are introduced and previous large-
system results reviewed. In Section 3 we relate multiuser detection to
spin glasses. Section 4 carries a detailed analysis of linear multiuser
detectors. Results for the optimal detectors are obtained in Section 5.
An interpretation of uncoded multiuser detection in general is presented
in Section 6. A statistical mechanics look at the information theoretic
spectral efficiency is presented in Section 7.
2. The CDMA Channel and Multiuser Detectors

2.1 The CDMA Channel
We consider a K-user symbol-synchronous CDMA system with a
spreading factor of N. Consider one symbol interval. Let user k's spread-
ing code be denoted as Sk = IN[Slk,S2k, ... ,SNk]T, where the Snk'S are
independently and randomly chosen ±I's . .Let d = [d 1 , .•. , dK]T be a
vector consisting of the K independent user's transmitted symbols, each
symbol being equally likely to be ±l. The prior probability distribution
is simply
(13.1)
Let PI, .. . , PK be the K users' respective energies. The received signal

in the nth chip interval is then expressed as
(13.2)
where {vn } are independent standard Gaussian random variables, and

0"5 the noise variance.
We can normalize the averaged transmitted energy by absorbing a
common factor into the noise variance. Without loss of generality, we
assume
(13.3)
The SIR 2 of user k under matched filtering in the absence of interfering

·users is Pk/0"5 and the average SIR of all users is 1/0"5. We assume
that the energies are known deterministic numbers, and as K -t 00,
the empirical distributions of {Pk} converge to a known distribution,
hereafter referred to as the energy distribution.
The characteristic of the Gaussian CDMA channel can be described
as
where the N x K matrix S = [SI, ... ,SK]. Let r = [rI, ... ,rN]T, A =
diag( y'F{ , ... , VPK) and v = [VI, ... , VN ]T, we have a more compact
form,
r = SAd + O"OV (13.5)
and
(13.6)
where II . II denotes the norm of a vector.
2.2 Multiuser Detectors: Known Results

Assume that all users' energies and the noise variance are fixed and
known. A multiuser detector observes a received signal vector r in
each symbol interval and tries to recover the transmitted symbols us-
ing knowledge of the instantaneous spreading codes S. In general, the
detector outputs a soft decision statistic for each user of interest, which
is a function of (r, S),
ih = A(r, S), k E {I, ... , K}. (13.7)
Whenever the soft output can be separated as a useful signal component
and an interference, their energy ratio gives the SIR. Usually, a hard
decision is made according to the sign of the soft output,
(13.8)
Assuming binary symmetric priors, the bit-error-rate for user k is
(13.9)
Another important performance index is the multiuser efficiency, which

is the ratio between the energy that a user would require to achieve the
same BER in absence of interfering users and the actual energy [1],
[ao. Q-l(P k)]2
"lk = Pk
(13.10)
Immediately, the BER can be expressed in the multiuser efficiency as
(13.11)
In this chapter, we study the matched filter, the decorrelator, the

MMSE detector and the optimal detectors. The BER and the SIR per-
formance of these CDMA detectors have received considerable attention
in the literature. In general, the performance is dependent on the sys-
tem size (K, N) as well as the instantaneous spreading codes S, and
is therefore very hard to quantify. It turns out that this dependency
vanishes in the so-called large-system limit, i.e., the user number K and
the spreading factor N both tend to infinity but with their ratio K / N
converging to a constant (3. In the following we briefly describe each of
these detectors and present previously known large-system results.
2.2.1 The Single-user Matched Filter. Assuming absence

of the MAl, optimal detection is achieved by matched filtering with the
desired user's spreading code. A soft decision is given for user k,
s~r (13.12)
yIP; dk + 2)S~Si)JP: di + aOWk (13.13)
i#
where Wk is a standard Gaussian random variable. The second term,

the MAl has a variance of f3 as K = f3N -+ 00. Hence the large-system
SIR is sim ply
(13.14)
It can be shown using the central limit theorem that the MAl converges
to a Gaussian random variable in the large-system limit. Thus the BER3
IS
(13.15)
The multiuser efficiency is the same for all users
'1l(mf) _ _1_
./ - {3 • (13.16)
1+ -(72o
The performance of the matched filter is solely determined by the mul-
tiuser efficiency, since the SIR can be obtained as
SIR (mf) _ Pk. (mf) (13.17)

k -
ao2 T/
and the BER by (13.15). Indeed, from every user's point of view, the
MAl has the same statistical property in the large-system limit. The
only difference in the users is their own energies. Due to normalization
with respect to one's own energy, the multiuser efficiency is the same for
every user.
2.2.2 The MMSE Detector. The MMSE detector is a linear

filter which minimizes the mean-square error between the transmitted
data and its outputs
(13.18)
The decision statistic for user k can be described as
J~mmse) = Hkkdk + L Hikdi + aWk (13.19)

i#
where Hik is the element of H = A -1 [SHS + a 2 A -2] -1 SHSA on the ith

row and the kth column, and Wk is a Gaussian random variable.
In the case of equal-energy users, the large-system SIR of the MMSE

detector was first obtained in [1] as
~(m=') = 1 - ~ ( J( + /i3) + U6 - J
1 2 (1 _ /i3) 2 + U6 ) 2
(13.20)
which is the solution to
f3TJ
"1+ 2=1. (13.21)
"I + 0'0
Equation (13.21) is generalized in [4] to an arbitrary energy distribution
using random matrix theory and the so-called Tse-Hanly equation is ob-
tained. In the large-system limit, with probability 1, the output decision
statistic given by (13.19) converges in distribution to a Gaussian random
variable [9, 13]. Hence the BER is determined by the SIR by a simple
expression similar to (13.15). The Tse-Hanly equation is distilled to the
following fixed-point equation for the multiuser efficiency in [10]
TJ+,6E{ PTJ 2}=1 (13.22)

P1J + 0"0
where the expectation is taken over the random variable P drawn ac-
cording to the energy distribution. Again, due to the fact that the
output MAl seen by each user has the same asymptotic distribution,
the multiuser efficiency is the same for each user.
2.2.3 The Decorrelator. The decorrelating detector, or, the

decorrelator, removes the MAl in the expense of enhanced thermal noise.
Its output is
(13.23)
where (.)+ denotes the Moore-Penrose psudo-inverse, which reduces to

the normal matrix inverse for non-singular matrices. In case of ,6 :::; 1,
the large-system multiuser efficiency is [1]
TJ(dec) =1- ,6. (13.24)
It is incorrectly claimed in [29] that the multiuser efficiency is 0 if ,6 > 1.

We give the correct answer in section 4.
2.2.4 The Optimal Detectors. The jointly optimal detector

maximizes the joint posterior probability,
d(jo) = arg max .po(dlr, S) (13.25)
dE{ -1,IF'
where
(dl S) ~ po(rld, S) (13.26)
Po r, - 2:dE{-I,I}K po(rld, S)Po(d)
The individually optimal detector maximizes the marginal posterior prob-
ability
(13.27)
where
po(dlr, S) (13.28)
dk E { -1,1}J(-1
where d k denotes the vector d with the kth element struck out. The
IO detector achieves the minimum possible BER among all multiuser
detectors.
The asymptotic (low-noise) multiuser efficiency of the optimal detec-
tors is shown to be 1 in [6]. For finite SNRs, the large-system perfor-
mance of the optimal detectors has been solved for the case where all
users' energies are the same [27, 29]. In parallel with the format of the
result in (13.22), we show that the multiuser efficiency of the individually
optimal detector is also the solution to a fixed-point equation
1
4)
ry- (13.29)
- 1+ 4- [1 -
----~--------------~--------~--~
(To V
k27r J e- z: tanh ( V14
(Til
z+
(To
dZ] .
The efficiency of the jointly optimal detector is also found in [27] but
omitted here. We solve the multiuser efficiency for an arbitrary energy
distribution for both the IO and the JO detectors in section 5.
2.3 Spectral Efficiency

Of fundamental importance about a CDMA channel is its spectral
efficiency, defined as the total number of bits per chip that can be trans-
mitted arbitrarily reliably. In [5] the large-system spectral efficiency of
the Gaussian CDMA channel for the detectors of interest in this paper
is obtained for equal-energy case.
Channel r Bayes Retrochannel

do d
po(rJd) p(dJr) = E~~t~~e)
p(d) p(rJd)
Conditional Mean Estimator

(d)
(d) = E {dJr}
Figure 13.1. The Bayes retrochannel and the conditional mean estimator.
If a linear detector discussed in section 2.2 is used, the spectral effi-

ciency for user k is
(13.30)
where SIR~I) is the user's output SIR. It can be easily justified by notic-
ing that these linear detectors output asymptotically Gaussian decision
statistics.
Without any constraint on the type of detector, the spectral efficiency
of the CDMA channel is [10]
f3 { log ( 1 +
C = -E
2
'f/
(mmse)
0"5
p) }+ 1
-log
2
1
'f/(mmse)
+ -21 ('f/(mmse) - 1)
(13.31)
where the expectation is taken over the received energy distribution due
to fading.
3. Conditional Mean Estimator and Statistical

Mechanics
3.1 Bayes Retrochannel and Conditional Mean
Estimator
Consider a general estimation problem as depicted in Fig. 13.1. The
J( -dimensionalinput vector do is drawn according to a prior distribution
Po(d). The channel outputs an N-dimensional vector r according to
the conditional probability po(rld). Let r be the input to the Bayes
retrochannel defined by the conditional distribution
p(dlr) = Z-l (r)p(rld)p(d) (13.32)
where p( d) is a postulated prior distribution and p(rld) a postulated

conditional distribution, and
Z(r) = LP(rld)p(d) (13.33)

d
is a normalizing factor such that p(dlr) becomes a valid probability

distribution. Note that the set of possible symbols of p{d) is not limited
to the support of po{d). It can in general be the Euclidean space, in
which case the sum in (13.33) shall be replaced by an integral. Clearly,
this p( dlr) is a posterior probability obtained through the Bayes formula.
The Bayes retrochannel can be regarded as a "stochastic estimator" since
the K -dimensional output of the channel, d, is a random estimate of the
original data do.
We also consider a so called conditional mean estimator, which gives
a deterministic output upon an input r as
~ b.
d = (d) = E{dlr} (13.34)
where the operator (-) gives the expectation taken over the distribution
p(dlr). Indeed, this estimator outputs the mean value of the random
estimate generated by the Bayes retrochannel. Clearly, this estimator
also depends on the same postulated prior and conditional distribution
as that of the Bayes retrochannel.
Interestingly, by tuning the postulated prior and conditional distribu-
tions, the conditional mean estimator arrives at many popular detectors.
For example, suppose K = N = 1 and the scalar prior is symmetric bi-
nary, Po(d = 1) = Po(d = -1) = !.
Let the postulated prior be the
same as the true prior, po(d) = p(d), V d, and that the postulated condi-
tional distribution be the true channel law, p(rld) = po(rld), Vr, d. The
conditional mean estimate is then
(d) = p(d = llr) - p(d = -llr) (13.35)
whose sign is the maximum a posteriori detector output.
3.2 CDMA Multiuser Detectors

We now study the conditional mean estimator in the Gaussian CDMA
channel setting. Recall that the spreading code matrix is denoted as
S. We denote the postulated prior distribution of the symbols by p(d),
which may be different from the true binary symmetric prior distribution
Po(d). Specific choices are made in the following subsection. Let the
conditional distribution be
(13.36)
which differs from the true channel law po(rld, S) by a positive control
parameter a, where in case a = 0'0, po(rld, S) = p(rld, S). The posterior
probability distribution is then
(13.37)
where
Z(r, S) = ~P(d) exp [- 2~211r - SAd 112 ] . (13.38)
The conditional mean estimator outputs the mean value of the pos-
terior distribution,
(13.39)
where the expectation is taken over p(dlr, S). We identify a few choices
for the prior distribution p( d) and the control parameter a for the condi-
tional distribution under which the conditional mean estimator becomes
equivalent to each of the detectors discussed in section 2.2.
3.2.1 The Linear Detectors. Assume standard Gaussian

priors,
(13.40)
Then the posterior probability distribution is
where
Here we use the superscript (1) for linear detectors.

Since the probability given by (13.41) is exponential in a quadratic

function in d, d is a Gaussian random vector conditioned on rand S,
i.e., dk's are jointly Gaussian given rand S. The conditional mean
E{dklr,S} is therefore consistent with E{dlr,S}, which maximizes the
posterior distribution p(l) (dlr, S) as a property of Gaussian distributions.
At the extremum, d is the solution to
8p(I)(dlr, S)
8d H = 0, (13.43)
which yields
(13.44)
where we use the subscript to denote an average over the posterior prob-
ability distribution with control parameter (j.
By choosing different values for (j, we arrive at different types of linear
detectors. If (j -+ 00, we get
(13.45)
Hence the conditional mean estimate is consistent in sign with the matched
filter output. If a = (jo, it is exactly the soft output of the MMSE re-
ceiver as given by (13.19). If a -+ 0, we get the soft output of the
decorrelator as given by (13.23). We have seen that the control param-
eter can be used to tune a parameterized conditional mean estimator to
the desired one in a set of detectors.
3.2.2 The Optimal Detectors. Let p( d) be the true binary

symmetric priors Po( d). The posterior probability distribution is
(13.46)
where
Z(O)(r, S) = L _exp [- 2~211r - SAd Il2 ] . (13.47)

dE{ -I,IF'
We use the superscript (0) for the optimal detectors.

Suppose that the control parameter takes the value of (jo, then the
conditional probability distribution is the true channel law. The condi-
tional mean estimate is
(13.48)
Clearly, this soft output is consistent in sign with the hard decision of
the 10 detector as given by (13.27).
Alternatively, if a --+ 0, most of the probability mass of the distribu-
tion p(o)(dlr, S) is concentrated on a vector d that achieves the minimum
of IIr-SAdll, which also maximizes the posterior distribution po(dlr, S).
The conditional mean estimator output
(13.49)
will single out the kth component of this d at the minimum. Therefore
by letting a --+ 0 the conditional mean estimator is equivalent to the JO
detector as given by (13.25).
Worth mentioning here is that if a --+ 00, the conditional mean es-
timator reduces to the matched filter. This can be easily verified by
noticing that
p(O)(dlr, S) =1- -1-llr - SAdW + 0

2a 2
(~)
a2
. (13.50)
We have expressed each of the multiuser detectors of interest as a

conditional mean estimator, which has the form of the expected value
with respect to the posterior probability distribution. In the rest of
this section we construct thermodynamic systems which take the same
distributions in equilibrium and relate the conditional mean estimator
output to macroscopic quantities of the thermodynamic systems.
3.3 Preliminaries of Statistical Mechanics

A principal goal of statistical mechanics is to study the macroscopic
properties of physical systems containing a large number of particles
starting from the knowledge of microscopic interactions between the
particles. Let the microscopic state of some macroscopic system be
described by the configuration of some K variables, {ddf=l. A con-
figuration of the system is d = (d 1 , ... , dK). The basic quantity charac-
terizing the microscopic states is called the energy, which is a function of
the state variables, denoted by H (d). The configuration of the system
evolves over time according to some physical laws. After long enough
time the system will be in thermal equilibrium. An observable quantity
of the system, which is the averaged value of the quantity over time, can
be obtained by averaging over the ensemble of the states. For instance,
the energy of the system is
(13.51)
where p(d) is the probability of the system being found in configuration

d. In other words, instead of solving the exact dynamic trajectories, the
system could be described statistically. Another fundamental quantity
is the entropy, defined as
- 2:p(d) logp(d). (13.52)

d
It is assumed that the system is not completely isolated from the

surrounding world. The interaction with the surroundings produces the
following effects in equilibrium. First, the averaged value of the energy is
conserved. Second, the entropy of the system is the maximum possible.
Consider now, what must be the probability distribution function, which
would maximize the entropy. One can use the method of the Langrange
multipliers,
f = - ~P(d) logp(d) - ( (~P(d)H(d) - £) -~ (~P(d) 1) -
(13.53)
where ( and 'Yare the Lagrange multipliers. Variation with respect to p

gives the Boltzmann distribution
p(d) = Z-l exp [-(H(d)] (13.54)
where
Z = 2: exp [-(H(d)] (13.55)

d
is a normalizing factor called the partition junction, and the parameter

( is called the inverse temperature, which is determined by the condition
Z-l 2: H (d)exp[-(H(d)] = E. (13.56)

d
A fundamental quantity in statistical mechanics is the free energy

defined as the energy minus the entropy times the temperature
1
F = E + <: 2:p(d) logp(d). (13.57)
d
which can also be expressed as
F = LP(d) [H(d) + ~logp(d)] (13.58)

d
LP(d) [H(d)+~lOg(Z-lexp[-(H(d)])] (13.59)

d
(13.60)
At thermal equilibrium, the probability distribution is negative ex-

ponential in the associated energy of a configuration, the entropy is
maximized, and the free energy is minimized.
3.4 Spin Glass and the Bayes Retrochannel

A spin glass is a magnetic system consisting of a large number of inter-
acting spins, which follows the law of statistical mechanics. In the mul-
tiuser detection context, we study an artificial spin glass system which
has its energy function defined as Hr,s (d), where the spreading codes
S and the received signal r are regarded as quenched random variables.
Let the energy E be such that the inverse temperature is ( = 1. Then, at
thermal equilibrium, the probability distribution that defines the Bayes
retrochannel is
p(dlr, S) = Z-l(r, S) exp[-Hr,s(d)] (13.61)
where Z(r, S) is such that the above is a valid distribution. We define

the free energy per user as
(13.62)
Compare (13.61) to the postulated posterior probability distributions

associated with the conditional mean estimators, p(l) (dlr, S) and p(o) (dlr, S),
given by (13.41) and (13.46) respectively. p(dlr, S) can be made to take
each of the two distributions if we choose the corresponding energy func-
tion as
(13.63)
and
(13.64)
The common exponential structure of the probabilities results in simple

expressions for the energies. In other words, by defining an appropriate
energy function, the distribution of the thermodynamic system at equi-
librium is exactly the same as the posterior distribution associated with
a multiuser system, assuming some prior and conditional distributions.
In this case, the probability that the transmitted symbols are d, given
the spreading codes S and the observation r, is the same as the prob-
ability that the thermodynamic system is at configuration d, given the
quenched random variables Sand r.
A first look at the above does not yield any new ideas in analyzing
the multiuser system. However, the thermodynamic model brings pow-
erful physical insights into the detection problem. In particular, the
performance of the multiuser detector has its counterpart as a certain
macroscopic property of the thermodynamic system, which falls at the
center of statistical mechanics.
3.5 Overlap
Here we aKe interested in one of the most important performance
measures, the bit-error-rate. Due to symmetry of the spreading codes,
as far as the BER is concerned, we can assume that all transmitted
symbols are +1, i.e., dk = 1 for all k. Note that the spreading codes are
randomly chosen and not dependent on the received energies.
For the time being consider equal-energy users, i.e., Pk = 1 for all
k. Conditioned on (r, S) and the transmitted symbols being all equal
to 1, the percentage of erroneously detected bits in the current symbol
interval is ~[1 - M(r, S)] where
1 K 1 K
M(r,S) = KLdk = KLsgn(dk») (13.65)
k=l k=l
is called the overlap of the detector output. We can regard the overlap
as a macroscopic quantity of the spin glass system. Essentially it is the
magnetism of the system.
A fundamental principle in statistical mechanics states that the over-
lap, as a macroscopic quantity, is self-averaging. That is, with probabil-
ity 1, M(r, S) converges to its average over the probability distribution
of (r, S), denoted as «M(r, S)), as the system size K tends to infinity.
Here, «.» is defined explicitly as
«f(r,S)) ~ f drEs{po(rld,S)f(r,S)}, Vf(r,S) (13.66)

where d is the all 1 vector and Es {.} denotes the average over the
subscript variable 5, which is the spreading codes in this case. Note
that the overlap
(13.67)
is independent of the spreading codes, and noise realization. Denoting

its limit as K -t 00 by M, the BER of every user converges in the
large-system limit to
1
P = -(1- M). (13.68)
2
The observation here is that the empirical average is equal to the en-
semble average with probability 1 in the large-system limit. The fluc-
tuation of macroscopic properties of physical systems vanishes in the
large-system (thermodynamic) limit, which in another way explains the
vanishing dependency of the performance on spreading codes and the
realization of the noise process.
To obtain the overlap for a general received energy distribution, we
first assume that the K users are divided into L equal-energy groups.
The number of users in the [th group, denoted by C I , is KI = OItI<,
and each user in this grou p takes the same energy Pl. Let the users be
numbered such that n = {2::~:i Ki + 1, ... , 2::~=1 Kd· The parameters
satisfy
011 + 012 + ... + OIL = 1 (13.69a)
0I 1 P 1 + 0I2P2 + ... + OILPL = 1 (13.69b)
so that average energy is 1. We call such a distribution a "simple" energy
distribution.
The Kz users in group CI share the same BER, which can be written
as
(13.70)
where
(13.71)
The reason is that the overlap in (0) is also self-averaging as K/ =

OIIK -t 00.
In case of an arbitrary energy distribution, we consider a sequence of

"simple" energy distributions that converge to it. We find the overlap for
the "simple" cases and then take the limit to obtain the overlap under
the target distribution.
4. Performance Analysis of Linear Detectors

We are now ready to present the analysis of the multiuser detectors
in statistical mechanics. We first find the BER for the linear detectors.
The evaluation of the free energy of the spin glass system is a key step
in the solution. In that evaluation we will have the opportunity to use
the replica method.
Let the conditional distribution be defined as (13.36) and assume the
standard Gaussian priors introduced in (13.40), so that the posterior
probability distribution
p( dlr, S) = p(l) (dlr, S) (13.72)
corresponds to the conditional mean estimator which can be tuned to
the matched filter, the MMSE detector, or the decorrelator by a control
parameter a.
4.1 Free Energy

The self-averaging principle states that in the large-system limit, the
free energy converges to its average value over the distribution of the
quenched random variables (r, S)
F=- lim
K-+oo \
//~IOgZ(r,s))).
\I< (13.73)
To circumvent the difficulty of evaluating the average of the logarithm

in (13.73), we can make use of the "replica method". This is a well-
known trick in statistical mechanics whose first step is to write (13.73)
in an alternative expression
F = - lim K1. lim 88 log ((ZU(r, S))) , (13.74)

I<.. -+00 u-+o U
which follows from
lim dd log E {ZU}

u-+o U
= E {log Z}, VZ> o. (13.75)
The second step is to evaluate the average in (13.74) only for integer
u. Finally, we take the formal limit as u -7 0 of the resulting expres-
sion. There are intensive ongoing efforts in the mathematics and physics
community to find a rigorous proof for the replica method.
Consider u independent replicas of the same spin glass with the same
quenched randomness (r, S). Statistically there is no difference among
these replicas. We can view this as u identical and independent Bayes
retrochannels in parallel. Let d a = [dab ... , daK]T denote the channel
outputs (or the spin glass configuration) of the ath replica. The posterior
probability distribution for the replicated system is
g{(211")-~
p({da}~=llr, S)
= Z-U(r, S) exp [-~lIdaI12] exp [- 2~211r - SAd aI12]}

(13.76)
(13.77)
where Z-U(r, S) is the inverse of
Z"(r,S) = f (gfi Ddo')

gg exp [- 2:' (rn - ~ t, ~ sn.do,) ']
(13.78)
Here we have used the notation
! Dzf(z) ~ Joo
-00 y
~e-z22 f(z), Vf
211"
(13.79)
for the integral with respect to a Gaussian measure. By definition, we

can evaluate the average in (13.74) as an integral,
((ZU(r, S))) = ! dr Es {po(rld, S)ZU(r, S)}. (13.80)
Note that in order to average with respect to r, we need to use the true
data prior which in this case puts unit mass on the vector d = [1, ... ,1 r
The above becomes

((ZU(r, S)))
= f (ll, (2~a5)-~ ll,

drn ) Es{ exp [ - 2~~ (rn - ~t,,jP,
sn.) '] f (gfiDd..)gg exp [ - 2~~ (rn - ~ t,,jP,
sn.d.k) '] } (13.81)
= J (IT IT Dd {Jdr
a=1 k=1
ak ) k ES { exp [- 2~2 (r - ~N
0 y
t
k=1
g
21T0'0
,jP,S') '] exp [ - 2~5 (r - ~ t, ,jP,Skd.k) '] } } N

(13.82)
where the expectation in (13.82) is now taken over Sk'S which are inde-
pendent random chips. For convenience, we introduce dOk = dk = 1 for
k = 1, ... , J(. For a = 0, 1, ... , u, define a set of random variables
(13.83)
(13.84)
We can rewrite (13.82) as
((ZU(r, S))) =J (IT ft Dd ak ) { k Jdr Ev {exp [- ~

g
a=1 k=1 21T0'0 20'0
(r - ,ffivo) ,] exp [ - 2~' (r - ,ffiv.) ,] } }

(13.85)
where the expectation is now over the distribution of the random vector
v = [Vo, VI, ... , VulT, which is dependent on dak'S. Note that the Sk'S are
i.i.d. random chips. For fixed dak'S, each Va is a sum of J( weighted inde-
pendent random chips, which converges to a Gaussian random variable
as I< --+ 00. Because of a generalization of the central limit theorem, the
variables {va}~=o converge to a set of jointly Gaussian random variables,
with zero mean and covariance matrix Q, where
(13.86)
Note that although not made explicit in the notation, Qab is a function
of {dak' dbdf=l' Trivially, Qoo = l.
Therefore the expectation over va's is dependent on the symbols {dad
only through Q in the large-system limit. We can separate the integral
in (13.85) into the symbol dependent part Go and the rest G l for further
calculations:
~ log ((ZU(r, S))) = ~ log J(rr'dQab) exp[I«Go(u) + ,a-1GI(u))]

a<b
(13.87)
where
exp[I<Go(u)] = J (
uK
l11]Dd ak
)U'(L
~ 8 ~Pl ~l dakdbk - I<Qab
)
(13.88)
where 8(·) is the Dirac function and
exp[G, (u)] = ~ f dr E. { exp [ - 2~6 (r - Vilvo(Q)), 1

g exp [ - 2~2 (r - Vilv.(Q)), ]}- (13.89)
In (13.87)-(13.89), rr~<b' is the product running over all pairs of (a, b)

that 0 ~ a ~ b ~ u but-b > O. L~<b' is similarly defined for future use.
In the following we evaluate G01u) and GJ(u). Note the exponential
factor I< in the integral in (13.87). In the thermodynamic limit (I< --+
00), the right hand side of (13.87) converges to
(13.90)
where the minimum is over the set of possible Q's that can be obtained
by varying dak'S in (13.86). To find the extremum with respect to Qab'S
directly is a prohibitive task. Instead, we assume replica symmetry, i.e.,

for a, b = 1, ... , u, a =f. b,
E {VOva} = m (13.91a)
E {VaVb} = q (13.91b)
E {vn = p. (13.91c)
The replica symmetry is a convenient choice over all possible ansatz. It is

valid in most cases of interest in the large-system limit due to symmetry
over replicas. The readers are referred to [26, 29] for more discussion in
the justification of this assumption. We find the extremum with respect
to m, q, p only.
We are now ready to evaluate G 1 (u). We can construct explicitly a set
of Gaussian random variables possessing the above covariance structure
by letting
Vo (13.92a)
(13.92b)
where W, Y and Ya's are independent standard Gaussian random vari-

ables. The averaging in (13.89) with respect to v can be now taken over
the distribution of these independent Gaussian random variables. Note
that equations (13.92) can be obtained by diagonalizing the covariance
matrix in (13.91). For ease of notation, let
v j3m 2 q-l j(2a5) (13.93a)

w j3qj{2a 2) (13.93b)
x 13(1 - m2q-l)j{2a5) (13.93c)
y j3(p - q)j(2a 2). (13.93d)
Then
[kVo
2a02
ylvW +vlxY (13.94a)
J~2Va V'WW+y'YYa , a=1, ... ,u. (13.94b)

Hence
exp[G 1 (u)]
= vk Jdr
211"a5
Ev {ex p [- ~
2ao
(r - ...jfivO)2]
g exp [ - 2!' (r - .n v.) ,] } (13.95)
= .);r f dr' Ew { Ey { exp [ - (r' - Vow - v'XY )']}

gEy. { exp [ - er' -y'wW - jYY.)']} } (1396)
= .);r f dr' Ew { (1+ 2x)-' exp [ - (r' ~ ::V)']
(1 + 2y)-~ exp [_ u (~r' - y'wW)2] } (13.97)

1 + 2y
= (1 + 2x)-'(1+ 2y)-'1 Ew {f dr' ~ exp [ - a(r' - VoW)'
- b(r' - y'wW)'] } (13.98)
where a = 1/{1+2x) and b = ua5J[(1 + 2y)a 2]. The integral in the right
hand side of (13.98) is
= (1 + 2x)
_1
2 (1 + 2y)
_~
2
[1+
1 2x
ua5 2ua5
+ (1 + 2y)a2 + (1 + 2x)(1 + 2y)a2
(y'v - :0 vw) 2]-t (13.99)

= (1+ 2yr "~' [1+ 2y + u(1+ 2x) :~ + 2u (:. v'v - Vw) T' 1
(13.100)
= ( 1 + af32 (p - q) ) - u-1
2
[ f3
1 + a2 (p - q) + :2 (a5 + f3 (1 - 2m + q))
] -!.
2.
(13.101)
Therefore,
GI(U) u-1 log (1f+3

= --2- a 2 (p - q) )
-~log[1+
2
f3 (p-q)+
a2
U
a2
(a 5+ f3 (1-2m+ q))].
(13.102)
Next we evaluate G o(u). Using the Fourier transform representation

of Dirac function
1 jjoo+t dQexp (
8(x)=-. )
Q.x, 'lit E IR, (13.103)
211") -joo+t
we have
(13.104)
(13.105)
(13.106)
where the integral over each Qab is from -joo +tab to joo + tab for some
real number tab, and
(13.108)
The integrand in (13.107) is an analytical function on the multi-dimensional

complex space. Due to the factor K in the exponent in (13.107), the
integral is determined by its value evaluated at its maximum. This can
be justified as follows. The saddle point of the integrand can be found
by setting its derivative with respect to each Qab to O. The exponential
function can be expanded by Taylor series at the vicinity of the sad-
dle point. Higher order terms can be shown to diminish with a rate
faster than k. Therefore, the leading term dominates the integral in
the K --+ 00 limit.
It turns out that at the saddle point Qab'S are real. They satisfy
a set equations that induces highly symmetric structure, due to the
symmetry in Qab'S. Again, we assume replica symmetry for the Qab'S,
i.e., for a, b = I, ... , u, a =1= b,
E (13.109a)
F (13.109b)
G
(13.109c)
2
In this case,
XI(U)
= J(g Dd.) exp [AOoo + At,0o.d.+ A~'O'bd.dbl

(13.110)
= J l1 (
u )
Dda exp
[U U
PIE~da + PzF oEb 1 U]
dadb + 2PzG~d;
(13.111)
(13.112)
We use a standard trick to linearize the exponent,

(13.113)
where we use z to represent a standard Gaussian random variable through-
out this paper. We have
XI(U)
= E, {J (gDd.) exp [ (AE + vlAF z) t,d,

- ~PI(F - G) t,d:]} (13.114)
= E, { {f Dd exp [(AE+ vlMz) d- ~A(F - G)d'] r}

(13.115)
= Ez {
(1 + PI(F -
- U
G))-z exp
[u (PzE+ v!PzF zr] }
2(1 + Pz(F _ G)) (13.116)
= (1 + Pz(F - -
G))--2 (1 + (1 - u)PIF - PzG)-z
u-1 - - 1
exp [ UP?I E2 ]
(13.117)
2(1 + (1 - u)PIF - PIG) .
Meanwhile,
U I U U u
L QabQab =L QOaQoa + L QabQab +L QaaQaa (13.118)
a<b a=1 O<a<b a=1
u(u-1) u
= uEm+ 2 Fq+ "2 Gp . (13.119)
Therefore,
GO(u)
L U i
= LfrZlogXZ(U) - L QabQab (13.120)
t,
Z=1 a~b
= "I [ - U; l]og(1 + P,(F - G)) - ~]Og(1 + (1- u)PIF - P,G)

-2 E 2
uPz 1 E u (U - 1) F uG
+ 2(1 + (1 - u)PzF - PIG) - u m- 2 q -"2 p.
(13.121)
By (13.74), the free energy is therefore
(13.122)
(13.123)
The parameters (m, q, p, E, F, G) are such that Go (u) and G 1 ( u) achieve

their respective extremum so that the saddle-point conditions are satis-
fied. It is not difficult to see that the parameters are the solution to the
following set of joint equations as u -+ 0
1
E (13.124a)
a2 + (3(p - q)
F
a5 + (3(1 - 2m + q) (13.124b)
[a 2 + (3(p - q)]2
G
(3(1 - 2m + 2q - p) + (a5 - ( 2 ) (13.124c)
[a 2 + (3(p - q))2
L P2 E
m
tt
~a
I
I
1 + PI (F - G)
(13.124d)
L P3E2 + PI2F
tt
q ~a I (13.124e)
l[l+PI(F-G)F
p (13.124f)
Immediately,
G F-E (13.125a)
p 1- m+q. (13.125b)
Equations (13.124) can then be simplified as
1
E (13.126a)
a2 + (3(1 - m)
L P2E
m
tt
~ al---,-I-=-=
l+PIE
(13.126b)
F
a5 + (3(1 - 2m + q)
(13.126c)
[a 2 + (3(1 - m)]2
(13.126d)
The free energy can also be simplified accordingly. Clearly, (13.126a)-

(13.126b) can be used to solve E and m independent of the other vari-
ables. We can then solve F and q from (13.126c)-(13.126d).
4.2 Solving the Overlap

Without loss of generality, we study the overlap of group 1 users.
Consider first the quantity
(13.127)
for an arbitrary integer i. Let 1 ~ al < '" < ai ~ u be a sequence of

indexes for replicas. Since all replicas are equivalent, (dak) = (d k), and
we have
(13.128)
We solve the average in (13.128) by defining a modified version of the

posterior distribution for the replicated system,
(13.129)
where Z-U(r,S;h) is the inverse of
(13.130)
Consider
(13.131)
(13.132)
(13.133)
where the last equality is by definition of (-). Since
lim ZU(r, S)
u-+o
= 1, (13.134)
we have
Similar to (13.85),
We can write
1
1(1 log ((ZU(r, Sj h)))
= ~, log J(0.' dQ'b) exp[K(Go(u; h) + f3-'G, (u))] (13.137)

where
exp[KGo(u; h)]
~ J(g gDd.•) exp [h t, ll, d._.]

fIfo (tp[ L dakdbk - KQab) (13.138)
a5;b [=1 kEGI
and G 1 (u) is the same as defined by (13.89). For the purpose of solving
the overlap by (13.135), only Go(u; h) is relevant. We have
il' [Qab t PI L dakdbk - KQabQabj

a;5,b
exp
1=1 kEGI
(13.140)
J(il' ~~jb) [il' (-KQabQab)]

g[1 (gDd}XP (At.'O.bd.db)r
= exp
a~b a~b
.[1 (gDd.) exp (At.'O.bd. db ) (h 11, d.exp m) r

(13.141)
= J(il'
a<b
~~a.b) J
exp [K (allogXl (u; h) + t
/=2
a/log X/(u)
- t'QabQab)] (13.142)
a9
where X/(u) is defined the same as in (13.108) and
Xl(u;h) = J(fI Dda )

a=l
exp [h Ii: dam]
m=l
exp [Plt'Qabdadbj.
a<b
(13.143)
Assume replica symmetry as before but note that G =F - E and p =

1 - m + q. Then
Xl (U; h)
= E, {J (g Dd.) exp [h 11, d. m]
exp t,
[(P,E + Jp,F z) d. - ~l"Et, d;] } (13.144)
Therefore,
:h logXI(U;h)lh=o
= E. {J (gDd o) [.ITl d om ] exp [(i\E+ JPlF z) t,d o
- ~PlEt,d:]}j E. {J (gDd o) exp [(PlE+ JPlF z) t,d o
- ~PlEt,d:] } (13.145)
= Ez {[Jo(z)](u-i)[h(z)Ji} (13.146)
Ez {[Jo(z)]u}
where
(13.147)
(13.148)
and
h(z) = J Ddd exp [(PIE + y'PIF z) d - ~PIEd2] (13.149)
= (PIE + y'P1F z) (1
- -
+ P1E)-'2 exp
- 3 [ (PI E + n;F z
2(1 + PIE)
r1 .
(13.150)
Therefore,
:h logX1(u; h)lh=O
- - i - [i(PIE+n;Fzf]
(PIE + y'PIF z) (1 + PIE)-T exp 2(1 + PIE)
3i
= Ez {
(1 + PlE)-9 exp [(U - i) ~~: ;l~ z)']} / E. {(I +

(13.151)
(13.152)
Take the limit U -7 0,
(13.153)
From (13.137),
T.'_I
L\.I
81og((ZU(r,S;h))) I
8h
= ~l X ( 'h)1
>lh og I U, . (13.154)
h=O U h=O
Combine (13.128), (13.135), (13.153) and (13.154), we have
J~= (U, t. (dk)') ) = E, { C"~: ~ z ) l (13.155)
By considering a series of polynomials converging to sgn(·), we have

by (13.71) and (13.155)
M= Ez {sg n(PI E1 ++ ..;p;p

I PIE
z) }. (13.156)
The above is true for all groups of users. Also noting that E > 0, we
have for every group l = 1, ... , L,
Mz Ez{sgn(PIE+JPIFz)} (13.157)
= P (0 -JA~2) -p (z< -Jp,e;) (13.158)
(13.159)
By (13.70), we can easily conclude that all users in group I have the
same BER which is
(13.160)
where E and F are obtained through saddle-point equations (13.126a)-

(13.126d).
4.3 Arbitrary Energy Distribution

We now generalize the above results to the case where the users do not
fall into a finite number of equal-energy groups but their energies follow
an arbitrary energy distribution F. Consider a sequence of "simple"
distributions, F(L), L = 1,2, ... , that converge to F. For each L, assume
that the system consists of L groups of equal-energy users. Let
Pz(L) = min {p ~ 0 I F(P) ~ 1 ~ 1 }. 1= 1, ... , L, (13.161)
and
(L) 1
al = L' 1= 1, ... , L. (13.162)
Note that Pz(L) exists due to left-continuity of distribution function F.

Let p(L) be
p(L)(p) = i,
L
if p(L)
1-1 -
< P < p(L).
I
(13.163)
Clearly,
p(L)(p) ---+ P(P), V P. (13.164)
Also, taking the limit L ---+ 00, we have for all continuous function f
I: a}L) f(Pz{L») = ! f(P)dP{L)(p) --+ E{f(P)}

L
(13.165)
1=1
where the expectation is taken over the energy distribution P. Hence
we can generalize averaging among groups to an expectation in case of
an arbitrary energy distribution.
From (13.123) and by the above reasoning, the free energy under an
arbitrary energy distribution is
I
F()
1 { log(l + PE) -
= -E p 2E + PF } + -(3m
E - 1 - q) + -(1
P - m)
2 1 + PE 2 2
1 (f3 ) 1 (75 + f3(1 - 2m + q)
+ 2f3 log 1 + (72 (1 - m) + 2f3 (72 + f3(1 - m) . (13.166)
The saddle-point equations now become

1
E (13.167a)
CT 2 + ,8(1 - m)
m 1- E{I +PpE} (13.167b)
F
CT5 + ,8(1 - 2m + q)
(13.167c)
[CT 2 + ,8(1 - m)]2
3 2
= { P E +P2F}
(1 + PE)2
q (13.167d)
E .
The BER of user k with energy Pk converges to
(13.168)
The multiuser efficiency [1] for user k as

E2 2 1
(13.169)
17 = pCTO = 1 + ~ (1 _ 0"0
2m + q)
which is independent of the user number. The effective energy of user

k [1], CT5PkE2 / F, is proportional to one's own transmit energy. The
output SIR for user k is clearly PkE2 / F.
4.4 Linear Multiuser Detectors

4.4.1 The Matched Filter. Let CT ---+ 00 so that the condi-
tional mean estimator becomes equivalent to the matched filter. The
saddle-point equations give E, F ---+ 00 and m, p ---+ 0 and therefore
by (13.169), the multiuser efficiency is exactly 17(mf) given by (13.16) in
section 2.
4.4.2 The MMSE Detector. Let CT = CTO so that the con-

ditional mean estimator considered in the above becomes the MMSE
detector. (13.167c) reduces to
,8(q - m)
(13.170)
F = E + [CT2 + ,8(1 - m)]2
Interestingly, (13.167b) and (13.167d) lead to
P 2 (F - E)}
q - m = E { (1 + P E) 2 •
(13.171)
Plugging into (13.170), we see that
p2E2 }
F - E = (F - E) . /3 E{ (1 + PE)2 . (13.172)
Unless
(13.173)
takes the value of 1, which is in general not true, we have
(13.174)
and as a consequence,
q=m. (13.175)
Plug (13.174)-(13.175) into the saddle-point equations (13.167) and elim-

inate all variables but E, we get
(13.176)
Note that TJ = G5E. A rearrangement of the above is exactly the Tse-

Hanly equation given as (13.22) in section 2. Interestingly, E has the
physical meaning of the average value of the output SIR of the MMSE
detector.
4.4.3 The Decorrelator. Consider now the case G -t 0, where

the conditional mean estimator converges to the decorrelator. Equa-
tions (13.167a)-(13.167b) give
E02 + /3 E {I :~E} = 1. (13.177)
Suppose /3 < 1. As 0 -t 0 we must have E -t 00. By (13.167b),

m -t 1. Plugging (13.167c) into (13.167d) and taking the limit E -t 00
and m -t 1, we get
0 2
q= 1+ _0_. (13.178)
1-/3
Therefore, by (13.169),
TJ(dec) = 1- /3, /3 < 1. (13.179)

If f3 > 1, letting (J -+ 0 in (13.177) yields a finite solution for E as the

unique solution to
f3 E { PE } = 1. (13.180)
I+PE
m is obtained immediately from (13.167b). q can then be solved from
(13.167c)-(13.167d). The multiuser efficiency is found as a positive num-
ber by (13.169). In the special case of equal-energy users, E = 1/f3 - 1
and some algebra gives
(13.181)
which was also obtained through matrix eigenvalue analysis in [31].

Somewhat counterintuitively, by letting J( > N the decorrelator gets
out of the poor performance at J( less than but close to N. The rea-
son is that when J( < N but J( ::::::: N, the decor relator may smother
the desired user's signal while trying to tune out interferences. When
J( > N, however, there is no hope of tuning out all interferences and the
decorrelator finds a least-square solution instead, which gives non-zero
multiuser efficiency. Nevertheless, since the BER has an error floor in
this case, the asymptotic (low noise) multiuser efficiency is 0 for f3 > 1.
5. The Optimal Detectors

The method developed in section 4 can be applied to solve the per-
formance of the optimal detectors. Assume binary symmetric priors and
the same conditional distribution as before. The posterior probability
distribution is then given by (13.46)
p(dlr, S) = p(o)(dlr, S). (13.182)
Consider first a "simple" distribution with L groups of equal-energy

users. Consider u replicas of the system. Let zu
(r, S) be the partition
function. We can evaluate
((ZU(r, S)))
= J(ITn=l dr n ) L
{dad
2- uK (21l"(J5)-~
Es {ITn=l
exp [- 2:
0
2 (rn - ~ tk=l Msnk) 2]
YlV
1111N [1 ( n- Vii1t; ..,JP;. Snkdak 2] }

u
exp - 20"2 r
K ) (13.183)
where E{dad is a sum over all dak

E {-I, I}, a = 1, ... , u, k = 1, ... , K.
Again, we can separate the integral into the symbol dependent part Go
and the rest G 1 as in (13.87), where G1(u) is exactly the same as given
before, and
(13.184)
where
(13.186)
Assume the same replica symmetry as in section 4, and notice p = 1, we

have
XI(U)
[PIEtda + ~PzF (tda) ~PI(F - G) td~]

2
= L:: 2- u exp -
{d a } a=l a=l a=l
(13.187)
= exp [PI B- ~ PI (F - G) ]
Ez {L::Tuex p [(PtE + JPIFz) tda]} (13.188)
{d a } a=l
= exp [PIB - ~Pz(F - G)] Ez {cosh U

(PtE + JPzF z)}. (13.189)
In case of an arbitrary energy distribution, we can take the limit and

arrive at the following free energy
F(o) = -Ep,z {log cosh (PE + y'PF z)} + Em + ~F(l - q)

1 (f3 ) 10"5+f3(1-2m+q)
+ 2f3 log 1 + 0"2 (1 - q) + 2f3 0"2 + f3(1 _ q) . (13.190)
The parameters {E, F, m, q} satisfy the saddle-point equations
1
E (13.191a)
0"2 + f3(1 - q)
m Ep,z {Ptanh (PE + y'PF z)} (13.191b)
0"5 + f3(1 - 2m + q)
F (13.191c)
[0"2 + f3(1 - q)J2
q Ep,z {Ptanh 2 (PE + y'PF z)}. (13.191d)
The overlap can also be solved similarly and we get
Hence we have the same expression for the overlap as (13.156). The
multiuser efficiency is given by the same (13.169).
The conditional mean estimator reduces to the matched filter if 0" --+
00. In this case m, q --+ 0 and we get (13.16) by (13.169).
In case of 0" = 0"0, we get the 10 detector. Notice that for all x,
Ez {tanh(x + v'x z) - tanh 2 (x + v'x z)}
~Je-(y;:)2 [tanh(y) - tanh 2 (y)] dy (13.193)
v27rx
-- 1 e - 2X
y'27rX
1 J e- Y!...
2 :r
eY - e- Y dy
(e Y + e- y)2
(13.194)
o (13.195)
since the integrand in (13.194) is an odd function. It can be shown that
the solution to the saddle-point equations satisfies F = E and q = m.
Therefore the multiuser efficiency, 0"5E, can be found as the solution to
the following fixed-point equation
1
1] -
- 1+ !I [1 - Ep,z { P tanh ( fJ z + ~ ) }] . (13.196)
In case (j -7 0, saddle-point equations (13.191) jointly determine the

performance of the JO detector. The fixed points can be found numeri-
cally. The multiuser efficiency can be obtained by (13.169).
6. Discussions
The parameters m and q have interesting physical meanings. Take
the linear detector for example. Consider a simple energy distribution.
By letting i = 1 in (13.155), we get
(13.197)
and by letting i = 2,
(13.198)
With slight abuse of notation we define
(13.199)
Then
« (d) )) (13.200)
By (13.126b) and making use of the limiting distribution arguments, one

easily get
«(d))) = m. (13.203)
Similarly,
(13.204)
The same expressions are resulted if we start with (13.192) for the op-
timal detectors. Indeed, (13.203)-(13.204) are true for all multiuser de-
tectors of interest.
Let
E = 1- 2m+ q. (13.205)
One finds that
E 1 - 2 (( (d) )) + (( (d) 2 ) ) (13.206)

(( (1 - (d))2)). (13.207)
Recall that the transmitted symbol dk = 1 for each user k. The right
hand side of (13.207) is the mean square error of the conditional mean
estimator output. Interestingly, the multiuser efficiency is then expressed
in the mean square error as
1
ry=---::- (13.208)
1 + EP...
(}"~
by (13.169). Comparing it with the matched filter efficiency (13.16),
'I1(mf) _ _1_
./ - {3 , (13.209)
1 + (}"2o
we see that a multiuser detector behaves as a matched filter with a

spreading factor expanded by a factor of l/E, or a user population re-
duced by a factor of E, or an enhanced Gaussian noise variance by E(3,
where E is the mean square error of the soft output of the corresponding
conditional mean estimator. The expression also reminds us a simple
interpretation of the multiuser efficiency. The 1 in the denominator
in (13.208) is contributed by the thermal noise, while the term E(3/(/5 is
contributed by the output MAL We conclude by giving a canonical inter-
ference canceller as shown in Fig. 13.2 which makes use of the conditional
mean estimator's output to reconstruct the interference for cancellation
before matched filtering by the desired user's spreading codes. The soft
output for user 1 is expressed as
d1 = s~ (r- t, JP.Sk(dk» (13.210)
K
.;p; d1 + Lsfsk.j"P; (dk - (dk)) + O"OWI (13.211)
k=2
Conditional
Mean Estimator
Figure 13.2. A canonical interference canceller.
where WI is a standard Gaussian random variable. An important ob-

servation here is that the MAl term is asymptotically Gaussian with
a variance Ej3, so that the resulting multiuser efficiency as expressed
by (13.208). The canonical structure suggests interference cancellation
structure for implementing the multiuser detectors, if the posterior mean
output for interfering users can be approximated by interference can-
cellers of the same structure.
7. Spectral Efficiency
The maximum sum of the rates of a CDMA channel is equal to the
maximum mutual information between transmitted symbols and the re-
ceived signal
D(PrldllPrIPd) (13.212)
E {log P(rld)} + E {-log P(r)}. (13.213)
Assume the spreading codes are known and fixed. By (13.6), P r1d is a
Gaussian density, and hence
E {log P(rld)} E {log P(0"511)} (13.214)

N
-2 (1 + log(27r0"5)) . (13.215)
The Gaussian prior is known to give the maximum of the conditional

divergence D (Prld II Pr IPd). Let dk'S be independent standard Gaussian
random variables so that the prior distribution is given by in (13.40).
Then Pr is given by
P(r) = ! dd(27r)-~exp [_~lldI12] (27ra6)-~exp [-2~21Ir- SAdIi2]
(13.216)
= (27ra6)-~ Z(l)(r, S) (13.217)
where Z(l) (r, S) is given by (13.42) with a = ao. Hence

N
E {-log P(r)} = 2log(27ra6) - log Z(l) (r, S). (13.218)
By definition of the free energy in (13.73),
lim ~E {-log P(r)} = ~ log (27ra6) + ,8.1"(1)117=170. (13.219)

K-too N 2
Therefore, in the large-system limit, the spectral efficiency can be ex-
pressed as
. 1
C = hm N1(d1, ... , dK; r) (13.220)
K-too
1 1
= -"2(1 + log(27ra6)) + "2log(27ra6) + ,8.1"(1)117=170 (13.221)
= ~E
2
{log(l + PE)} + ,8E (m -
2
1) + ~2 log (1 + a,82 (1 -
o
m)) .
(13.222)
By (13.167a)-(13.167b) and that 1](mmse) = aJE, the spectral efficiency
is exactly (13.31), which is repeated here,
,8 { log ( 1 + 1] (mmse))
C = -E } + -log
1 1 + _(1](mmse)
1
2 aJ P 2 1](mmse) 2
- 1).
(13.223)
Not surprisingly, independent fading on an equal transmit energy system
is a perfect cause of the energy distribution. Therefore we have obtained
the spectral efficiency as a function of the multiuser efficiency. In fact,
we can identify the first term on the right hand side of (13.223) to be
the spectral efficiency of the linear MMSE receiver.
The spectral efficiency gives the maximum total number of bits per
second per chip that can be transmitted reliably from all users to the
receiver over the multiuser channel. Unfortunately it does not easily
break down to a rate combination of individual users that achieves it
as (13.223) may seem to suggest.
8. Conclusions
This paper exploits the connection between large-system multiuser
detection and statistical mechanics, and presents a new interpretation
of multiuser detection in general.
We first introduced the concept of Bayes retrochannel, which takes
the received signal as the input and outputs a stochastic estimate of
the transmitted symbols. By assuming an appropriate prior distribu-
tion and a channel characteristic that may be different to the true ones,
a multiuser detector can be expressed as a conditional mean estima-
tor that outputs the mean value of the stochastic output of the Bayes
retrochannel.
The Bayes retrochannel is equivalent to a spin glass in the sense that
the distribution of its stochastic output conditioned on the received sig-
nal is exactly the distribution of the spin glass at thermal equilibrium.
The performance of the multiuser detector is then found as a certain
macroscopic property of the spin glass. In particular, the BER can
be obtained through calculating the overlap of the spin glass. In the
large-system limit, the macroscopic properties as such can be solved by
powerful tools developed in statistical mechanics.
In this paper we have solved through a unified analysis the large-
system uncoded BER of the matched filter, the MMSE detector, the
decorrelator, the individually and jointly optimal detectors. We show
that under arbitrary received energy distribution, the large-system BER
is uniquely determined by the multiuser efficiency, which has a very
simple relationship with the output mean square error of the conditional
mean estimator. The relationship also implies that a multiuser detector
is in general equivalent in performance to interference subtraction using
a conditional mean estimator obtained for certain prior and conditional
distribution (depending on the detector), and the remaining interference
is always Gaussian distributed in the large-system limit.
Using the techniques developed in this paper, we have also solved the
spectral efficiency of the multiuser channel.
Notes
1. Imagine a system consisting molecules with random magnetic spins that evolve over
time, while the random positions of the molecules are fixed for each concrete instance as in
a piece of glass.
2. The SIR is defined as the energy ratio of the useful signal to the noise in the output.
In contrast, the SNR of user k is usually defined as Pk/(2a~).
3. Precisely, we refer to the large-system BER. Unless otherwise stated, all performance
indexes such as BER, SIR, multiuser efficiency and spectral efficiency refer to large-system
performance hereafter.
REFERENCES 275
References
[1] S. Verdu, Multiuser Detection. Cambridge University Press, 1998.
[2] D. Guo, L. K. Rasmussen, and T. J. Lim, "Linear parallel interfer-
ence cancellation in long-code CDMA multiuser detection," IEEE
J. Selected Areas Commun., vol. 17, pp. 2074-2081, Dec. 1999.
[3] A. J. Grant and P. D. Alexander, "Random sequence multisets for
synchronous code-division multiple-access channels," IEEE Trans.
Inform. Theory, vol. 44, pp. 2832-2836, Nov. 1998.
[4] D. N. C. Tse and S. V. Hanly, "Linear multiuser receivers: effective
interference, effective bandwidth and user capacity," IEEE Trans.
Inform. Theory, vol. 45, pp. 622-640, March 1999.
[5] S. Verdu and S. Shamai, "Spectral efficiency of CDMA with ran-
dom spreading," IEEE Trans. Inform. Theory, vol. 45, pp. 622-640,
March 1999.
[6] D. N. C. Tse and S. Verdu, "Optimum asymptotic multiuser effi-
ciency for randomly spread CDMA," IEEE Trans. Inform. Theory,
vol. 46, pp. 2718-2722, Nov. 2000.
[7] R. R. Miiller, "Multiuser receivers for randomly spread signals: fun-
damentallimits with and without decision-feedback," IEEE Trans.
Inform. Theory, vol. 47, pp. 268-283, Jan. 2001.
[8] P. Viswanath, D. N. C. Tse, and V. Anantharam, "Asymptoti-
cally optimal water-filling in vector multiple-access channels," IEEE
Trans. Inform. Theory, vol. 47, pp. 241-267, Jan. 2001.
[9] J. Zhang, E. K. P. Chong, and D. N. C. Tse, "Output MAl distri-
bution of linear MMSE multiuser receivers in DS-CDMA systems,"
IEEE Trans. Inform. Theory, vol. 47, pp. 1128-1144, March 2001.
[10] S. Shamai and S. Verdu, "The impact of frequency-flat fading on
the spectral efficiency of CDMA," IEEE Trans. Inform. Theory,
vol. 47, pp. 1302-1327, May 2001.
[11] D. Guo, S. Verdu, and L. K. Rasmussen, "Asymptotic normality
of linear CDMA multiuser detection outputs," in Proceedings 2001
IEEE International Symposium on Information Theory, p. 307,
Washington, D.C., June 2001.
[12] L. Li, A. Tulino, and S. Verdu, "Asymptotic eigenvalue moments
for linear multiuser detection," Communications in Information and
Systems, vol. 1, pp. 273-304, Sept. 2001.
[13] D. Guo, S. Verdu, and L. K. Rasmussen, "Asymptotic normality
of linear multiuser receiver outputs," accepted for IEEE Trans. on
Inform. Theory, 2001.
[14] U. Grenander and J. W. Silverstein, "Spectral analysis of networks

with random topologies," SIAM J. Appl. Math., vol. 32, pp. 499-
519, March 1977.
[15] D. Jonsson, "Some limit theorems for the eigenvalues of a sample
covariance matrix," Journal of Multivariate Analysis, pp. 1-38, Dec.
1982.
[16] J. W. Silverstein and Z. D. Bai, "On the empirical distribution
of eigenvalues of a class of large dimensional random matrices,"
Journal of Multivariate Analysis, vol. 54, pp. 175-192, Aug. 1995.
[17] T. M. Cover and J. A. Thomas, Elements of Information Theory.
New York: Wiley, 1991.
[18] N. Sourlas, "Spin-glass models as error-correcting codes," Nature,
vol. 339, pp. 693-695, June 1989.
[19] N. Sourlas, "Spin-glasses, error-correcting codes and finite-
temperature decoding," Europhysics Letters, vol. 25, Jan. 1994.
[20] A. Montanari and N. Sourlas, "The statistical mechanics of turbo
codes," Eur. Phys. J. B, vol. 18, no. 1, pp. 107-119, 2000.
[21] Y. Kabashima and D. Saad, "Statistical mechanics of error-
correcting codes," Europhysics Letters, vol. 45, no. 1, pp. 97-103,
1999.
[22] H. Nishimori and K. Y. M. Wong, "Statistical mechanics of image
restoration and error-correcting codes," Pysical Review E, vol. 60,
pp. 132-144, July 1999.
[23] K. Nakamura, Y. Kabashima, and D. Saad, "Statistical mechanics
of low-density
parity check error-correcting codes over Galois fields," Europhysics
Letters, vol. 56, pp. 610-616, Nov. 2001.
[24] H. Nishimori, Statistical Physics of Spin Glasses and Information
Processing: An Introduction. Number 111 in International Series of
Monographs on Physics, Oxford University Press, 2001.
[25] G. 1. Kechriotis and E. S. Manolakos, "Hopfield neural network
implementation of the optimal CDMA multiuser detector," IEEE
Trans. Neural Networks, Jan. 1996.
[26] V. Dotsenko, The Theory of Spin Glasses and Neural Networks.
World Scientific, 1994.
[27] T. Tanaka, "Analysis of bit error probability of direct-sequence
CDMA multiuser demodulators," in Advances in Neural Informa-
tion Processing Systems (T. K. Leen et aI., ed.), vol. 13, pp. 315-321,
The MIT Press, 2001.
REFERENCES 277
[28] T. Tanaka, "Average-case analysis of multiuser detectors," in Pro-

ceedings 2001 IEEE International Symposium on Information The-
ory, p. 287, Washington, D.C., June 2001.
[29] T. Tanaka, "Large-system analysis ofrandomly spread CDMA mul-
tiuser detectors," preprint, 200l.
[30] J.- P. Bouchaud and M. Potters, Theory of Financial Risks: From
Statistical Physics to Risk Management. Cambridge University
Press, 2000.
[31] Y. C. Eldar and A. M. Chan, "On wishart matrix eigenvalues and
eigenvectors and the asymptotic performance of the decorrelator,"
IEEE Trans. Inform. Theory, 2001. submitted.

Multiuser Detection and Statistical Mechanics: Dongning Guo

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Multiuser Detection and Statistical Mechanics: Dongning Guo

Uploaded by

Copyright:

Available Formats

Chapter 13

MULTIUSER DETECTION AND

Abstract A framework for analyzing multiuser detectors in the context of sta-

in solving the capacity (or the spectral efficiency) of CDMA channels

value of the stochastic output of the Bayes retrochannel. By carefully

2. The CDMA Channel and Multiuser Detectors

Let PI, .. . , PK be the K users' respective energies. The received signal

where {vn } are independent standard Gaussian random variables, and

The SIR 2 of user k under matched filtering in the absence of interfering

r = SAd + O"OV (13.5)

where II . II denotes the norm of a vector.

2.2 Multiuser Detectors: Known Results

Another important performance index is the multiuser efficiency, which

Immediately, the BER can be expressed in the multiuser efficiency as

In this chapter, we study the matched filter, the decorrelator, the

2.2.1 The Single-user Matched Filter. Assuming absence

where Wk is a standard Gaussian random variable. The second term,

The multiuser efficiency is the same for all users

SIR (mf) _ Pk. (mf) (13.17)

2.2.2 The MMSE Detector. The MMSE detector is a linear

The decision statistic for user k can be described as

J~mmse) = Hkkdk + L Hikdi + aWk (13.19)

where Hik is the element of H = A -1 [SHS + a 2 A -2] -1 SHSA on the ith

In the case of equal-energy users, the large-system SIR of the MMSE

which is the solution to

TJ+,6E{ PTJ 2}=1 (13.22)

2.2.3 The Decorrelator. The decorrelating detector, or, the

where (.)+ denotes the Moore-Penrose psudo-inverse, which reduces to

TJ(dec) =1- ,6. (13.24)

It is incorrectly claimed in [29] that the multiuser efficiency is 0 if ,6 > 1.

2.2.4 The Optimal Detectors. The jointly optimal detector

2.3 Spectral Efficiency

Channel r Bayes Retrochannel

Conditional Mean Estimator

If a linear detector discussed in section 2.2 is used, the spectral effi-

3. Conditional Mean Estimator and Statistical

retrochannel defined by the conditional distribution

p(dlr) = Z-l (r)p(rld)p(d) (13.32)

where p( d) is a postulated prior distribution and p(rld) a postulated

Z(r) = LP(rld)p(d) (13.33)

is a normalizing factor such that p(dlr) becomes a valid probability

(d) = p(d = llr) - p(d = -llr) (13.35)

whose sign is the maximum a posteriori detector output.

3.2 CDMA Multiuser Detectors

Z(r, S) = ~P(d) exp [- 2~211r - SAd 112 ] . (13.38)

3.2.1 The Linear Detectors. Assume standard Gaussian

Then the posterior probability distribution is

Here we use the superscript (1) for linear detectors.

Since the probability given by (13.41) is exponential in a quadratic

3.2.2 The Optimal Detectors. Let p( d) be the true binary

Z(O)(r, S) = L _exp [- 2~211r - SAd Il2 ] . (13.47)

We use the superscript (0) for the optimal detectors.

p(O)(dlr, S) =1- -1-llr - SAdW + 0

We have expressed each of the multiuser detectors of interest as a

3.3 Preliminaries of Statistical Mechanics

where p(d) is the probability of the system being found in configuration

- 2:p(d) logp(d). (13.52)

It is assumed that the system is not completely isolated from the

f = - ~P(d) logp(d) - ( (~P(d)H(d) - £) -~ (~P(d) 1) -

where ( and 'Yare the Lagrange multipliers. Variation with respect to p

p(d) = Z-l exp [-(H(d)] (13.54)

Z = 2: exp [-(H(d)] (13.55)

is a normalizing factor called the partition junction, and the parameter