Professional Documents
Culture Documents
5, MAY 2010
solution at those points. To avoid these situations of the inverse model become
the data is first checked for contradictions. If x [ x1 x2 y2 y3 ]T and output vector
contradiction exists, the inverse data is divided
into sub-groups such that each sub-group does not becomes y [ y1 x3 x4 ]T . Figure 1 shows the
contain any contradictory data. Multiple sub- diagrams of a neural network forward model and
models are then developed using the divided data. an inverse model. The inverse model of Fig. 1(b)
The sub-models are then combined using a special is formulated by swapping partial inputs and
technique to form the overall inverse model. The outputs of the forward model of Fig. 1(a). Note
description of various techniques is provided in that two input parameters and two output
the Section 2. parameters are swapped.
We also review the recent advances of neural
network approaches for behavioral modeling of y1 y2 y3 y1 x3 x4
nonlinear microwave circuits. We focus on the
specific artificial neural network (ANN) structures
that are capable of learning and representing
dynamic behaviors of nonlinear circuit blocks.
Two ANN-based techniques, i.e., recurrent neural
networks (RNN) [11–13] and dynamic neural
networks (DNN) [14] techniques, are described
from the perspective of nonlinear behavioral
modeling. Numerical examples of modeling RF
amplifiers and mixers are included. x1 x2 x3 x4 x1 x2 y2 y3
not selected. If the outputs of other inverse sub- activation functions. The FFNN outputs are linear
models are also found outside their training range functions of the responses of hidden neurons in the
then we compare their magnitude of distances hidden layer. Let the trainable parameters of the
from the boundary of training range. An inverse RNN be denoted as w. As can be observed from
sub-model producing the lowest distance is Fig. 2, the overall RNN structure, including both
selected in this case. the FFNN part and feedback connections, realizes
the following nonlinear dynamic relationship [12]
III. ANN-BASED DYNAMIC
BEHAVIORAL MODELING OF y (kT )
MICROWAVE CIRCUITS f FFNN ( y ((k 1)T ),..., y ((k M y )T ), (6)
A. Recurrent Neural Networks (RNN) for
Time Domain Modeling u(kT ), u ((k 1)T ),..., u(( k M u )T ), w , p)
Conventional feed-forward neural networks
(FFNN) [1] are well known for their learning and where k is the index for time step, T is the time
generalization capabilities. However, they are only step size, and is a delay element. The number of
suitable for mapping static input-output delayed time steps M y and M u , of y and u
relationships. To model nonlinear circuit responses respectively, represent the effective order of
in time-domain, a neural network that can include original nonlinear circuit as seen from input-output
temporal information is necessary. RNNs have data. f FFNN represents a static mapping function
been found to be a suitable candidate to
accomplish this task. In the past, RNNs were that could be any of the standard FFNN structures,
successfully used in various engineering e.g., a multilayer perceptron (MLP) neural
applications such as system control, speech network [1]. The formulation (6) is a
recognition, etc [16]. For microwave dynamic generalization over the conventional RNN
modeling, the structure of a typical RNN is shown structure [13] by introducing the extra delay ,
y(kT-) which represents the delay between the input and
RNN output signals [12]. It has been found that this
modified structure could help simplify the overall
FFNN training of the model.
… Hidden
Neurons
B. Dynamic Neural Networks (DNN) for
Nonlinear Behavioral Modeling
For the purpose of circuit simulation, the most
… … … ideal format to describe nonlinear dynamics is the
continuous-time domain formulation. In theory,
y((k-1)T-) y((k- My)T-) u((k-1)T) u((k-Mu)T)
this format best describes the fundamental essence
of nonlinear behavior and in practice it is flexible
u(kT)
Time-independent to fit most or all the needs for nonlinear circuit
parameters (p) simulation. On the other hand, for large-scale
nonlinear microwave circuits, the detailed state
Fig. 2. RNN structure with output feedback (My). equations [14] could be too complicated,
The RNN is a discrete time structure trained with computationally expensive, and sometimes even
sampled input-output data [12]. unavailable at system level. Therefore, a
in Fig. 2 [12]. The RNN inputs include time- simplified (reduced order) model approximating
varying inputs u and time-independent inputs p. the same dynamic input-output relationships is
The RNN outputs are the time varying signal y. highly required. With these motivations, DNN
The input layer of the FFNN contains buffered technique [14] was presented for large-signal
(time-delayed) history of y fed back from the modeling of nonlinear microwave circuits and
output layer, buffered history of u, and p. The systems. Let Nd be the order of DNN representing
hidden layer contains neurons with sigmoid the reduced order for original nonlinear circuit. Let
427 ACES JOURNAL, VOL. 25, NO. 5, MAY 2010
v1 (t ) v 2 (t ) y [ x3 x4 y3 y4 ] T [ Lv Lh Pv Ph ] T (8)
x [ x1 x2 y1 y2 ] [ D fo M 23 M 14 ] . (9)
T T
v N d -1 (t ) v Nd (t ) (7)
where D is the circular cavity diameter, fo is the
v N d (t ) g ANN (v N d (t ), v N d 1 (t ), , center frequency, M23 and M14 are coupling values,
v1 (t ), u(Nd ) (t ), u(N d -1) (t ), , u(t )) Lv and Lh are the vertical and horizontal coupling
slot lengths and Pv and Ph are the loading effect of
the coupling iris on the two orthogonal modes,
where inputs and outputs of the DNN model are respectively. After formulating the inverse model
u(t ) and y (t ) v1 (t ) , respectively. The overall training data were generated and the entire data
DNN model is in a standardized format for typical was used to train the inverse model. Direct
nonlinear circuit simulators. For example, the left- training produced good accuracy in terms of least
hand-side of the equation provides the charge or square (L2) errors. However the worst-case error
the capacitor part, and the right-hand-side provides was large. Therefore in the next step the data was
the current part. This format is the standard segmented into four sections. Models for these
representation of nonlinear components in many sections were trained separately, which reduced
harmonic balance (HB) simulators. In this way, the worst-case error. The final model accuracy of
DNN can provide dynamic current-charge the two methods is shown in Table 1. We can
parameters for general nonlinear circuits with any improve the accuracy further by splitting the data
number of internal nodes in original microwave set into more sections and achieve as accurate
circuit [14]. The DNN technique has been result as required.
successfully used in modeling of nonlinear The second inverse model of the filter is the IO
microwave circuits such as mixers and amplifiers. iris model. The input parameters of IO iris inverse
The trained DNN behavioral models were also model are circular cavity diameter D, center
used to facilitate fast high-level HB simulations of frequency fo, and the coupling value R. The output
circuits and systems. As a recent advance, the parameters of the model are the iris length L, the
work in [17] introduced a mathematical way to loading effect of the coupling iris on the two
determine the order of the DNN formulation from orthogonal modes Pv and Ph, and the phase loading
the training data. A further enhancement is made on the input rectangular waveguide Pin. The
in [18] where a modified HB formulation inverse model is defined as
incorporating constraint functions was presented
to improve the robustness and efficiency of DNN- y [ x3 y2 y3 y4 ] T [ L Pv Ph Pin ] T (10)
based HB simulation of high-level nonlinear
microwave circuits. x [ x1 x2 y1 ] T [ D fo R ] T . (11)
worst-case error is high. To reduce the worst-case Table 1: Comparison of error between
error we split the first set of data into several conventional and proposed method for waveguide
segments and trained separately. These models filter model [8].
produced acceptable accuracy. Same method was Inverse Model error (%)
applied to the rest of the three models. The result Waveguide modeling
is presented in Table 1, which shows that the junctions methods L2 Worst
proposed methodology produce more accurate case
result than the direct modeling method. Coupling iris Conventional 0.46 14.2
The last neural network inverse model of the Proposed 0.32 7.20
filter is developed for tuning screw model. The IO iris Conventional 1.30 54.0
input parameters of this model are circular cavity Proposed 0.45 18.4
diameter D, center frequency fo, the coupling Tuning screw Conventional 7.51 94.25
between the two orthogonal modes in one cavity Proposed 0.59 8.10
M12, and the difference between the phase shift of
the vertical mode and that of the horizontal mode
across the tuning screw P. The model outputs are
the phase shift of the horizontal mode across the B. Dual Mode 6-pole Waveguide Filter Design
tuning screw Ph, coupling screw length Lc, and the Using the Developed Neural Network Inverse
horizontal tuning screw length Lh. The inverse Models
model is formulated as In this example we design a 6-pole waveguide
filer using the proposed methodology [8]. The
y [ y3 x3 x4 ] T [ Ph Lh LC ] T (12) filter center frequency is 12.155 GHz, bandwidth
x [ x1 x2 y1 y2 ] T [ D fo M 12 P ] T . (13) is 64 MHz and cavity diameter is chosen to be
1.072". In addition to the three inverse models that
All four techniques in Section II were applied to were developed in Example A, we developed
develop this model. The final model result is another inverse model for slot iris. The inputs of
presented in Table 1, which shows that the the slot iris model are cavity diameter D, center
accuracy of the tuning screw model is improved frequency fo and coupling M and the outputs are
drastically using the proposed method. Minor iris length L, vertical phase Pv and horizontal
improvement is realized for the coupling iris phase Ph. The normalized ideal coupling values are
model in terms of L2 error (the improvement is
mostly realized in terms of worst-case error), R1 = R2 = 1.077
because the input-output relationship of this model
is relatively simpler than that of the tuning screw 0 0.855 0 0.16 0 0
model. The proposed method becomes more 0.855
0
0
0.719
0.719
0
0
0.558
0
0
0
0
M .
efficient and effective for complex devices.
Three-layer multilayer perceptron neural 0.16 0 0.558 0 0.614 0
network structure was used for each neural 0 0 0 0.614 0 0.87
network model and quasi-Newton training
algorithm was used to train the neural network
0 0 0 0 0.87 0
(14)
models. Testing data were used after training the
Irises and tuning screw dimensions are
model to verify the generalization ability of these calculated by the trained neural network inverse
models. Automatic model generation algorithm of models developed in Example A. The filter is
NeuroModelerPlus [19] was used to develop these manufactured and tuned by adjusting irises and
models, which automatically train the model until tuning screws to match the ideal response and the
model training, and testing accuracy was satisfied. dimensions are listed in Table 2. Very good
The training error and test errors were generally correlation can be seen between the initial
similar because sufficient training data was used in dimensions provided by the neural network
the examples. inverse models and the measured final dimensions
of the fine tuned filter. Figure 3 presents the
429 ACES JOURNAL, VOL. 25, NO. 5, MAY 2010
S11/S21 (dB)
EM model, neural network inverse models and -30
approximately 100 seconds to run the entire current values of intermediate frequency and radio
envelope simulation for the 3G WCDMA input frequency, respectively. The DNN model includes,
whereas each RNN only takes 0.16 seconds to
reproduce accurately the output for the same 3G ( n) ( n1) ( n2)
WCDMA input. iRF (t ) = f ANN1[iRF (t ), iRF (t ),, iRF (t ),
( n) ( n1)
vRF (t ),vRF (t ),,vRF (t )]
(15)
and
vIF(n) (t) fANN2[vIF(n1) (t), vIF(n2) (t),, vIF (t),
(n) (n1)
vRF (t), vRF (t), , vRF (t),
(n) (n1)
vLO (t), vLO (t),, vLO(t),
iIF(n) (t), iIF(n1) (t),, iIF (t)]
(16)
where n is the order of the DNN.
(a) The training data were generated by varying the
RF input frequency and power level from 11.7
GHz to 12.1 GHz with a step-size of 0.05 GHz and
from -45 dBm to -35 dBm with a step-size of 2
dBm respectively. Local oscillator signal was
fixed at 10.75 GHz and 10 dBm. Load was
perturbed by 10% at every harmonic in order to
allow the model learn the loading effects. The
DNN was trained with different number of hidden
neurons as shown in Table 3. Testing was done in
ADS [20] using input frequencies from 11.725
GHz to 12.075 GHz with a step-size of 0.05 GHz
and power levels at -44 dBm, -42 dBm, -40 dBm, -
38 dBm, -36dBm. The agreement between model
(b) and ADS was achieved in time and frequency
Fig. 4. (a) AM/AM distortion between ADS and domains even though those test information was
RNN power amplifier (PA) behavioral model. (b) never seen in training. Figure 5 illustrates a
AM/PM distortion between ADS and RNN PA comparison between the output of the DNN model
behavioral model [12]. and the ADS solution in time-domain.
REFERENCES
[1] Q. J. Zhang, K. C. Gupta, and V. K.
Devabhaktuni, “Artificial neural networks for
RF and microwave design-from theory to
practice,” IEEE Trans. Microwave Theory and
Tech., vol. 51, pp. 1339–1350, Apr. 2003.
[2] Q. J. Zhang and K. C. Gupta, Neural Networks
for RF and Microwave Design, Artech House,
Boston, 2000.
[3] P. M. Watson and K. C. Gupta, “EM-ANN
Fig. 5 Mixer vIF output: time-domain comparison models for microstrip vias and interconnects
between DNN (—) and ADS solution of original in dataset circuits,” IEEE Trans. Microwave
circuit (o). Good agreement is achieved even Theory and Tech., vol. 44, pp. 2495–2503,
though such data were never used in training [14]. Dec. 1996.
[4] B. Davis, C. White, M. A. Reece, M. E. Jr.
Bayne, W. L. Thompson, II, N. L. Richardson,
V. CONCLUSION and L. Jr. Walker, “Dynamically configurable
We have reviewed recent advances of neural pHEMT model using neural networks for
network modeling techniques for fast modeling CAD,” IEEE MTT-S Int. Microwave Symp.
and design of RF/microwave circuits. Inverse Dig., vol. 1, pp. 177–180, June 2003.
modeling technique is formulated and non- [5] M. Isaksson, D. Wisell, and D. Ronnow,
uniqueness of input-output relationship has been “Wide-band dynamic modeling of power
addressed. A method to identify and divide amplifiers using radial-basis function neural
contradictory data has been proposed. Inverse networks,” IEEE Trans. Microwave Theory
models are divided based on derivatives of and Tech., vol. 53, no. 11, pp. 3422–3428,
forward model and then trained separately to Nov. 2005.
produce more accurate inverse sub-models. A [6] H. Kabir, Y. Wang, M. Yu, and Q. J. Zhang,
method to correctly combine the inverse sub- “Applications of artificial neural network
models is presented. The proposed methodology techniques in microwave filter modeling,
has been applied to waveguide filter modeling and optimization and design,” Progress In
design. Very good correlation was found between Electromagnetic Research Symposium,
neural networks predicted dimensions and that of a Beijing, China, pp. 1972–1976, March 2007.
perfectly tuned filter following the EM method. [7] P. Burrascano, M. Dionigi, C. Fancelli, and M.
We have also reviewed recurrent neural network Mongiardo, “A neural network model for
and dynamic neural network modeling techniques. CAD and optimization of microwave filters,”
These time-domain neural networks are trained to IEEE MTT-S Int. Microwave Symp. Dig.,
learn the nonlinear dynamic input-output Baltimore, MD, vol. 1, pp. 13–16, June 1998.
relationship based on the external circuit signals. [8] H. Kabir, Y. Wang, M. Yu and Q. J. Zhang,
The resulting neural network models are fast and “Neural network inverse modeling and
accurate to predict the behavior of RF and applications to microwave filter design” IEEE
microwave circuits. Trans. Microwave Theory and Tech., vol. 56,
no. 4, pp. 867–879, Apr. 2008.
[9] Y. Wang, M. Yu, H. Kabir, and Q. J. Zhang,
“Effective design of cross-coupled filter using
ACKNOWLEDGMENT neural networks ad coupling matrix,” IEEE
Authors would like to thank Dr. Ming Yu of
MTT-S Int. Microwave Symp. Dig., pp. 1431–
COMDEV for his support and collaboration in the
1434, June 2006.
KABIR, CAO, CAO, ZHANG: ADVANCES OF NEURAL NETWORK MODELING METHODS FOR RF/MICROWAVE APPLICATIONS 432
[10] M. M. Vai, S. Wu, B. Li, and S. Prasad, [15] J. Xu, M. C. E. Yagoub, R. Ding, and Q. J.
“Reverse modeling of microwave circuits Zhang, “Exact adjoint sensitivity analysis for
with bidirectional neural network models,” neural-based microwave modeling and
IEEE Trans. Microwave Theory and Tech., design,” IEEE Trans. Microwave Theory and
vol. 46, pp. 1492–1494, Oct. 1998. Tech., vol. 51, pp. 226–237, Jan. 2003.
[11] Q. J. Zhang and Y. Cao, “Time-domain [16] S. Haykin, Neural Networks: A
neural network approaches to EM modeling Comprehensive Foundation, 2nd ed.,
of microwave components,” Springer Prentice-Hall, pp. 751–756, 1999.
Proceedings in Physics: Time Domain [17] J. Wood, D. E. Root, and N. B. Tufillaro, “A
Methods in Electrodynamics, pp. 41–53, Oct. behavioral modeling approach to nonlinear
2008. model-order reduction for RF/microwave ICs
[12] H. Sharma and Q. J. Zhang, “Automated time and systems,” IEEE Trans. Microwave
domain modeling of linear and nonlinear Theory Tech., vol. 52, no. 9, pp. 2274–2284,
microwave circuits using recurrent neural Sep. 2004.
networks,” Int. J. RF Microwave Comput.- [18] Y. Cao, L. Zhang, J. J. Xu and Q. J. Zhang,
Aided Eng., vol.18, no. 3, pp. 195–208, May “Efficient harmonic balance simulation of
2008. nonlinear microwave Circuits with dynamic
[13] Y. H. Fang, M. C. E. Yagoub, F. Wang, and neural models,” IEEE MTT-S Int.
Q. J. Zhang, “A new macromodeling Microwave Symp. Dig., , pp. 1423–1426,
approach for nonlinear microwave circuits June 2006.
based on recurrent neural network,” IEEE [19] Q. J. Zhang, NeuroModelerPlus, Department
Trans. Microwave Theory Tech., vol. 48, no. 12, of Electronics, Carleton University, K1S
pp. 2335–2344, Dec. 2000. 5B6, Canada.
[14] J. J. Xu, M. Yagoub, R. T. Ding, and Q. J. [20] Advanced Design System (ADS) ver. 2006A,
Zhang, “Neural based dynamic modeling of Agilent Technologies, Inc., 2006.
nonlinear microwave circuits,” IEEE Trans.
Microwave Theory Tech., vol. 50, pp. 2769–
2780, Dec. 2002.