packages:cran:amore [R Wiki

]

http://rwiki.sciviews.org/doku.php?id=packages:cran:amore

The AMORE package: A MORE flexible neural network package
Overview
The AMORE package was born to provide the user with an unusual neural network simulator: a highly flexible environment that should allow the user to get direct access to the network parameters, providing more control over the learning details and allowing the user to customize the available functions in order to suit their needs. The current status (version 0.2-9) of the package is capable of training a multilayer feedforward network according to both the adaptive and the batch versions of the gradient descent with momentum backpropagation algorithm. Thanks to the structure adopted, expanding the number of available error criteria is as difficult as programming the corresponding R costs functions. Everyone is invited to grow the number of available functions and share their experiences.

Description of the package
The end users wanting to merely use common neural networks already have various and sundry good simulators to choose from. Most likely, among these, those familiar with R may wish to be able to train their nets without having to leave the R environment. The AMORE package may be useful for that aim, but in truth these users may find other faster alternatives. But for the researchers interested in neural networks and robust modeling, those wanting to gain fine control on the operating procedures of the training algorithms, this package offers a useful platform to program their own algorithms with relative ease. The users may choose among three different degrees of involvement with the package: using the already available functions, programming their own functions playing in the easy R arena during the first steps and trials, and using the C programming language to speed up the training heavy tasks. We hope that the careful reading of the functions may serve as a way of gaining a greater understanding about the training methods already programmed, as well as an inspiration about how to expand the current possibilities. In order to ease the understanding of these readings, we will show in the following lines the main elements needed to make the whole thing work. We describe in the following sections the basic objects and their corresponding functions, using the net shown in the following figure for exemplification purposes: a multilayer feedforward network, featuring two hidden layers with four and two neurons, which considers two input variables in order to provide a bidimensional output.

The AMORE artificial neural network standard.
We have modeled the ANN using lists wherever a complex structure was needed. The preferred method to create the network is by using the newff function. For our example we would use something like: > net.start <- newff(n.neurons=c(2,4,2,2),learning.rate.global=1e-2,

1 of 11

1/14/13 1:21 PM

In our example we especified the option error. the first containing the neurons number -1 and -2.start$deltaE function (arguments) { prediction <. and deserve a deeper explanation. in the following order: layers A list of as many numerical vectors as layers are needed. The input neurons are merely virtual neurons.5. thus we should be using the Least Mean Squares cost function: > net.criterium="LMS".layer="tansig". 3 and 4 and 5. The reader may notice that the indexes of the two input neurons are negative. finally. deltaE The cost function used to measure the errors amongst the outputs and the targets.target 2 of 11 1/14/13 1:21 PM . and the last with neurons 7 and 8. it would contain the value of Input 1 and Input 2. a second hidden layer with two neurons and.arguments[[2]] residual <.neurons parameter: that is to say. hidden. The absolute value of the indexes of these input neurons gives information about which particular component of the input vector we are referring to.sciviews. Currently we provide cost functions for the Least Mean Squares. the second with neurons numbered 1.2) for the n. we have especified the value c(2. Least Mean Log Squares and TAO robust criteria. It is written in R code so as to ease its modification according to the user needs. The layers element of net. output. Stao=NA. we have four layers. In the previous line of our example.global=0.start should have the following contents: > net.2.layer="purelin". We provide it in the following section. a first hidden layer with four neurons.packages:cran:amore [R Wiki] http://rwiki. It contains the input signals to be propagated through the network. Each vector contains the indexes of the neurons that belong to that layer.php?id=packages:cran:amore momentum.arguments[[1]] target <.prediction . that we want our net to have two input neurons.start$layers [[1]] [1] -1 -2 [[2]] [1] 1 2 3 4 [[3]] [1] 5 6 [[4]] [1] 7 8 Thus. neurons A list containing the neurons. input A numerical vector that contains the values of the input variables for a single case from the data set under consideration. which are lists in themselves as well. 2. target A numerical vector that contains the expected target values towards which the network’s output must approach. an output layer with two output neurons. the third with neurons 5 and 6. method="ADAPTgdwm") The resulting net contains the following elements.4.criterium=”LMS”.org/doku. output A numerical vector that contains the resulting predictions once the input signals have been propagated. error. In our example. they are conceptual supports to get access to the different input vector components.

> net.links" [7] "weights" "bias" [9] "v0" "v1" [11] "f0" "f1" [13] "method" "method. The newff function calls the init. > net.php?id=packages:cran:amore return(residual) } As the reader may see. The custom case allows the user willing to perform the needed changes on the f0 and f1 functions to use their own ones.elements This element is a list that contains auxiliary elements.start$neurons[[1]]$id [1] 1 > net.start$neurons[[5]]$id [1] 5 > net. sigmoid and hardlim. Currently only the Stao parameter used by the TAO robust cost function is contained in this list. We recall the reader that we have considered that the input neurons do not “really” exist. It is supposed to be redundant since it should be numerically equal to the R index of the element in the neurons list. for the neurons considered: > net.start$neurons[[5]]$type [1] "hidden" > net. purelin. Not surprisingly.start$neurons[[8]]$id [1] 8 type Information about the nature of the layer this neuron belongs to: It can be either hidden or output.variables" We will spend a few lines describing the meaning of each element by means of the neurons of net. other. The neuron: the basic entity. The neuron contains the following elements in the following order: > names(net. We have chosen to represent an artificial neuron as a list whose elements contain the neuron weights. a hyperbolic tangent (tansig) or a sigmoidal (sigmoid) activation function.start$neurons[[1]]$type [1] "hidden" > net. and the necessary properties to allow the propagation of the signal from the inputs of the neuron to its output. activation function (written in R code). it is remarkably easy to modify this function so as to use a different criterium.start.start$neurons[[1]]) [1] "id" "type" [3] "activation.links" [5] "output.org/doku.function The name of the activation function that characterizes the neuron.neuron function in order to properly create the network neurons. id The index of the neuron. It allows the user to choose the neuron to be either an output neuron or a hidden one.function 3 of 11 1/14/13 1:21 PM .sciviews. or even define it as custom in case we would like to setup its activation function accordingly programming the R code of the function and its derivative.dep. Available functions are tansig.start$neurons[[1]]$activation.function" "output. bias term. to represent a linear activation function (pureline).aims" "input.start$neurons[[8]]$type [1] "output" activation.packages:cran:amore [R Wiki] http://rwiki.

start$neurons[[1]]$output.start$neurons[[8]]$activation.32384606 0.links [1] -1 -2 > net.25087121 > net.start$neurons[[8]]$input.start$neurons[[1]]$input.links [1] 1 2 3 4 > net.start$neurons[[3]]$output.function [1] "purelin" output. that is to say.07988028 bias The value of the neuron’s bias term.start$neurons[[5]]$activation. output.packages:cran:amore [R Wiki] http://rwiki.links [1] 7 8 > net. This requires the inputs to be ordered. > net.start$neurons[[1]]$bias [1] 0.links The indexes of those neurons that feed this one.aims [1] 2 input.start$neurons[[5]]$bias 4 of 11 1/14/13 1:21 PM .links.31094491 -0.links [1] 5 6 weights The weights of the conexions indicated by input. and each input has to be weighted.start$neurons[[8]]$output.aims The neuron may use the outputs of many other neurons as inputs. > net.links [1] NA Most frequently.07560576 > net. according to the output.start$neurons[[8]]$output.links [1] 5 6 > net.start$neurons[[5]]$input. > net. output neurons do not point to any other neuron and a “NA” value reflects this.function [1] "tansig" > net.start$neurons[[5]]$weights [1] 0.php?id=packages:cran:amore [1] "tansig" > net.31000780 -0.links The indexes of the neurons towards which the output of this neuron will be propagated.09150842 > net.03621645 [3] 0.start$neurons[[1]]$output.org/doku. and this element accounts for that order: the position of this neuron’s output at the subsequent neuron’s input. > net.24677257 0.aims [1] 1 1 > net.start$neurons[[5]]$output.aims [1] 3 3 > net. the indexes of the neurons that use this neuron’s output as one of their inputs. > net.links.sciviews.start$neurons[[1]]$weights [1] -0.start$neurons[[8]]$weights [1] -0.

tansig)^2)) } 5 of 11 1/14/13 1:21 PM .start$neurons[[5]]$f0 function (v) { a.start$neurons[[8]]$bias [1] -0.0.start$neurons[[8]]$v0 [1] 0 v1 It stores the last value provided by applying f1.start$neurons[[1]]$v0 [1] 0 > net.sciviews. not surprinsingly: > net.1.tansig <.0.start$neurons[[8]]$v1 [1] 0 f0 The activation function.tansig <.start$neurons[[8]]$f0 function (v) { return(v) } f1 The derivative of the activation function. while the outputs are linear.0.start$neurons[[1]]$f1 function (v) { a.71590470857554 b.tansig * tanh(v * b.start$neurons[[1]]$v1 [1] 0 > net.71590470857554 b. the neurons at the hidden layers have been defined as tansig.1.04253238 v0 It stores the last value provided by applying f0.start$neurons[[1]]$f0 function (v) { a.tansig <.start$neurons[[5]]$v0 [1] 0 > net.1.666666666666667 return(a.tansig <.packages:cran:amore [R Wiki] http://rwiki.tansig*b. thus.2522307 > net. > net.php?id=packages:cran:amore [1] -0.tansig)) } > net.71590470857554 b. Following our example. > net. In our example.tansig * tanh(v * b.666666666666667 return(a.org/doku.tansig)) } > net.666666666666667 return(a.tansig <.tansig*(1-tanh(v*b.start$neurons[[5]]$v1 [1] 0 > net. > net.tansig <.

former. It is usually set to be equal to the learning.variables Those variables specifically needed by the training method.0. .delta. BATCHgd. momentum: Similarly to the learning.change: It contains the former values of the bias change that was applied during the previous iteration. but now concerning the correction of the bias term. Their are shown in the table below.start$neurons[[1]]$method [1] "ADAPTgdwm" > net.bias: Similar to sum. > net.x.rate.tansig <.start$neurons[[8]]$method [1] "ADAPTgdwm" method..1. sum.x: Used by the batch methods to accumulate the individual error effects during the forward pass over the whole training set so as to calculate the weight’s correction later during the backward pass. this variable is usually equal to the momentum.global. it is obtained through the multiplication of v1 times the summatory of the weights times the delta value of the other neurons pointed by this one.start$neurons[[5]]$f1 function (v) { a.global. this is obtained simply by multiplying the value of deltaE times v1.666666666666667 return(a.start$neurons[[5]]$method [1] "ADAPTgdwm" > net.tansig*(1-tanh(v*b.rate: It contains the learning rate value for this particular neuron.tansig)^2)) } whilst the outputs are linear. for the output neurons.php?id=packages:cran:amore > net. while for the hidden neurons. new training methods may make use of this variable to assign different momentum rate values to each neuron.packages:cran:amore [R Wiki] http://rwiki.org/doku.delta.tansig * b.dep.rate variable.bias. but more sophisticated training methods may make use of this variable to assign different learning rate values to each neuron.sciviews. delta: This element stores the correction effect due to the derivative of the cost deltaE over the weights and the bias. former..tansig <. so > net.start$neurons[[8]]$f1 function (v) { return(1) } method The training method.71590470857554 b. ADAPTgdwm. learning.delta.change: It contains the former value of the weight change that was applied during the previous iteration. the user may choose amongst the adaptative and the batch mode of the gradient descent backpropagation training method. but again. ADAPTgd ADAPTgdwm BATCHgd BATCHgdwm 6 of 11 1/14/13 1:21 PM .weight. sum. Depending on the neuron’s type. both with or without momentum: The names of the methods are ADAPTgd. Currently.

change [1] 0 0 $former. by just substituting line 2 which reads: 7 of 11 1/14/13 1:21 PM . while editing the function.rate [1] 0.variables $delta [1] 0 $learning.bias sum.01 $momentum [1] 0.dep.rate [1] 0.php?id=packages:cran:amore delta delta delta learning.MLPnet function to simulate the response.sciviews.packages:cran:amore [R Wiki] http://rwiki.change [1] 0 Training the net Only a few functions are needed. the train function to train it.rate momentum former.change momentum former.rate [1] 0.x learning.delta. editing and customizing it so as to provide our prefered results during the training process.report function.weight.weight.change [1] 0 > net.change former.org/doku.change Training methods and method dependent variables.change [1] 0 > net. We have the newff function to correctly create the net.dep. We can alse make use of the training. for example.weight.delta.variables $delta [1] 0 $learning. and the sim.change [1] 0 0 0 0 $former.bias former.bias. fix(training.bias.01 $momentum [1] 0.report).start$neurons[[1]]$method.rate sum.01 $momentum [1] 0.change sum.delta. or even to plot some graphics.start$neurons[[5]]$method.weight.rate learning.x delta learning.rate sum.change [1] 0 0 $former.bias.bias.variables $delta [1] 0 $learning.bias. Looking at our example: > net.5 $former.start$neurons[[8]]$method.5 $former.dep.weight.5 $former. This can be done.delta.

P) plot(P.sim <. error. ''target'' is the output. 8 of 11 1/14/13 1:21 PM .criterium="LMS".newff(n. col="red".layer="tansig". pch="+") points(P. P <.neurons=c(1. ''P'' is the input data set. the desired graphical output every show. P.start.P^2 + rnorm(500. pch="+") points(P. replace=FALSE).step.shows=5 ) ## Several graphs. y <.5) ## We create the neural network object net. output.train(net. report=TRUE.sim.y. col="red".global=0.rate.5. 0. col="blue".length=500).target.P.packages:cran:amore [R Wiki] http://rwiki. momentum.MLPnet(net.php?id=packages:cran:amore P. as defined in the train function.MLPnet(net. P) for the corresponding plotting commands: P.step=100. Stao=NA.sim(result$net. mainly to remark that ## now the trained network is is an element of the resulting list.org/doku. P) plot(P.1).sim. error. result <.sciviews. pch="x") The following pictures show two resulting plots.T.sim. show. In the following paragraph we provide the commands to train a simple network: require(AMORE) ## We create two artificial data sets. 0. learning.1. dimension of the data permitting.global=1e-2. 500. ncol=1) target <.criterium="LMS".start <. target. pch="+") That change will provide. n.sim <. hidden. method="ADAPTgdwm") ## We train the network according to P and target.matrix(sample(seq(-1.layer="purelin".3.

org/doku.php?id=packages:cran:amore Epoch number 750.packages:cran:amore [R Wiki] http://rwiki. 9 of 11 1/14/13 1:21 PM .sciviews.

DPI2006-03060 and DPI2007-61090. Francisco Javier Martínez de Pisón Ascacibar.sciviews. Eliseo P. Right now. Maintainer: Manuel Castejón Limas License: GPL version 2 or newer. Fernando Alba Elías.org/doku. Alpha V. Vergara González.packages:cran:amore [R Wiki] http://rwiki.php?id=packages:cran:amore Epoch number 1000. 10 of 11 1/14/13 1:21 PM . Ordieres Meré. Ana González Marcos. and of the Autonomous Government of La Rioja for its support through the 3rd Plan Riojano de I+D+i. Released package info Version: 0. which we hope to deliver soon. Future Directions The natural development would be to provide the package with more sophisticated training methods and to speed up the training functions. while keeping the flexibility. Joaquín B. DPI-2006-02454. of the European Union through project grant RFSR-CT-2008-00034. mostly the C code. Acknowledgments The authors gratefully acknowledge the financial support of the Ministerio de Educación y Ciencia through project grants DPI2006–14784.2-11 Date: 2009-02-19 Authors: Manuel Castejón Limas. we are developing an extension of AMORE so as to provide support for RBF networks. Perní‐ a Espinoza.

es packages/cran/amore. Spain.castejon at unileon.sciviews.org/doku. Campus de Vegazana sn.txt · Last modified: 2009/05/27 11 of 11 1/14/13 1:21 PM . León. Área de Proyectos de Ingeniería. e-mail: manuel.php?id=packages:cran:amore Correspondence Author: Manuel Castejón Limas.packages:cran:amore [R Wiki] http://rwiki. Escuela de Ingenierías Industrial e Informática. Universidad de León. Castilla y León.

Sign up to vote on this title
UsefulNot useful