Professional Documents
Culture Documents
net/publication/263469084
CITATIONS READS
15 4,882
2 authors:
Some of the authors of this publication are also working on these related projects:
Project of Lithuanian Research Council "Modeling of acoustic effects of aerosol particles system by discrete element method". application registration number. MIM-
13204, the term: 2013-2015. View project
Artificial intelligence based solutions in finance market including sentiment behaviuor of investors View project
All content following this page was uploaded by Algirdas Antano Maknickas on 11 May 2016.
Abstract. Expert methods, which widely applied for human decision making, were employed for neural
networks. It was developed an exchange rates prediction and trading algorithm with using of experts in-
formation processing techniques - Delphi method and prediction compatibility. Proposed algorithm lim-
ited to eight experts. Each of experts represented recurrent neural network, Evolino-based Long Short-
Term Memory (LSTM) by using of genetic learning algorithm, EVOlution of recurrent systems with
LINear Outputs (EVOLINO). Statistical investigation of offered algorithm shows the significantly in-
crease of the reliability of prediction. Developed algorithm was applied for trading of historical forex ex-
change rates. Obtained test trading results were presented
122
N.Maknickienė, A.Maknickas
acteristics of time series and predict the TOPIX 2. Description of EVOLINO neural network
values of the next day.
It is very important to achieve stability of the fore-
The Multi-layer perceptron as well as Radial
casting, when a certain amount of unknown data
Basis Function neural network architectures were
sets in time could be found with some precision.
implemented as classifiers to forecast the closing
On the other hand, predicted data must be exam-
index price performance (Patel, Marwala 2006).
ined at first on historical data sets for evaluating
Following networks classifiers were based on a
correlation between them.
profitable trading strategy that outperforms the
One of the best known neural networks tools
long-term “Buy and hold” trading strategy which
for non- linear prediction is recurrent neural net-
was used for investigation of Dow Jones Industrial
work (Zhang, Xiao 2000) LSTM (Schmidhuber
Average, Johannesburg Stock Exchange All Share,
et al. 2006). Schmidhuber et al introduced a gen-
Nasdaq and the Nikkei Stock indices.
eral framework of sequence learning algorithm,
Authors Rai and Rai (2011) summarized the
EVOlution of recurrent systems with LINear Out-
comparison of different Neural Network types for
puts (EVOLINO) (Schmidhuber et al. 2005;
stock prediction. Despite enormous previous ef-
Schmidhuber et al. 2006). Evolino uses evolution
forts and a wide range of methods applied to this
to discover good RNN hidden node weights, while
problem, efficient stock market prediction remains
using methods such as linear regression or quad-
a difficult task mainly due to complex and varying
ratic programming to compute optimal linear
in time dependencies between factors affecting the
mappings from the hidden state to the output.
price.
When quadratic programming is used to
Exploit of RNN (Recurrent Neural Network)
maximize the margin, it is impossible to obtain the
based on genetic learning algorithm (EVOLINO)
first evolutionary recurrent support vector ma-
has a statistically significant number of successful
chines. Evolino-based Long Short-Term Memory
predictions, when parameters of RNN are properly
(LSTM) can solve tasks that Echo State nets can’t
chosen for prediction, but not without risk by in-
(Schmidhuber et al. 2005). There was introduced a
vesting (Maknickienė et al. 2011). The most popu-
new class of recurrent, truly sequential SVM-like
lar way of increasing confidence among investors
devices with internal adaptive states, trained by a
is the investment portfolio. The concepts of effec-
novel method called EVOlution of systems with
tiveness, riskiness and reliability are three corner-
KErnel-based outputs (EVOKE), an instance of
stones of possibilities investing in exchange and
the recent Evolino class of methods.
capital markets (Rutkauskas, Stasytytė 2009).
Evoke evolves recurrent neural networks to
Adequate investments portfolio seems to be theo-
detect and represent temporal dependencies while
retically sound and practically effective instrument
using quadratic programming/support vector re-
for investment decision making in global capital
gression and pseudo-inverse regression. Evoke is
and exchange markets (Rutkauskas et al 2009;
the first SVM-based mechanism which knows how
Rutkauskas 2005; Stasytytė 2008). The adequate
to classify a context-sensitive language. It also
investment portfolio anatomy and decisions using
outperforms recent state-of-the-art gradient-based
the imitative technologies (Rutkauskas 2006),
recurrent neural networks (RNNs) on various time
forecasting by artificial intelligence (Lawrence
series prediction tasks. RNN learning is used for
1997; McNelis 2005; Zhang, Xiao 1999; Rut-
context-sensitive languages recognition and is a
kauskas et al. 2010) - all forecasting models are
difficult and often increasing problem for standard
confronted with the reliability and risk reduction
recurrent neural networks (RNNs), because it re-
decision.
quires unlimited memory resources. Authors
Complex problems, which are solving by
(Goodman, Brette 2008; Schmidhuber et al. 2005;
number of competent professionals, often carried
Schmidhuber et al. 2006) found that Evolino based
out using the methods of experts. Traditional pa-
LSTM learns on average faster and it is able to
per-based on Delfi procedures, Web- based Delfi
generalize substantially better that gradient-based
process (Colton, Haatcher 2004; Hsu, Sandford
LSTM. It is possible using Evolino to learn func-
2007) can be used to answer difficult questions.
tions composed of multiple superimposed oscilla-
Aim of this paper is to use recurrent neural
tors such as double sine and triple sine. Investi-
networks based on genetic learning algorithm
gated network reached good learning and still
(EVOLINO) as the experts for using their deci-
makes very accurate predictions (Schmidhuber
sions in the trading and research profitability of
et al. 2005; Schmidhuber et al. 2006; Wierstra
this model.
et al. 2005).
The Mackey-Glass system is a standard
benchmark for non linear time series prediction.
123
APPLICATION OF NEURAL NETWORK FOR FORECASTING OF EXCHANGE RATES AND FOREX TRADING
Authors (Schmidhuber et al. 2005) show deviation The Delphi method is based on the assump-
between the curves of Evolino generated and tion that group made decision are more valid than
Mackey-glass system. Evolino is capable of mak- individual made decision. Our observations of
ing precise (0.0019) prediction in tasks like the neural network prediction shows, that some of the
Mackey-Glass benchmark. predictions are very accurate, but the others are in
The block diagram of Evolino recurrent neural a contradictory, unstable and must be rejected.
network is shown in Figure 1. Evolino RNN forms Delphi method makes it possible to achieve a cer-
LSTM network with N = 4n memory cells, where tain consensus or clustering of forecasts. Three
N is total amount of neurons and n is amount of steps were needed for applying of Delphi method
memory cells. The genetic evolution algorithm is accordingly: finding of historical orthogonal data;
applied to each quartet of memory cells separately. forming of eight data sets; calculating of medians
The cell has an internal state S together with a for- and quartiles.
get gate GF , which determines how much the state At first the upper and lower quartiles must be
is attenuated at each time step. The input gate GI determined, where value of a lower range quartile
controls access to the cell by the external inputs Q1 evaluates cuts off lowest 25 % of data and
that are summed into the ∑ unit, and the output value of upper range quartile Q3 is cuts off highest
gate GO controls when and how much the cell 25 % of data. Thus, the median and two quartiles
fires. Q1, Qavg, Q3 forms four considered the most desir-
Dark black nodes represent the multiplication able intervals. Now must be calculated compatibil-
function and the linear regression. Moore-Penrose ity of neural networks predictions, as a consensus
pseudo-inverse method used to compute the output of experts.
(light big circle). The detail description of Evolino Consequently, the Dlephy method could be
RNN algorithm could be found in (Schmidhuber continuously iterated until consensus is determined
et al. 2005; Schmidhuber et al. 2006). and achieved. In practice, the amount of iterations
is limited in decision making process.
The next stage was compatibility of neural
nets predictions. The evaluation of the perform-
ance could be considered enough reliable only if
all evaluations of experts give the good compati-
bility of responses. Therefore it is necessary to
achieve the compatibility of expert assessments.
There are two ways to obtain the compatibility.
First is variational response. In this case, the dis-
tance measured between the two events, usually
measured in min and max of interval. Second,
there are interquartile coefficients. Variational re-
sponse is the difference between the first and third
quartiles Q3 − Q1. Interquartile coefficient is the
ratio of variational response and median:
Q3 − Q1
q= , (1)
Qavg
124
N.Maknickienė, A.Maknickas
125
APPLICATION OF NEURAL NETWORK FOR FORECASTING OF EXCHANGE RATES AND FOREX TRADING
Three trading days steps were chosen to in- allowed to achieve up to 4 % profit in testing pe-
vest, because the model prediction accuracy for riod for EUR/USD cross ratios.
three days was the most. 10.000 units of condi- Investigation of trading model for historical
tional cash were invested every time with the gains data shows a consistent profit growth by using the
or losses. Three tests were performed at one time Delphi method and the calculation of the compati-
one by one, day after day. bility of forecasting of LSTM based recurrent neu-
The percentage of profit growth during the in- ral networks. The Delphi method improved
vestigation period, using three days predicting boundaries of predictions.
steps, shown in Figure 3. Furthermore, the compatibility of opinion of
The same test repeated with period of 5 trad- ANN-experts enabled elimination of absolutely
ing days, 5 tests was performed in one time, one wrong predictions. The short term investing adop-
by one, day after day. The predictions of five trad- tion of proposed model needs future investigations
ing days were more risky, but the profitability in a longer trading period with a different input
wasn’t higher. The percentage growth of profit data of exchange rates.
during the investigation period, using five days
predicting steps, shown in Figure 4. References
Choudhry, R.; Garg, K. 2008. A Hybrid Machine
Learning System for Stock Market Forecasting,
World Academy of Science, Engineering and Tech-
nology 39: 315−318.
Colton, S.; Haatcher, T. 2004. The Web-based Delphy
Research technique as a Method for Content valida-
tion on HRD and Adult Education Research, Acad-
emy of Human Resources Development Interna-
tional Conference (AHPD), 183−189.
Goodman, D.; Brette, R. 2008. Brian: a simulator for
spiking neural networks in Python. Front, Neuroin-
form 2: 5.
Fig. 3. The percentage growth of profit for 3 trading http://dx.doi.org/10.3389/neuro.11.005.2008
days steps. 1 – forecasted profit, 2 − obtained profit Hassan, M. R.; Nath, B.; Kieley, M. 2007. A fusion
(Source: made by authors) model of HMM, ANN and GA for stock market
forecasting, Expert Systems with Applications 33(1):
171−180.
http://dx.doi.org/10.1016/j.eswa.2006.04.007
Hsu, Ch.; Sandford, B. A. 2007. The Delphi Technique:
Making Sense of Consensus, Practical Assessment
Research and Evolution 12(10): 1−8.
Kim, K.; Han, I. 2000. Genetic algorithms approach to
feature discretization in artificial neural networks
for the prediction of stock price index, Expert Sys-
tems with Applications 19(2): 125–132.
http://dx.doi.org/10.1016/S0957-4174(00)00027-0
Kimoto, T.; Asakawa, K.; Yoda, M.; Takeoka, M. 1990.
Stock market prediction system with modular neural
Fig. 4. The percentage growth of profit for 5 traiding networks, International Joint Conference on San
days steps. 1 – forecasted profit, 2 − obtained profit Diego, CA, USA, 1: 1-6.
(Source: made by authors) Lawrence, R. 1997. Using Neural Network to Forecast-
ing Stock Market Prices, Department of Computer
Science, University of Manitoba working paper.
5. Conclusions Maknickienė, N.; Rutkauskas, A. V.; Maknickas, A.
2011. Investigation of financial market prediction
Neural network predictions could be treated as
by recurrent neural network, Innovative Technolo-
human like experts predictions. The same charac- gies for Science, Business and Education Vilnius:
teristics reliability and compatibility of opinion Vilnius Business College 2(11): 3−8. ISSN 2029-
could be successfully used and for ANN-experts. 1035.
Experts methods adapted to the neural networks McNelis, P. D. 2005. Neural Networks in Finance:
can improve the quality of forecasting and profit Gaining Predictive Edge in the Market, Elsevier
accordingly. The usage of proposed trading model Academic press. London. 262 p. ISBN 0-12-
485967-4.
126
N.Maknickienė, A.Maknickas
Patel, P. B.; Marwala, T. 2006. Forecasting closing for Sequence Learning in Proceedings of the 19th In-
price indices using neural networks, in Systems, ternational Joint Conference on Artificial Intelli-
Man and Cybernetics SMC '06, IEEE International gence, 466−477.
Conference 2351−2356. ISBN: 1-4244-0099-6. Sher, G. I. 2011. Evolving chart pattern sensitive neural
Rai P.; Rai K., 2011. Comparison of Stock Prediction network based forex trading agents.
Using Different Neural Network Types, Interna- 2011arXiv1111.5892S.
tional Journal of Advanced Engineering & Applica- Stasytytė, V. 2008. From two-dimensional profit-risk to
tion 1: 157−160. three-dimensional profit-reliability-risk in capital
Rutkauskas, A. V. 2005. Portfelio sprendimai valiutų markets, in EURO mini conference "Continuous op-
kursų ir kapitalo rinkose, Verslas: teorija ir praktika timization and knowledge-based technologies" (Eu-
[Business: theory and practice]. Vilnius: Technika rOPT'2008): the 20th international conference. Ner-
6(2): 107−116. ISSN 1648-0627. inga, Vilnius: Technika: 149−153.
Rutkauskas, A. V. 2006. Adekvačiojo investavimo port- ISBN 9789955282839.
felio anatomija ir sprendimai panaudojant imitacines Szeto, K. T.; Fong, L. Y. 2000. How Adaptive Agents
technologijas. Ekonomika [Economic]. Vilnius: Vil- in Stock Market Perform in the Presence of Random
niaus universiteto leidykla, 75: 52-76. ISSN 1392- News: A Genetic Algorithm Approach, in IDEAL
1258. '00 Proceedings of the Second International Confer-
Rutkauskas, A. V.; Stasytytė. V.; Stankevičienė J. 2009. ence on Intelligent Data Engineering and Auto-
Profit, riskness and reliability - three-dimensional mated Learning, Data Mining, Financial Engineer-
base for investment decisions management. Model- ing, and Intelligent Agents. Springer-Verlag Lon-
ing and Analysis of Safety and Risk in Complex don, UK 2000: 505−510. ISBN:3-540-41450-9.
Systems: in Proceedings of the Ninth International Tome, J. A. B.; Carvalho, J. P. 2005. Market index pre-
Scientific School MA SR Russia, Saint Petersburg: diction using fuzzy Boolean nets Hybrid Intelligent
SUAI, 105−110. ISBN 9785808804609. Systems, in HIS '05, Fifth International Conference.
Rutkauskas, A. V.; Stasytytė, V.; Borisova, J. 2009. ISBN: 0-7695-2457-5.
Adequate portfolio as a conceptual model of in- Wang J. H.; Leu, J. Y. 1996. Stock market trend predic-
vestment profitability, risk and reliability adjustment tion using ARIMA-based neural networks, in Neu-
to investor's interests, Ekonomika ir vadyba [Eco- ral Networks, 1996, IEEE International Conference
nomics and management] Kauno technologijos uni- on Washington DC, USA, 4: 2160−2165.
versitetas, 14: 1170−1174. ISSN 1822-6515. Wierstra, D.; Gomez, F. J.; Schmidhuber, J. 2005. Mo-
Rutkauskas, A. V.; Maknickienė, N.; Maknickas, A. deling Systems with Internal State using Evolino, in
2010. Modelling of the history and predictions of fi- Conference on genetic and evolutionary computa-
nancial market time series using Evolino, in The 6th tion GECCO, Washington, D. C., ACM Press, New
International Scientific Conference Business and York, NY, USA, 1795−1802.
Management. Selected papers. Vilnius: Technika, 1: Wong B. K., Bodnovich T. A., Selvi Y. 1997 Neural
170–175. ISSN 2029-4441. network applications in business. A review and
http://dx.doi.org/10.3846/bm.2010.024 analysis of the literature (1988-95), Decision Sup-
Schmidt, A.; Bandar, Z. 1997. A Modular Neural Net- port Systems 19: 301−320.
work Architecture with Additional Generalization http://dx.doi.org/10.1016/S0167-9236(96)00070-X
Abilities for Large Input Vectors, in Third Interna- Yamashita, T.; Hirasawa, K.; Jinglu Hu. 2005. Appli-
tional Conference on Artificial Neural Networks and cation of multi-branch neural networks to stock
Genetic Algorithms (ICANNGA 97), Nor- market prediction Neural Networks, in IJCNN '05,
wich/England. Proceedings IEEE International Joint Conference 4:
Schmidhuber, J.; Gagliolo, M.; Wierstra, D.; Gomez, F. 2544−2548. ISBN: 0-7803-9048-2.
2006. Evolino for Recurrent Support Vector Ma- Zhang, J.; Xiao, X. 2000. Predicting Chaotic Time Se-
chines, in ESANN'2006 proceedings - European ries Using Recurrent Neural Network. Chin.Phys.
Symposium on Artificial Neural Networks Bruges Lett., 17(2): 88. http://dx.doi.org/10.1088/0256-
(Belgium): 593−598. ISBN 2-930307-06-4. 307X/17/2/004
Schmidhuber, J.; Wierstra D.; Gomez, F. 2005. Evolu-
tion: Hybrid Neuroevolution / Optimal Linear Search
127