Read without ads and support Scribd by becoming a Scribd Premium Reader.
See Premium Plans
 
Implementation of aFast Artificial Neural NetworkLibrary (fann)
Steffen Nissen
lukesky@diku.dk
October 31, 2003
Department of Computer ScienceUniversity of Copenhagen (DIKU)
Abstract
This report describes the implementation of a fast artificial neural networklibrary in ANSI C called fann. The library implements multilayer feedforwardnetworks with support for both fully connected and sparse connected net-works. Fann offers support for execution in fixed point arithmetic to allow forfast execution on systems with no floating point processor. To overcome theproblems of integer overflow, the library calculates a position of the decimalpoint after training and guarantees that integer overflow can not occur withthis decimal point.The library is designed to be fast, versatile and easy to use. Several bench-marks have been executed to test the performance of the library. The resultsshow that the fann library is significantly faster than other libraries on systemswithout a floating point processor, while the performance was comparable toother highly optimized libraries on systems with a floating point processor.
Keywords:
ANN, artificial neural network, performance engineering, fixedpoint arithmetic, ANSI C.
 
Preface
This report is written by Steffen Nissen as a graduate project on DIKU
1
. AssociateProfessor Klaus Hansen is connected to the project as supervisor.The source code for this project can be found on the internet address:http://softman.dk/˜lukesky/fann/. Furthermore the source code is also located inappendix B. The library will be releasedunder the LGPL licence [FSF, 1999] accom-panied by this report. It will be released as a SourceForge.net project [OSDN, 2003]on the internet address: http://SourceForge.net/projects/fann/, shortly after thecompletion of this report.A CD-ROM accompanies this report, when delivered as a graduate project. Theroot of the CD-ROM consist of this report as a PDF file, a small README fileand a directory called
fann
. This directory contains the entire CVS structure usedwhile creating this library and report, I will now describe the important directoriesin this CVS:
doc
Contains all the material used in the report, including some of the articlesused for writing this report.
libraries
Contains the ANN libraries and programs, which was used for the bench-marks.
src
Contains the source for the fann library and the files generated during thebenchmarks.
src/test
Contains the test and benchmark programs.
src/datasets
Contains the datasets used for the quality benchmarks.
1
DIKU: Datalogisk Institut Københavns Universitet, Department of Computer Science Univer-sity of Copenhagen
ii
 
CONTENTS CONTENTS 
Contents
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Requirements For the Library . . . . . . . . . . . . . . . . . . . . . . 11.3 The Structure of This Report . . . . . . . . . . . . . . . . . . . . . . 2
2 Neural Network Theory 3
2.1 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Artificial Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . 42.2.1 The Artificial Neuron . . . . . . . . . . . . . . . . . . . . . . 42.2.2 The Artificial Neural Network . . . . . . . . . . . . . . . . . . 52.2.3 Running Time of Executing an ANN . . . . . . . . . . . . . . 62.3 Training an ANN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.3.1 The Backpropagation Algorithm . . . . . . . . . . . . . . . . 82.3.2 Running Time of Backpropagation . . . . . . . . . . . . . . . 9
3 Analysis 10
3.1 Usage Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.2 Fixed Point Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3 Performance Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 113.3.1 Algorithmic Optimization . . . . . . . . . . . . . . . . . . . . 113.3.2 Architectural Optimization . . . . . . . . . . . . . . . . . . . 123.3.3 Cache Optimization . . . . . . . . . . . . . . . . . . . . . . . 123.3.4 Common Subexpression Elimination . . . . . . . . . . . . . . 133.3.5 In-lining of Code . . . . . . . . . . . . . . . . . . . . . . . . . 133.3.6 Specializations for Fully Connected ANNs . . . . . . . . . . . 133.3.7 Loop Unrolling . . . . . . . . . . . . . . . . . . . . . . . . . . 133.3.8 Table Lookup . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4 Design and Implementation 15
4.1 API Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.2 Architectural Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.2.1 Connection Centered Architecture . . . . . . . . . . . . . . . 154.2.2 Neuron Centered Architecture . . . . . . . . . . . . . . . . . 164.3 Algorithmic Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.3.1 The Density Algorithm . . . . . . . . . . . . . . . . . . . . . 184.3.2 The Activation Functions . . . . . . . . . . . . . . . . . . . . 194.4 Fixed Point Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.4.1 The Position of the Decimal Point . . . . . . . . . . . . . . . 204.5 Code Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5 User’s Guide 23
5.1 Installation and Test . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.2 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.2.1 The Training . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.2.2 The Execution . . . . . . . . . . . . . . . . . . . . . . . . . . 245.3 Advanced Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255.3.1 Adjusting Parameters . . . . . . . . . . . . . . . . . . . . . . 255.3.2 Network Design . . . . . . . . . . . . . . . . . . . . . . . . . . 265.3.3 Understanding the Error-value . . . . . . . . . . . . . . . . . 265.3.4 Training and Testing . . . . . . . . . . . . . . . . . . . . . . . 275.3.5 Avoid Over-fitting . . . . . . . . . . . . . . . . . . . . . . . . 285.3.6 Adjusting Parameters During Training . . . . . . . . . . . . . 285.4 Fixed Point Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285.4.1 Training a Fixed Point ANN . . . . . . . . . . . . . . . . . . 28iii
Search
Search History:
Searching...
Result 00 of 00
00 results for result for
  • p.
    • Notes
    Load more