31YB: Biologically Inspired Computing

Autumn 2000 Examination Answer any two questions. All questions carry equal marks. The distribution of marks among the parts of each question is indicated. Exam duration: One and half hours

Q1. a) McCulloch-Pitts neurons are the simplest form of model neurons. i) describe the operation of a McCulloch-Pitts model neuron and its relationship to a real neuron [3] Answer: bookwork - weighted sum of inputs forming an activation level, which then passes through a threshold activation function and how these operations mimic dendrites, synapses, axon and cell body of a real neuron ii) describe how a single McCulloch-Pitts neuron (with either a bias input or a threshold) may be used to implement a 2-input NOR gate. [5] Answer: Problem solving (unseen) - apply e.g. 3-input McCulloch-Pitts neuron (comprising 1 bias input, and 3 weights) to solving NOR problem and show the resulting decision boundary b) Both the Perceptron Learning Rule (PLR) and the Delta Rule (DR) can solve problems which are linearly separable. i) Write down the formulae for the PLR and the DR, defining each term, and outline the main differences between the PLR and the DR. [4] Answer: bookwork - differences in error values and step size effect ii) Describe with a diagram a layered feedforward net with 3 layers, and briefly explain how the Generalised DR (also known as Back Propagation) using this network can solve decision problems which are not linearly separable. [6] Answer: bookwork - non-linear decision problems solved due to approximation of threshold activation in Perceptron with a differentiable activation-function and use of more than one (hidden) layers etc. c) Briefly explain how neural networks can be used for learning time sequences, and any one limitation you would expect to encounter in practice. [2,1] Answer: bookwork - treat time as space, problem of synchronisation, choice of input window size d) Briefly describe two types of recurrent neural-networks highlighting their differences, and one application example for each. [4] Answer: bookwork -Any two from Hopfield Net, Boltazmann Machine, Jordan Net, Elman Net - applications include, CAM, optimisation, sequence recognition, prediction Q2 Discuss in detail using a realistic example, how artificial neural networks can be used to solve real world problems. Your discussion should cover the following: reasons for using neural networks to solve your selected example problem; selection and formatting/pre-processing of neural network inputs and outputs; training,

testing and generalisation issues; choice of neural network structure; an algorithm of the neural network learning strategy proposed; and, possible problems arising in practice and ways to minimise them. [25] Answer: Transferable Skills, general problem solving, and bookwork Discuss above mentioned issues in detail Q3. a) Genetic algorithms and neural networks are both adaptive, but in different ways. Briefly describe the sense in which each type of algorithm is biologically inspired. Answer: Bookwork -GA inspired by Darwinian selection and NN by real neural networks

[4]

b) Genetic algorithms can perform optimisation in a highly complex and nonlinear space. i) Briefly describe how a Genetic Algorithm can achieve this, explaining terms such as generation, fitness, crossover and mutation. [8] Answer: problem solving(seen) -describe GA algorithm and its application to optimisation problem solving c) Research into neuromorphic systems is part of the research into the larger field of computational neuroscience. i) Give one popular definition of Neuromorphic Systems. [2] Answer: Bookwork - NS are implementations in silicon of systems whose architecture and design are based on neurobiology ii) Discuss 3 main reasons behind the recent emergence of interest in research into neuromorphic systems, and the current limitations? [6] Answer: Bookwork -discuss convergence of work in neuroscience, neural networks and chip design, and their limitations e.g. modeling real neurons with available technology (electrons and transistors based), interconnections of large no. of active elements in VLSI etc. iii) Give 2 real world examples of where Neuromorphic Systems might be applied, and the reasons which make them attractive for these applications. [2,3] Answer: Bookwork - e.g. sensing for independent robots, prosthetics, motor control etc. because Neuromorphic systems (NS) offer possibility of small low-power devices which can process real data directly i.e. data which has been sensed directly, perhaps using a transducers (also part of the chip)