Coursework report 2012–2013

Coursework report 2012–13

CO3311 Neural networks

General remarks on Coursework assignments 1 and 2

This year’s coursework assignment involved some ‘database mining’ of a
simple dataset that is often used to try out neural networks. The aims of
the coursework assignments are many but the main ones are:
• To encourage students to read more widely than the limited amount
of material presented in the module. This is particularly useful for
students to get to know of applications – especially their details. This
is also a major way for students to know of new developments and
applications of these established techniques.
• Another aim is to have students look in depth at a couple of neural
networks which are trying to model a real set of data. This year we
have chosen the ‘Iris data set’ which has been used a great deal by
those wanting to understand neural network capabilities. The Iris
database is often used as a test for new data mining algorithms, but
we will use it to see how good (or bad) Perceptrons, Backpropagation
and Kohonen networks can be at determining the class or species of a
flower from measurements made on it.

Specific comments on assignments

Coursework assignment 1
Question 1
This concerns a review of the literature on the Iris database, with a view to
students making use of their reading in the experimental work required in
their subsequent work.
The question asks for no more than 1,000 words, so markers were looking
for a reasonable word count. Good answers will use such a count in their
planning of answers – materials in the guide need not (and should not) be
reproduced in the answers.
Full references are expected and these need to be ‘on topic’. Also, as some
websites are more authoritative than others, good websites or references
to reviewed work were sought. The articles used by good students were up
to date! Examiners found it surprising that some students were unable to
produce a good set of references together with the appropriate citations in
their text. The ability to find relevant articles and to reference correctly are
essential graduate skills.
The question asks that the search be focused on Perceptrons,
Backpropagation and Kohonen networks. Poor answers did not have this
breadth of answer.
Details of experiments and their results, as well as the conclusions made
by the authors were an important source of marks.
Another important aspect of any write up of experimental work is the
motivation for the experiment and applications that the results may

1
CO3311 Neural networks

shed light on. This ensures that readers do not get the impression that
the experiments were just random ones. Rather they contribute to the
knowledge of the topic as a whole.
There were a few answers that ignored the request that the applications be
on the Iris data; such answers attracted low marks.
Some answers gave the impression that the writers had a poor grasp of the
concept of ‘linear separability’. This concept is an important one for our
subject. Finally, good answers tended to be coherent, well presented and
very readable.
The main deficiencies included:
• Lack of technical details on the networks: type of network, its
architecture, etc.
• Lack of detail on the experiments conducted: parameter values,
number of epochs, stopping criteria, etc.
• Lack of detail of the results obtained. These would ideally include
number of replications, mean and standard deviations of errors, etc.
• Many omitted Kohonen networks.
• Some poor answers included an introduction to the neural networks
concerned – this was not asked for and with a word limit of only 1,000
words resulted in very shallow descriptions of the actual meat of the
works described.

Question 2
In this series of experiments we were told that we are looking at how well
the networks can predict the last attribute (Iris type or class) given some
of the other attributes.
For parts 2a to 2c students were asked to use Perceptrons with just one
attribute as input. One aim here was for them to see that some attributes
may be much better than others, albeit on an impossible task – one
attribute will not be enough!
In part 2d) a combination of more than one attribute is used as input. As
before some combinations are more successful than others.
We move from Perceptron to Backpropagation networks and repeat the
experiments for part d). Clearly better results should be possible with
backpropagation, as these networks have a far richer set of Input/Output
relationships that they can model.
The final part asks if all three classes can be distinguished using a
suitable Backpropagation network and clearly from the theory they can
– but marks would not have been lost had students failed to find such a
network, as long as they explained why!
Advice was given concerning how the experiments were to be conducted
and any software development that might be needed. The advice was not
always followed.
A professional attitude to the software development process and a
systematic manner of experimentation were looked for. Students were
advised that long lists of weights were not wanted – such lists were,
however, sometimes provided.
The principle that experiments should be repeatable was stated and so
markers looked for enough detail to be provided for this to be possible.

2
 Coursework report 2012–2013

It was good to see that some students made the effort to cite sources and
give the correct Harvard format for the references. This is an important
aspect of academic writing.
The use of hand calculations as a means to check spreadsheet or other
formulae was also recommended and it was good to see these in a number
of cases.

Coursework assignment 2
There were nine parts to this coursework: five in Question 1 and four in
Question 2.
Question 1 attempts to allow students to ‘see the whole picture’ in terms
of the types of neural networks that the module covers, their architecture,
parameters, algorithms and typical applications.
Question 2 allows students to go through a typical application of Kohonen
networks and (together with Coursework assignment 1) gives them some
experience of comparing different network types.

Question 1
Part a) was a straightforward bookwork exercise requiring students to list
the types of neural network that they meet in this module. Surprisingly,
not everyone managed to score well on this, as some students insisted on
omitting some types.
Examples of applications that are typically found in the literature was the
topic of part b). This aimed to encourage the student to look beyond the
subject guide materials – as the subject guide itself advises them to do.
The architecture of different networks was the subject of part c); again, it
was disappointing to see many students omitting some types of network.
Although not specifically asked for, it is hard to see how architecture can
be described without the use of a diagram!
Part d) asks for a description of the parameters of such networks and how
initial values are chosen. This is often a weakness in the knowledge that
students have in their learning of neural networks.
The final part, part e) asks for the algorithms used in the training of
the networks. Except for omissions of some types of network, this part
was generally adequately done. Some students attempted to give the
algorithms using words without using symbols or equations. In the
experience of markers the use of both is essential! Equations give a clear
brief account of the algorithms but words are needed for continuity and
motivation.
The main reasons for lost marks were omitted parts or networks and lack
of detail.
Some students produced answers that went far beyond that required in
terms of the detail and the interpretations that they offered. It was good to
see such an interest in our subject.
Conversely, many students did not organise and structure their work in a
way that helped them to notice when sections (sometimes whole network
types) were omitted.
One area of misunderstanding that was surprising was the occasional
confusion between recurrent networks and Backpropagation. Both are
forms of feedback but have fundamentally different results.
Recurrent networks feed back some outputs to earlier inputs so that the
outputs affect results on each step. In contrast, Backpropagation concerns

3
CO3311 Neural networks

feeding errors back in order to have better estimation of the required

weights.
Another issue worth mentioning is that threshold units are not normally
used with Backpropagation. For the mathematics of Backpropagation
to work as it should, the activation function needs to be differentiable
and have the derivative used in the Backpropagation formula. Of course,
one might be able to ignore the mathematics and try out any number of
activation functions and algorithms but the results are likely to be poor.

Question 2
Although structured as a question with parts, this question required many
experiments to be undertaken. Credit was given for answers that show a
professional attitude to any software development or experimentation that
was undertaken.
Much advice was given in the coursework assignment itself and in the
module’s two guides about what should and should not be included in the
writing up of such experiments. In principle, sufficient detail should have
been given to allow others to duplicate the work – this was often lacking.
Very good answers to this question acknowledged the results of
Coursework assignment 1 and compared the findings of both.
Good answers were organised in a way that made comparison of the
results of different runs easy.
Poor answers reported pages of weights and showed little evidence of their
author having understood what the results mean.
Students were advised to check some calculations ‘by hand’ using a
calculator – evidence that they had done this was often missing and marks
were deducted in this case.
The question was to some extent open ended, so full marks could be
obtained without having done everything 100 per cent correctly.
It was good to see that a number of students noticed that normalisation
was not appropriate for single input Kohonen networks – the
disappointment was that some did not.
There were a number of poor answers that gave little explanation and
little analysis of the results obtained – leaving your reader to do the work
does not result in good marks.
When undertaking an investigation as required for this coursework
assignment, it is essential that technical terms be used appropriately. One
confusion noted was that of ‘example’ (as in training example) being
confused with ‘attribute’ (as in an attribute of an example). Such confusion
can cloud the explanations given.
When reading accounts of experiments one also looks for self consistency
among the results reported. Good students would comment on (and
actually investigate) strange occurrences; for example, if four attributes
produce poorer results than two!
The most encouraging aspect of some of the best answers was the
attention the authors gave to detail. They thought clearly about the
message that they were trying to convey and how best to display the
results to achieve this.