You are on page 1of 13

The Development of a Verbal and Visual Skills Profile for use

in the Configuration of Blended Learning Objects

Trevor Barker, Asli Adisen

University of Hertfordshire, t.1.barker@herts.ac.uk

ABSTRACT
The research presented in this paper relates to the development and testing
of computer software to measure visual and verbal skills and to investigate
their relationship to Riding’s WAVI cognitive style dimensions. The context of
this work relates to the development of psychological mental and student
models to assist navigation and task performance on adaptive educational
computer systems, blended learning objects. Based upon our experimental
work, there was little evidence of correlation between the common visual
skills measured in our test and Ridings WAVI dimensions as measured by his
Cognitive Skills Analysis (CSA) test. Initial findings related to common verbal
skills also failed to find significant relationships. We discuss possible reasons
for this finding and attempted to relate it to the validity of the CSA test. We
also present our ideas on a visual skills profile that we believe is a more
useful measure of the visual abilities of learners than single numbers along
Ridings VI dimension in our work. The potential for the skills profile for use in
the configuration of adaptive learning objects is discussed.

KEYWORDS
Student Models, Computer-Based Learning, Cognitive Style, Visual Skills

INTRODUCTION

The research reported here relates to understanding how individual learners interact
with computers when as they learn (Adisen et al., 2004). We have developed in the
past, a psychological student model based on the characteristics and skills of learners
(Barker, 2006; Barker et. al. 2002a; 2002b). We have used these models to configure
adaptive multimedia and other software to assist learners. The concept of cognitive
style has been important in our models to measure verbal and visual skills of learners.
This research investigates the potential of cognitive style as a descriptor in a
psychological model of learners.

Riding has described two bipolar dimensions of cognitive style, the Wholist/Analytic
(WA) and Verbaliser/Imager (VI) dimensions. A simple computer based test, the

SOLSTICE 2007 Conference, Edge Hill University 1


Cognitive Style Analysis (CSA), has been developed by Riding that is able to classify
learners according to their position along the WA and VI dimensions. Simply stated,
Riding's WA dimension relates to whether individuals process information in wholes or
parts. The VI dimension classifies whether individuals represent information, during
thinking, in words or pictures. Riding and others have provided evidence to relate the
WAVI cognitive styles to learning processes (Riding, 1997). Our own research over the
last ten years has been concerned with the use of Riding’s cognitive style in
understanding how people learn with computers (Barker, 2006). The increasing
processing power of the computer has provided opportunity to create complex
interactive visual environments within which we can work and learn. The research
described in this paper is concerned with understanding how Riding’s WAVI dimensions
relate to the skills required for a range of complex visual activities. These include the
abilities to memorise and compare images, manipulate and analyse information held in
image form and the ability to form and hold images in the mind. These visual skills will
form part of a student model that will be used in order to configure the presentation of
information in an adaptive computer-based learning application.

VISUAL SKILLS PROFILE

An important question for our research relates to the validity of Riding’s test. We were
interested in how Riding’s WAVI dimensions were related to simple visual skills, such as
are required when students are interacting with computers to perform simple tasks like
browsing the web, navigating in virtual space or working with a graphics application. To
this end we identified a set of visual skills that we could measure simply and attempt to
relate to the WAVI dimensions. The set of skills identified are shown in table one below.
Other measures of visual ability have been included in addition to those objective skills
listed above, such as the students’ perception of their visual ability, and their reported
method used to complete visual tasks. This was because some published estimates of
verbaliser-visualiser ability are based on self reporting using introspective methods. In
our study, a visual skills test was developed to measure the skills shown in table one.
The main independent variables used in the study were the accuracy of responses to
questions in the above categories and also the time taken to respond. These are set
out in table four. Two variables measured in the study were Riding’s WA and VI scores
obtained by participants in the CSA test.

VISUAL SKILLS TEST

The Visual Skills Test was written especially for this study. In addition to recording
participant information, the test consisted of 101 questions measuring visual skills,
classified into the 14 skills classes shown in table one. To avoid any effect of fatigue,
the visual skills were distributed in small groups throughout the test. Riding’s CSA test
employed in the study was the standard computer-based test, delivered and
administered as recommended (Riding 1991a; 1991b; 1997) and described in the next
section.

SOLSTICE 2007 Conference, Edge Hill University 2


Visual Skills categories measured by the test.
Remember an image and compare with the one seen earlier
Compare an image with one present on the screen at the same time
Remember details of an Image Not Present
Ask students whether they adopt a V or I strategy - hypothetical
Measure whether students adopt a V or I strategy – actual task
Ask students whether they adopt a W or A strategy - hypothetical and actual
Analyze information contained in a graph or chart
Remember verbal instructions about an image not present and perform task
Remember verbal instructions about an image present and perform task
Remember and estimate the size of an image not present
Hold a consistent map in their mind
Rotate and manipulate an image present on the screen
Remember objects in an image not present and compare to other similar images

Table 1: Common visual skills assumed to be important in computer-based learning:


Independent variables.

Method

The sample consisted of 34 volunteers (19 male, 15 female) who were mainly university
teaching staff, postgraduate students and experienced trainers from a large
Government organisation. All participants were good English speakers with normal
eyesight. Ages ranged from 18 to 54. All data was collected online by the software and
held in files on computers. It was initially collated by software especially written for this
purpose. All timings were made automatically from within the software and were
accurate to approximately 0.1 second. Prior to the study, all testing and analysis
software, instruments and procedures were tested in a small pilot study with four expert
users. A few small problems were identified and corrected.

Procedure

The study took place on standard desktop computers in two equivalent locations, quiet
computer laboratories with controlled lighting. Prior to each test a scripted introduction
was read to each participant. After these instructions, participants were randomly
allocated take either our visual skills test followed by Riding’s CSA test, or vice versa,
with a short break in between. Additional on-screen information was provided for both
tests which participants had to read both before and during the tests. The visual skills
test took approximately 40-60 minutes to administer and Riding’s CSA test
approximately 15-20 minutes.

Results

The summary of the scores obtained in Riding’s CSA test and in both WA and VI
dimensions are shown below in table two.

SOLSTICE 2007 Conference, Edge Hill University 3


Table 2. Summary of the scores obtained in Riding’s CSA test

The scores obtained for Riding’s CSA test are mostly in line with other reported studies
(Riding 1997). To test the independence of the dimensions, Pearson’s PM correlation
was performed on the data summarized in table two. The results of this correlation are
shown in table three below.
Correlations

WA VI
WA Pearson Correlation -.061
Sig. (2-tailed) . .751
N 34
VI Pearson Correlation -.061
Sig. (2-tailed) .751 .
N 34

Table 3 Pearson’s PM correlation performed on the data summarized in table 2.

The value of the correlation coefficient (r=-0.061, N=34, p=0.75) supports the
assumption of independence between the WA and VI dimensions. Any relationships
between then can be ascribed to chance alone. The mean scores and time taken to
complete tasks in the visual skills test are summarized in table 4 below.

Data analysis

The data summarized in tables two and four was subjected to further analysis to
investigate the significance of any differences or relationships. A limitation of the data
collected in the CSA test is the fact that it is based on ratio measurements (Riding,
1998). It is difficult therefore to justify the use of parametric data analysis, as some of
the assumptions of parametric analysis are invalid with ratio data. The use of such
parametric analyses with Riding’s test is commonly reported in the literature however,
presumably on the assumption that the underlying dimensions being measured are
normally distributed and linear. We have used the SPSS software application to
perform parametric data analysis in this study, with a mind to the limitations described
above. Table five below shows the relationship between the time taken to complete the
visual skills test (TIME) and the score obtained (ACC), with the WA and VI scores
obtained in the CSA test.

SOLSTICE 2007 Conference, Edge Hill University 4


VARIABLE DESCRIPTION ( N=34) Mean (S.Dev).
Total Accuracy 51.76 (5.03)
Accuracy in remembering images 3.73 (0.70)
Time taken remembering images 10.24 (3.12)
Accuracy comparing images 7.38 (1.15)
Time taken to compare images 11.92 (4.51)
Accuracy remembering images 11.66 (2.50)
Time taken remembering images 7.44 (2.12)
Accuracy analysing graphical information 20.58 (2.82)
Time taken analysing graphical information 30.08 (8.45)
Score reporting on strategy 7.14 (1.50)
Time taken to undertake tasks requiring V or I strategy 30.38 (4.90)
Comparisons using a V or I strategy 6.82 (1.56)
Time taken making comparisons using a V or I strategy 27.85 (9.87)
Accuracy in estimating size of images 3.47 (1.81)
Time taken in estimating size of images 17.27 (4.99)
Accuracy manipulating images 6.29 (2.24)
Time taken in manipulating images 15.04 (6.56)
Accuracy answering questions requiring V or I strategy 1.41 (0.70)
Time taken answering questions requiring V or I strategy 8.96 (4.73)
Time taken in actual task used to estimate V or I strategy 40.00 (6.71)
Estimate of V or I strategy used in actual task 5.38 (3.19)
Report of W or A strategy used in WA questions 2.50 (2.23)
Accuracy answering questions requiring WA strategy 1.91 (0.51)
Time taken answering questions requiring WA strategy 6.61 (2.51)
Accuracy answering mind map questions 4.11 (3.36)
Time taken answering mind map questions 136.91 (62.53)

Table 4. Mean scores and times obtained in the visual skills test (N=34).

Variable TIME ACC WA VI


TIME Pearson
1 .202 .290 -.099
Correlation

Sig. (2- . .251 .097 .577


tailed)

ACCURACY Pearson
.202 1 .206 .143
Correlation

Sig. (2- .251 . .243 .419


tailed)

Table 5. Pearson’s PM correlation showing the correlation between the score obtained
in the visual skills test, the time taken, WA and VI scores.

SOLSTICE 2007 Conference, Edge Hill University 5


The results of this analysis show a small and insignificant positive correlation between
the WA dimension and the TIME variable (p two-tailed >0.05). No other significant
correlations were found.

Correlations WAVI & visual skills

The following correlations were obtained between some of the dependent variables and
Riding’s VI and WA dimension. One-tailed statistics were used, as it was possible in all
cases to predict a direction for the relationships found. Most values of the correlation
coefficient were small. Only those correlations that were significant (p<0.05) or of
interest (p<0.10) are given.

Verbaliser-Imager

• Time to remember an image and answer questions about parts of it (p=0.009) r =


0.44
• Ability to remember an image and answer questions about parts of it (p=0.10) r =
0.22

Wholist-Analytic

• Time to analyse graphical information (p=0.02) r = 0.35


• Time taken to manipulate an image (p=0.045) r = 0.29
• Reported strategy to solve problems (p=0.06) r = 0.27
• Ability to solve problems relying on Analytical skills (p=0.025) r = 0.33
• Mind map (time and distance related) (p=0.01) r = 0.40
• Time to remember an image and compare with the one seen earlier
(p=0.04) r = 0.3
• Ability to remember an image and compare with the one seen earlier
(p=0.08) r = -0.25
• Time to remember an image and answer questions about parts of it
(p=0.06) r = -0.26
• Ability to remember an image and answer questions about parts of it
(p=0.085) r = 0.24

Analysis of Variance (ANOVA) - WAVI groups and visual skills

An ANOVA was performed on the data summarised table four. No significant


differences were found between the performance of Verbalisers and Imagers, or
between Wholists and Analytics. To allow for the effect of those classified as
Intermediates (Riding, 1991a) the data was divided into three equal groups at
appropriate points along each of Riding’s WA and VI dimensions. Intermediate values
were removed from the data set and the remaining values were subjected to ANOVA to
test for the significance of any difference in the means of the remaining extreme WAVI
groups for all the dependent variables as before. With the middle values removed, some

SOLSTICE 2007 Conference, Edge Hill University 6


significant differences in the means were found (p<0.05). These are summarised below
along with some interesting differences that, although not significant, were approaching
significance (p>0.05 <0.10).
• Time taken to manipulate images (A greater than W p<0.05)
• Reporting a Wholist Strategy to solve image problem (W greater than A p<0.07)
• Ability to hold a mental map (A greater than W p<0.05)
• Ability to remember an image (I greater than V p<0.065)
• Time taken to compare images present (I greater than V p<0.05)
• Ability to remember images (I greater than V p<0.07)

It is interesting to note that even significant correlations were relatively small, with r in
the region of 0.3; that the direction of correlations is sometimes difficult to explain and
that many of the dependent variables relating to ability had no significant relationships
with the WA or VI dimensions or showed no significant differences in performance
between WA or VI groups. Our overall interpretation of these results is given in the
discussion section of this paper.

VERBAL SKILLS TEST

In order to relate verbal skills to Riding’s Cognitive Styles test, a similar experiment was
conducted using the set of verbal skills shown in table six below.

Answer a question related to a passage that is present on the


screen
Answer a question related to a passage that is not present on the
screen
Answer questions regarding a passage listened to
Complete the sentences with the most appropriate choice given
Find the analogies between words
Find the antonyms of the given words
Find the synonyms of the given words
Answer some questions that requires analytical skills
Remember the words that were presented on the screen
Remember the words that were listened to
Answer some questions regarding verbal instructions

Table 6. Important verbal skills required to undertake complex tasks on computers.

The results of this study are currently undergoing detailed analysis. Initial findings from
this work however, suggest that Riding’s CSA test was not useful as a measure of the
range of verbal skills identified in table five above. We were unable to find significant
differences between those classified as verbalisers and imagers in performance of
these skills, nor were we able to identify significant correlation between Riding’s
dimensions and performance measures. The implications of the lack of a significant
relationship between Riding’s WAVI dimensions and common verbal and visual skills is
discussed in the next section of this paper.

SOLSTICE 2007 Conference, Edge Hill University 7


DISCUSSION

It is claimed that those classified as Verbalisers according to the CSA test find “speech
and text easier than diagrams” and “learn best from verbal presentations”. Imagers
“learn best from visual displays” and “find pictures easier than words” (Riding, 1998).
Although Riding’s summary of the relationship between visual and verbal skills and CSA
test scores is necessarily a generalization, our results suggest that it is an over-
simplification and this claim per se. certainly is not supported by our results. Few
significant correlations and differences in performance were found. Most of those found
related to time to undertake tasks, rather than differences and relationships in visual
skill levels and the WAVI dimensions. In the research reported here we also considered
the verbal skills end of the VI dimension, initial findings from a similar verbal skills test,
producing a similar lack of evidence. It is surprising that many common visual are not
performed better by imagers when compared to verbalisers, and that we found no
relationship between skill level and scores along the VI dimension. Likewise we world
expect verbalisers to perform better at verbal tasks than those classified as imagers.
We might expected some sort of relationship, based on Riding’s claims. In the study
reported in this paper, no significant differences were found in performance between
those classified as verbalisers and imagers, when divided at the median point (1.02).
Even when intermediate scores were removed, many verbals and visual skills failed to
show significant differences between the performance of verbaliser and imager groups.
It is interesting therefore, to consider how these findings might relate to the validity of
the VI dimension of the CSA test.

There have been criticisms of the reliability of Riding’s CSA test recently (Peterson et
al.,2003a; 2003b; Coffield et al, 2004; Razaei and Katz, 2004). There has however
been little evidence to suggest that the CSA test is not a valid measure of cognitive
style. Riding suggests that a test may be valid yet not reliable (Riding, 2003). He
further states that validity is determined by the relationship of a test to objectively
observed behaviours. We have been unable to find any direct examples where
complex visual skills have been tested in the way we reported and measured against
the CSA test. The often reported correlation in the literature between learning, or
learning preference and the WAVI dimensions is too coarse a tool we argue, to support
the validity of the WAVI dimensions on their own. Learning after all, is a broad,
multidimensional and poorly understood complex concept in its own right. The reported
and observed independence of the WA and VI dimensions from themselves and from
other factors such as intelligence, although necessary to determine validity, are not
sufficient, as is claimed by Riding (2003). Other measures of validity, including
predictive validity, content validity and criterion validity might be better employed to this
end. Much of what is offered in support of the validity of the CSA test is little more than
face validity, un-supported by direct objective measures against other tools. It is of
course possible that the CSA test is valid in simple cases, but when visual tasks get
difficult and complex it is not possible to relate real-word tasks to the VI dimension
alone. There is also ample evidence from our work and from others for the interaction
of the WA and VI dimensions in real-world tasks (Kozhevnikov et al., 2003). The WA
and VI dimensions have consistently been shown to be independent in the CSA test. It

SOLSTICE 2007 Conference, Edge Hill University 8


seems likely to us that the form of the CSA test itself (VI dimension measured only
verbally, and the WA dimension measured only with images) is an important factor or
reason for this independence. If visual skills are measured by the ability to deal with
images themselves, then interaction with the WA dimension is likely to take place. In
real life tasks, the interpretation of images is a multi-dimensional problem, involving
analysis and manipulation of whole and part images. Since the CSA test was first
developed, information presentation has moved from a textual, monochrome, paper-
based, non-interactive world to the world of full colour, real-time interaction in virtual,
image based environments. Tasks involving images have got more complex and there
are many more of them. Objective measures of the VI dimension such as the CSA test,
in future may need to reflect this fact. Working with images is a complex
multidimensional skill involving imagination, colour perception and skills, ability to
remember, compare, form, analyse and manipulate two-dimensional, three-dimensional
and moving images on a computer screen, on paper and in the mind. We are currently
engaged in developing visual and verbal skills profiles that we will use to understand
how well students perform many kinds of visual and related tasks (Barker 2006). We
are making no claims as to the validity or reliability of such profiles. Indeed we expect
such profiles to change with time, due to the effects of learning and the context of their
use, unlike Riding’s dimension which are claimed to be ‘biologically wired-in’ (Riding,
1997). We are certain however, that such profiles will be more useful to us in our
research than any single measure along the VI dimension for an individual to measure
verbal and visual skills. We intend to measure, record and change these profiles as
users interact with the visual and verbal applications on their computers.

Since the Riding’s CSA Test failed to relate to the important visual and verbal skills
required to interact with complex computer applications, such as is necessary when
learning from computers, we decided to develop and investigate visual and verbal skills
profiles based on the skills identified in the verbal and visual skills tests. A factor
analysis was therefore performed on the data obtained in the visual and verbal skills
tests in order to identify important components that might be useful in a psychological
student model. Based on this factor analysis there are six visual factors and four verbal
factors that might be included in the new psychological model. Table seven and eight
shows the results of this factor analysis for visual and verbal skills respectively.

SOLSTICE 2007 Conference, Edge Hill University 9


Variable Component
1 2 3 4 5 6
Remember an image and .546
compare with the one seen
earlier
Compare an image with one .608
present on the screen at the
same time
Remember the features of an
image that is not present on
the screen and answer some
questions about it.
Analyse the information .721
presented with a graphical
format
Ability to estimate a size of .659
an image that is not present
on screen
Ability to manipulate an .661
image
Self reporting on VI -.727
Strategy used for VI .563
questions
Self reporting on WA .809
Number of items recalled in a .664
photograph
Number of removed items .546
recalled in a photograph
Noticing the items removed .590
from a photograph

Table 7: Component Matrix relating to the skills considered in Visual Skill Test

Six important factors were identified for the visual skills profile which are as follows:
• Factor 1= Analysing the information given in image or graph format
• Factor 2= Predicting information based on an existing image
• Factor 3= Self reporting on WA dimension
• Factor 4= Recalling information about information seen earlier
• Factor 5, 6= Self reporting on VI dimension

SOLSTICE 2007 Conference, Edge Hill University 10


Variable Component
1 2 3 4
Answering questions about a passage .667 .818
read
Answering questions about a passage .760
listened to
Completing sentences .690
Finding the synonym of a given word .805
Finding the antonym of a given word .877
Answering analytical skill questions .637

Recalling the words read .706


Recalling the words listened to .742

Table 8: Component Matrix relating to the skills considered in Verbal Skill Test

Four important factors were identified for the verbal skills profile which are as follows:
• Factor 1= Finding the synonyms and antonyms of the given word
• Factor 2= Analytical skills required to recall information
• Factor 3= Completing sentences
• Factor 4= Ability to recall the information heard

More work will take place in future on the identification and testing of verbal and visual
skills factors, as these are likely to be important components of the psychological
student model. Other psychological descriptors that could be considered for inclusion in
the new psychological student model for configuring blended learning objects are
Riding’s WA dimension. This is because this dimension has been shown in our
experiments to be important and related to the performance of students undertaking
complex tasks. The other reason for potentially including this dimension in the model
relates to the findings of the other studies reported in literature such that this dimension
showed many correlations and differences in different approaches students take to
interact with different computer applications. The two other features that will be included
in the new student model are students’ language level which is an important indicator in
terms of how well they can understand the textual information and their domain
knowledge which is a good indicator of their existing mental model of the application.

It is important for our research to capture valid, simple and accurate measurements of
verbal and visual skills. It is expected that the factors identified for the visual and verbal
skills profiles, a learner’s language ability and the student’s domain based mental model
will change with time, due to the effects of learning and the context of their use. We
intend to measure, record and change these profiles as students interact with the visual
and verbal applications on their computers.

In summary, the reliability of any of the measures employed to obtain those features
used in our student model are far less important for this study than validity of those
features. This may seem strange at first, but it is intended to adapt presentation of

SOLSTICE 2007 Conference, Edge Hill University 11


visual and verbal information in learning objects based on performance on a computer.
For this purpose the reliability of the test and re-test measures of these features are not
important at all. The most important condition for the new student model is simple and
accurate initial measurements that relate accurately to the important skills under
consideration. To this end it is thought the visual and verbal skills profiles developed will
be far more useful than a single value measure (such as Riding’s CSA measure) which
tries to summarise all the skills in one or two measurements. The drawback of using a
single measurement for multiple skills is the difficulty of deciding the relation between
different skills when a change in the value to occurs. In the new visual and verbal skills
profiles there are no such problems as different skills are represented in different
factors.

REFERENCES

Adisen, A, Barker, T. & Britton, C. (2004) Investigating the potential of mental models in
adaptive user modelling, Proceedings of HCI 2004: Design for Life, Leeds Metropolitan
University, September 6-10. Volume 2, 161-163.

Barker T, Jones S, Britton C and Messer D J, (2000b), The use of the Verbaliser-Imager
cognitive style as a descriptor in a student model of learner characteristics in a
multimedia application, Proceedings of 5th Annual Conference of the European
Learning Styles Information Network, (ELSIN) University of Hertfordshire June 26 & 27,
2000,

Barker T, Jones S, Britton C and Messer D J, (2002a), The use of a co-operative


student model of learner characteristics to configure a multimedia application, User
Modelling and User Adapted Interaction, 12(2-3), 207-241

Barker, T. 2006, Attending to Individual Students: How student modelling can be used
in designing personalised blended learning objects. Journal for the Enhancement of
Learning and Teaching, ISSN 1743-3932, Vol. 3 (2) pp 38-48

Coffield F., Moseley, D., Hall, E and Ecclestone, K. (2004), Learning Styles and
Pedagogy in Post 16 Learning: A systematic and critical review. Learning Skills and
Research Centre.

Kozhevnikov, M, Hegarty, M and Mayer, R.E, (2003), Revising the Visualizer–Verbalizer


Dimension: Evidence for Two Types of Visualizers, Cognition and Instruction 20 (1), 44-
77

Peterson, E. R, Deary, I, J and Austin, E.J (2003a), The reliability of Riding’s Cognitive
Style Analysis test, , Personality and Individual Differences 34, 881-891

Peterson, E. R, Deary, I, J and Austin, E.J (2003b), On the assessment of cognitive


style: four red herrings, Personality and Individual Differences 34, 899-904

SOLSTICE 2007 Conference, Edge Hill University 12


Razaei A, R and Katz L. (2004), Evaluation of the validity and reliability of the cognitive
styles analysis. Personality and Individual Differences 36, 1317-1327

Riding R, (2003), On the assessment of cognitive style: a commentary on Peterson,


Deary and Austin, Personality and Individual Differences 34

Riding, R. (1991a), Cognitive Style Analysis, Birmingham Learning and Training


Technology.

Riding, R. (1991b) Cognitive Style Analysis, Users’ Manual, Birmingham Learning and
Training Technology.

Riding, R. (1997), “On the Nature of Cognitive Style, Learning Styles and Strategies”,
Educational Psychology 17(1/2), 29-49.

Riding, R. (1998), Cognitive Style Analysis, Research Applications, Birmingham


Learning and Training Technology.

SOLSTICE 2007 Conference, Edge Hill University 13