Professional Documents
Culture Documents
net/publication/45277161
CITATIONS READS
9 2,916
4 authors:
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by George Brown on 03 March 2018.
Malmö University
School of Technology
Henriette Lucander
SE 205 06 Malmö
Sweden
Telephone: + 46 40 6657241
e-mail: henriette.lucander@mah.se
Keywords
SOLO-taxonomy, develop learning quality, promoting deep approach to learning
2009-09-28
ABSTRACT
surface approach to learning – is not the desired outcome for a dental student or a dentist in
understanding, develop expertise and relate information and knowledge into a coherent whole.
The aim of this study was to investigate whether the SOLO taxonomy could be used as a
model to assist and promote the dental students to develop a deep approach to learning
Malmö University, were introduced to the SOLO taxonomy and constituted the test group.
The control group consisted of 35 students participating in course eight in 2006. The effect of
assessment by using the SOLO taxonomy. The evaluators consisted of two teachers who
performed the assessment of learning outcomes independently and separately on the coded
material.
The SOLO taxonomy as a model for learning was found to improve the quality of learning.
Compared to the control group significantly more strings and structured relations between
these strings were present in the test group after the SOLO taxonomy had been introduced,
(P<0.01, one tailed test for both results). The SOLO taxonomy is recommended as a model
Norris (1) defines critical thinkers as those who seek reasons, attempt to be well-informed,
use and acknowledge credible sources, consider alternatives and other points of view,
withhold judgement until they have sufficient evidence and seek to be as precise as possible.
This paper is based on the assumption that dental education like other forms of higher
education should lead to such higher order thinking (2) – allowing students to develop deep
knowledge in their disciplines and learn to think and critically reflect on their experience.
During the last 20 years a new area of research that focuses on learning in an institutional
context has emerged (3-8). Studies have identified that students have broadly two different
approaches to learning – the surface approach and the deep approach (3-10). Schöns (11)
ideas presume that a deep approach to learning will promote the development of a reflective
practitioner. Thus, much effort has been expended, at different dental schools as well as other
medical schools, in changing curricula in order to promote a deep approach to learning (12-
16). The presence of a surface approach can be interpreted as a signal of need for change in
the teaching and learning environment towards the deep approach as defined by an intention
to seek understanding, develop expertise and relate information and knowledge into a
Biggs (17) argues persuasively that all the different factors within the learning environment,
environment that encourages students to adopt the deep approach to learning. This notion of
“Constructivism” is based on the theory that learners use their own activities to create
meaning and thereby construct their knowledge (17). “Alignment” is based on the need for the
learning activities and the assessment tasks to be aligned with the learning outcomes (17). The
University, corresponds well to the constructivist approach (18). It is widely agreed that
assessment is a strong motivator for learning (19, 20). Thus, it is essential that the assessment
is constructed in alignment with the intended learning outcomes. When striving for a deep
approach, the learning outcomes must be defined accordingly, the assessment need to
encourage such an approach – and – the performance be evaluated in accordance with these
These considerations led to the development of a “learning spiral” which illustrates students’
construction of knowledge (Figure 1). The learning spiral is a metaphor that illustrates how
breadth and depth. When learning outcomes collaborate with each other and stimulate each
other, the spiral rapidly expands upwards and outwards (21). This learning spiral was used in
The SOLO taxonomy (Structure of Observed Learning Outcome), developed by Biggs and
Collis is a way of evaluating responses of students and their performance in assessment, and
thereby evaluate the quality of learning (22). The SOLO taxonomy is a way of relating
responses to a certain level of learning quality. According to the SOLO taxonomy, the
structural complexity is increasing with increased student learning. In the early stages, most of
the learning is quantitative and the amount of detail increases. Further, the learning becomes
more qualitative as details are integrated in a structural pattern (10). The quantitative
dimension aims for an increase in knowledge while the qualitative dimension aims for a
deeper understanding: both are major curriculum aims. The taxonomy has five levels of
The deep approach to learning is a prerequisite for reaching higher levels of complexity. If the
students are to construct knowledge with increasing complexity, the ability to reflect on ones
learning is of the essence. Boulton-Lewis (23) suggests that students need to learn more about
learners. From this perspective, we thought it would be interesting to evaluate whether the
SOLO taxonomy could be used as a model for improving students´ meta-reflection and as an
effect, promote a deep approach to learning. It is acknowledged that there are other
taxonomies and other ways of assessing cognitive structure (19, 24). The SOLO taxonomy
was chosen since it makes it possible to evaluate the display of learning outcomes.
differences between the different levels of learning quality. Illustrations serve a supportive
function when constructing knowledge and solving problems (25). Other reasons for our
choice were SOLO is a general instrument applicable across modes of learning and it is not
too difficult to use. As far as we know, the SOLO taxonomy has not been used as a model for
Dental students at Malmö University must complete successfully ten full semester courses.
The Problem Based Learning curriculum at the Faculty of Odontology, Malmö University is
based on strong links between aims, objectives, learning activities, criteria used for
assessment, feedback to students and course evaluation. In the undergraduate PBL dental
The theme of the course is the “Comprehensive oral health care of children, adolescents and
adults”. In this course the practical studies and the progressing PBL are assessed in a
formative way with substantial feedback while the theoretical studies are assessed in a
summative way resulting in a grade of pass or fail. The criteria for the assessments are known
to the students. The summative assessment with tasks that are integrated or multidisciplinary
General discussions at seminars with students who participated in course eight revealed that
most students use different approaches for the formative and summative assessments,
respectively. In the formative assessment of the practical competencies the students seem to
understand the need for a deep approach to learning, for relating the different facts, i.e. strings
of details to each other and showing the ability to generalize using related aspects. In the
summative assessment of the theoretical studies however, most students seem to have a
surface approach where they focus on passing the assessment by learning how to pass the
AIM
The aim of this study was to investigate whether the SOLO taxonomy could be used as a
model to assist and promote the dental students to evolve a deep approach to learning. This
Students
The test group consisted of the 32 students who participated in course eight “Comprehensive
oral health care of children, adolescents and adults” in 2007 at the Faculty of Odontology at
Malmö University. The control group consisted of the 35 students who participated in course
eight in 2006. The groups were comparable in terms of entry requirements, gender, ethnicity
The seminars
During a seminar in the middle of the course, the students were introduced to the SOLO
taxonomy. This introduction, made by one author with experience from working with the
taxonomy in Graphic Design courses, suggested that the taxonomy could be used not only as
a method for evaluating the level of understanding that a student has reached (or at least
exhibits) but also as a framework for reflecting on one’s own learning in conjunction with
ideas of lifelong learning (19, 26). It was made clear to the students that the taxonomy was
introduced with the purpose to promote a deep approach to learning and that the assessment
criteria was to remain the same as previous years. The students were introduced to the
different levels of the taxonomy in accordance with Figure 2. Problems that arise in dental
healthcare with examples of responses that display understanding at the different levels of the
SOLO taxonomy, were discussed. After the seminar, the students performed an exercise task
in tutorial groups. The exercise involved writing responses at the Multistructural and
Relational levels (Figure 2) to a question that was relevant to the upcoming summative
assessment.
The responses were discussed at a second seminar two weeks later. The teachers provided oral
feedback on the quality of the responses and pointed out how the display of the relations in
the structural pattern seemed to enhance the level of detail and understanding. The responses
showed that the students had understood the principles of the different levels of the SOLO
taxonomy.
A question from the assessment of 2006 was randomly chosen and incorporated in the
assessment of 2007.
Mohammad, a 6-year-old boy, visits your clinic for an examination. He has lived in
Sweden for 4 months. He and his mother do not understand Swedish. Thus, it is
During the clinical (visual tactile) examination you find occlusal caries in the fissures of
the right maxillary first permanent molar [tooth 16] and the right first primary molar
[tooth 54]. He has microbiological plaque in connection with the posterior teeth in both
jaws.
besides those presented in the case, do you want to perform? Give reasons for
your answer.
b) What are your treatment strategies and your treatment plan as suggested to the
The examination responses from 2006 and 2007 were mixed and coded by an independent
person not involved in the study, thereby enabling a blinded evaluation. All responses to the
A marking protocol, in alignment with the SOLO-levels, was jointly constructed by the
analyzing group and the examination group. The analyzing group consisted of two teachers
who performed the evaluation of learning independently and separately. Neither of these
teachers was involved in the course or the assessment. One teacher was a dentist, a radiology
specialist, at the Faculty of Odontology at Malmö University. The other was a teacher at the
Department of Technology and Society at Malmö University. The latter was also the teacher
who introduced the SOLO taxonomy to the students. These two teachers did the markings,
which were compared. The few minor differences were discussed until a mutual agreement
could be reached.
The marking protocol showed relevant strings of details below each subheading and these
strings could be joined by different relations. The relations could be apparent within a group,
i.e. below the same subheading, or between groups of strings of details. An example of
One marking protocol was used for each student’s response. In this protocol the strings of
details – as well as each display of a relation between strings of details – were marked. The
The Student’s t-test was used for analyzing differences in number of strings of details, and
number of relations, between the test group and the control group. The level of significance
was set at P=0.05. For the statistical analyses we used SPSS 17.0.
RESULTS
Both results were highly significant (p<0.01) using a one tailed t-test. The test group produced
more strings of details (n = 372 for the test group and n=350 for the control group). More
relationships between strings of detail were found in the test group in comparison with the
control group (n = 117 for the test group and n = 30 for the control group). From these results
one can conclude that the test group used more strings of details and were considerably better
examination question after a short introduction to the SOLO taxonomy. Compared to the
control group, more test group students gave responses at the Relational level.
A few similar studies (27, 28) have been performed in which the SOLO taxonomy was
introduced with discussions and examples in an attempt to shape students’ learning outcomes.
knowledge were disappointing to the authors as the majority of the students remained at the
Multistructural level. The more favourable results found in this study indicate that a brief
intervention can enhance the development of deep approaches to learning in PBL. But some
caution is necessary in interpreting the results. The experimental cohort did demonstrate
deeper approaches to learning on this occasion. But this result is likely to be due to the
tendency of dental students to be compliant and their desire to succeed rather than their desire
to be deep learners. Without clear explicit assessment criteria which stress deep learning it is
unlikely that dental students will use deep approaches to learning. Put simply, if you want
dental students to become deeper learners then change the assessment criteria to include deep
learning.
However, the positive results from this study are robust. The assessment responses from the
test and control group were mixed and coded which means that the teachers who did the
marking were unaware of which group each response belonged to. The close agreement of the
results of the marking by the two teachers, despite their different backgrounds, further
indicated that the SOLO taxonomy was a useful and reliable tool for evaluating the quality of
learning.
The results may be generalisable to other dental students and other contexts but again some
caution is necessary. The results were obtained in the context of a PBL course in which
students are very familiar with notion of alignment and expect the assessments to be in line
with the learning outcomes of a course. Both cohorts took the test under examination
conditions and both cohorts had access to previous examination papers. But the test group, or
rather some individuals in the test group, might have tried the question in private study as part
of their revision. It could be argued that it was this opportunity to have a preliminary attempt
at the question not their knowledge of SOLO which produced the highly significant results.
However, students in the control group also had access to past papers and many of these
contain exactly the same questions as used in their examinations. Yet the overall performance
of the 2006 cohort was consistent with that of earlier years. What distinguished the control
group from the test group was the test group was taught SOLO.
It is interesting to note that all the students in both groups passed in the formal examinations
but, as the results show, overall the test group performed at the higher Relational level, the
control group at the Multistructural level or lower. Again, this result highlights the importance
of providing guidance to students on deep learning approaches and providing criteria which
These findings have wider implications for teaching and assessing deeper approaches to
learning. The findings indicate that the focus should be on the relationships between strings
of facts rather than the number of strings of facts taught or learnt. Fewer facts and more
connections between the facts is the path towards deep learning. The SOLO taxonomy is
helpful on this path. It can remind teachers to focus upon cognitive relationships. The
graphic output of the marking protocols illustrate clearly the results of marking can be seen in
the examples in Figure 4 and 5. Graphic output can be used in teaching as well as assessment.
The approach is likely to be appealing to dental students who, usually, prefer visualisation to
verbalisation.
As indicated in the introduction, students in both the control group and test group reported
they used surface approaches to learning for examinations. This study showed it is possible to
through the use of the SOLO taxonomy. Members of the test group were of the opinion that
SOLO should have been introduced earlier in the curriculum. This, they said, would have
raised their awareness of learning and given them a guide to develop their understanding.
The message from this study is clear. The SOLO taxonomy is a useful tool for developing
and assessing deep learning in dentistry. This result does not exclude that other
taxonomies and other ways of developing and assessing cognitive structure might
This study showed that the SOLO taxonomy could be used as a model for promoting and
1 Norris SP. Can we test validly for critical thinking. J Educ Res 1989: 18: 21-26.
2 Barnett R. The idea of higher education. Buckingham: The Society for Research
of the learner’s conception of the task. Br J Educ Psychol 1976: 76: 115-127.
1982.
implications for problem-based learning. Eur J Dent Educ, 2004: 12: 51-58
10 Biggs JB, Tang C. Teaching for quality learning at university: what the student
students and their relationship to students’ learning outcomes. Med Teach 2001: 5:
503-507
approaches to learning and studying in year two medical students. Med Educ 2007:
41: 754-762
17 Biggs J. Enhancing teaching through constructive alignment. High Educ 1996: 32:
347-364.
In: Serder V, ed. Kvalitetsarbete då och nu – med sikte på framtiden [Working with
quality then an now – aiming for the future]. Malmö: Malmö University, 2006: 63-
88.
22 Biggs JB, Collis KF. Evaluating the quality of learning: the SOLO taxonomy
(Structure of the observed learning outcome). New York: Academic Press, 1982.
learning in higher education. High Educ Res Develop 1995: 14: 143-154.
25 Lewalter D. Cognitive strategies for learning from static and dynamic visuals.
learning through assessment and evaluation. Oxford: Oxford Centre for Staff and
1
2
3
Table 1
4
5
6 The test group was compared (T-test) to the control group according to number of
7
8 strings of details as well as number of relationships between these strings of detail.
9
10
11
12 Variable Test Test Control Control T value Sig
13 group group group group level
14 Mean SD Mean SD
15 (n = 32) (n = 35)
16
17
Strings 11.63 1.93 9.99 2.03 3.38 P<0.01
18 Relationships 3.66 2.05 0.77 0.88 7.35 P<0.01
19
20 SD = standard deviation T value = T test result sig level = significance level (eg P<
Fo
21 0.01)
22
23
One tailed test used.
24
rP
25
26
27
28
29
ee
30
31
32
rR
33
34
35
36
37
ev
38
39
40
41
iew
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Fo
19
20
21
22
r
23
24
Pe
25
26
27
28
er
29
30
31
32
Re
33
34
35
36
vi
37 Figure 1. The Learning Spiral (12). The learning spiral was developed in a quality project at Malmö
38 University as a basis for discussions concerning the progression of learning outcomes and how the
39 relations among learning outcomes within Knowledge and understanding (red), Skills and abilities
ew
40 (green) and Judgement and approach (blue) interact and expand the construction of knowledge
41 upwards and outwards. Consequently, learning outcomes should be written so that they interact
42 and stimulate the progression of competence.
43 78x71mm (600 x 600 DPI)
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Official Journal of the Association for Dental Education in Europe
Page 21 of 24 European Journal of Dental Education
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Fo
19
20
21
22
r
23
24
Pe
25
26
27
28
er
29
30
31
32
Re
33
34
35
36
vi
37
38
39
ew
40
41
42
43
44
45
46
47
48 Figure 2. The Solo taxonomy (13). At the prestructural level, the answer misses the point. At the
49 unistructural level, the answer displays one string of relevant details. At the multistructural level,
the answer contains several strings of details that are unrelated to each other. At the relational
50
level, the answer shows how the different strings of details relate to each other. The relevant
51 aspects are integrated into a coherent overall structure. The final level is the extended abstract, in
52 which the answer shows the construction of knowledge and the coherent whole is generalised or
53 reconceptualised to a higher level of abstraction.
54 78x105mm (600 x 600 DPI)
55
56
57
58
59
60
Official Journal of the Association for Dental Education in Europe
European Journal of Dental Education Page 22 of 24
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Fo
19
20
21
22
r
23
24
Pe
25
26
27
28
er
29
30
31
32
Re
33
34
35
36
vi
37
38
39
ew
40
41
42
43
44
45
46
47
48 Figure 3. This example of a marking protocol shows the main groups of strings of details. Each
49 group consists of several strings as shown under the treatment plan. Each string of details that was
present was marked in the marking scheme. The relations were marked with arrows between the
50
relevant strings within or between the groups of strings.
51 70x99mm (600 x 600 DPI)
52
53
54
55
56
57
58
59
60
Official Journal of the Association for Dental Education in Europe
Page 23 of 24 European Journal of Dental Education
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Fo
19
20
21
Figure 4. Graphic representation of minimum and maximum result in a marking protocol
22 157x59mm (600 x 600 DPI)
r
23
24
Pe
25
26
27
28
er
29
30
31
32
Re
33
34
35
36
vi
37
38
39
ew
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Official Journal of the Association for Dental Education in Europe
European Journal of Dental Education Page 24 of 24
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Fo
19
20
21
Figure 5. Graphic representation of the average result of the test group and control group
22 157x59mm (600 x 600 DPI)
r
23
24
Pe
25
26
27
28
er
29
30
31
32
Re
33
34
35
36
vi
37
38
39
ew
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Official Journal of the Association for Dental Education in Europe