Validation of A New Assessment Tool For Qualitative Research Articles

JAN JOURNAL OF ADVANCED NURSING
RESEARCH METHODOLOGY
Validation of a new assessment tool for qualitative research articles

Lone Schou, Helle Høstrup, Elin Egholm Lyngsø, Susan Larsen & Ingrid Poulsen
Accepted for publication 22 October 2011
Correspondence to L. Schou: SCHOU L., HØSTRUP H., LYNGSØ E.E., LARSEN S. & POULSEN I. (2012)
e-mail: lone.schou@frh.regionh.dk Validation of a new assessment tool for qualitative research articles. Journal of
Advanced Nursing 68(9), 2086–2094. doi: 10.1111/j.1365-2648.2011.05898.x
Lone Schou MSc RN
PhD Student
Abstract
Telemedicine Research Unit, Frederiksberg
University Hospital, Copenhagen, Denmark Aim. This paper presents the development and validation of a new assessment tool
for qualitative research articles, which could assess trustworthiness of qualitative
Helle Høstrup MSc MSA RN research articles as defined by Guba and at the same time aid clinicians in their
Quality Consultant assessment.
Department of Development and Quality, Background. There are more than 100 sets of proposals for quality criteria for
Hillerød, Denmark qualitative research. However, we are not aware of an assessment tool that is
validated and applicable, not only for researchers but also for clinicians with different
Elin Egholm Lyngsø MSc RN
levels of training and experience in reading research articles.
Head of Section
Quality and Health Data, Viborg, Denmark Method. In three phases from 2007 to 2009 we delevoped and tested such an
assessment tool called VAKS, which is the Danish acronym for Appraisal of Quali-
Susan Larsen MSc RN tative Studies. Phase 1 was to develop the tool based on a literature review and on
Clinical Development Nurse consultation with qualitative researchers. Phase 2 was an inter-rater reliability test in
Department of Hematology, Aarhus which 40 health professionals participated. Phase 3 was an inter-rater reliability test
University Hospital, Denmark
among the five authors by means of five qualitative articles.
Results. The new assessment tool was based on Guba’s four criteria for assessing the
Ingrid Poulsen DrMedSci RN
trustworthiness of qualitative inquiries. The nurses found the assessment tool simple
Head of the Research Unit
Department of Neurorehabilitation, TBI Unit to use and helpful in assessing the quality of the articles. The inter-rater agreement
Copenhagen University Hospital, Glostrup, was acceptable, but disagreement was seen for some items.
Denmark Conclusion. We have developed an assessment tool for appraisal of qualitative re-
search studies. Nurses with a range of formal education and experience in reading
research articles are able to appraise, relatively consistently, articles based on
different qualitative research designs. We hope that VAKS will be used and further
developed.
Keywords: assessment tool, evidence-based practice, instrument development,

nursing, qualitative research, trustworthiness, VAKS
(Dixon-Woods et al. 2004). Ailinger (2003) found that

Introduction
qualitative research contributed in six different areas to
Qualitative research is recognized to make important contri- evidence-based practice: hypothesis generation, development
butions to the quality of evidence-based practice (Ailinger and validation of instruments, provision of context to
2003), and to the study of quality and safety in health care evaluation, development of nursing interventions,
2086 2011 Blackwell Publishing Ltd

13652648, 2012, 9, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/j.1365-2648.2011.05898.x by Manipal Institute of Technology, Wiley Online Library on [04/12/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
JAN: RESEARCH METHODOLOGY Validation of a new assessment tool for qualitative research articles
development of new research questions and application of 1998), (2) McMaster University in 1998 (Law et al. 1998),
Qualitative Outcome Analysis. In addition, in quantitative (3) Malterud in 2001 (Malterud 2001) and (4) The Quali-
research, it is important that clinicians, managers etc. can tative Research and Health Working Group (LSTM) in 2002
have confidence in the quality of the research studies. To (Bromley et al. 2002). All these tools seem to be developed
resolve this, there have been developed many appraisal from criteria similar to quantitative research and it could be
instruments. There is, however, no consensus about what questioned whether it was more relevant to adjust an
characterizes good quality and which criteria should be used assessment tool to the premise of qualitative research. In a
to appraise qualitative research (Dixon-Woods et al. 2004). study by Henderson (2004), they developed an instrument
Scientific articles demand visible and systematic collection based on Guba¢s Model of Trustworthiness of Qualitative
of data, description of the research process, analysis and Research. Moreover, they expanded Sackett¢s Rules of
interpretation of data. Furthermore, evaluation of both Evidence Model to provide for representation of qualitative
qualitative and quantitative research articles demands that studies. The instrument and the methodology were evaluated
the reader has at least a little knowledge of methodology and by physical therapy students and they found it easy to use and
methods (Morse 2006). According to Dixon-Woods et al. helpful in clinical decision-making (Henderson 2004). Our
(2004), there are more than 100 sets of proposals for quality concerns by using Sackett¢s hierarchy system in the appraisal
criteria for qualitative research. However, we are not aware of qualitative studies are that nurses could be confused when
of an assessment tool that is validated and applicable, not using a hierarchy, mostly developed for a quantitative
only for researchers but also for clinicians with different approach. Another study by Daly et al. (2007) describes four
levels of training and experience in reading research articles. levels of a qualitative hierarchy of evidence-for-practice. The
This article presents the development and validation of hierarchy is based on the level of generalization of four
such an assessment tool, which aims at assessing strengths different qualitative research methodologies (Daly et al.
and weaknesses of qualitative research articles, and to meet 2007). This is a relevant way of thinking hierarchy; however,
the wishes of the clinicians also introduces a rating scale for some methodologies are missing and the hierarchy does not
the assessment. seem to have been evaluated in praxis.
The UK National Centre for Social Research developed in
2003 a framework to assess quality in qualitative research,
Background
drawing on existing frameworks and interviews with research-
The issue of quality in qualitative research has troubled ers in the field. To use this lengthy and thorough framework,
nursing for at least a quarter of a century (Sandelowski & knowledge and expertise in using qualitative methods are
Barroso 2002). Hope & Waterman (2003) suggest that the required (Spencer et al. 2003). The framework contains 18
reason for the lack of clarity, leading to confusion in the separate domains and it aims to aid assessment done by
application of criteria to evaluate qualitative research is that researchers and policymakers. We consider this framework
there is a need for consensus about how to decide whether a unwieldy for clinicians who might not possess the required
standard has been reached. expertise. Thus, attempts to produce consensus on criteria for
Educational research faces similar problems as addressed evaluating qualitative research have been difficult.
by Guba who attempts to assess the trustworthiness of
evidence obtained within the naturalistic inquiry paradigm.
The study
He suggests that there are four major concerns relating to
trustworthiness and that naturalistic inquiry has its own set Within the framework of the Danish Nursing Society, a
of criteria for adequacy. Guba finds it inappropriate to apply group of nurses with Masters or PhD qualification, initiated
criteria from what he calls the rationalistic paradigm, to in 2006 the development of an assessment tool for qualitative
inquiry performed within the naturalistic paradigm (Guba research articles.
1981).
Aim
Existing instruments
The aim of the study was to develop and validate a new
Over the last decade, international effort has been made to assessment tool for qualitative research articles – VAKS (the
establish assessment tools to evaluate qualitative research Danish acronym), which could assess trustworthiness of
articles. To mention some of the assessments tools: (1) qualitative research articles as defined by Guba and at the
Critical Appraisal Skills Programme (CASP assessment tool in same time aid clinicians in their assessment.
2011 Blackwell Publishing Ltd 2087

L. Schou et al.
cles (Blomqvist & Edberg 2002, Thome et al. 2004, Hughes

Methods
et al. 2007, Werntoft et al. 2007, Montgomery et al. 2009)
based on VAKS.
Developing the instrument
The assessment tool was developed and tested in three Statistical analysis
phases. The assessment tool has also been translated from
Danish into English and back translated into Danish using The validation in Phases 2 and 3 was conducted as inter-rater
standardized methods (Maneesriwongul & Dixon 2004). reliability analysed by descriptive statistics. Data are pre-
sented as median and inter-quartile range. We used SPSS
Phase 1 (Statistical Package for the Social Sciences, version 18; SPSS
Beside a literature review, we performed extensive informa- Inc., Chicago , IL, USA) for the statistical analysis.
tion retrieval and studied various recommendations of
assessment-tools. We consulted other qualitative researchers Sample
and established discussion forums to qualify our work and
Sixty health professionals recruited from across Denmark
item generation.
joined Phases 1 and 2. Most of the respondents were nurses
We presented the first assessment tool: VAKS, based on the
with Masters degree, but anthropologists and sociologists also
existing tools, at a National Nursing Conference in 2007, and
took part. Four nurses with Bachelor degree, and two with
asked the participants to help evaluate it and report their
PhD, also validated the tool. Ten per cent of the respondents
needs for other items and for a manual.
had little experience in reading scientific articles. In Phase 3,
five nurses (the authors), three with Masters Degrees, one PhD
Phase 2
student, and one with a PhD, validated the assessment tool.
In Phase 1, a content validity analysis and a face validity
analysis helped us to understand that qualitative research
must be judged on its own terms and not through a template A new assessment tool for qualitative research studies
for the evaluation of quantitative studies. Inspired by Guba To avoid pushing qualitative research into the frame of
(1981), Lincoln & Guba (1985) and Henderson (2004), four quantitative research, we based the tool on the four quality
quality criteria, credibility, transferability, dependability and criteria by Guba and added a fifth one of formal requirements.
confirmability (Table 1), were listed in the second version of Each criterion is addressed in the tool by a number of statements
VAKS. Following Guba¢s four quality criteria for the reli- describing content of quality. The quality criteria are briefly
ability and validity of a qualitative research study, we hoped introduced in the manual (http://www.dasys.dk/text.
to provide a framework for a more clear and dynamic php?id=7).
checklist. We also developed a guide and a rating scale to
determine whether the article could be recommended or not Rating scale
e.g. in a evidence-based clinical guideline. The scale is built as a Likert Scale and thus the five criteria consist
In 2008, forty health professionals validated the second of a number of questions; the criteria are weighted equally. Each
version of VAKS using a peer-reviewed scientific article question is evaluated on a four-point scale from 4 = ‘totally
(Zoffmann & Kirkevold 2009), and a professional article agree’ to 1 = ‘totally disagree’ with two points in between:
(Norlyk 2008). 3 = ‘agree’ and 2 = ‘disagree’. The scale indicates the extent to
which a criterion has been met. On each subject, a score is cal-
Phase 3 culated by adding the points for each question and dividing the
To examine the assessment tool further, five nurses (the total by the number of questions. Questions without relevance
authors) in 2009 evaluated five peer-reviewed scientific arti- for the article assessed are left out and are not included in the
calculation of results. In the example below, a total of 14 points
Table 1 Terminology on the four aspects of natural science. has been given. This is divided by the number of relevant
Criteria Term questions i.e. five. The sum score is then 14/5 = 2Æ8 (Table 2).
Truth value Credibility

Total score
Applicability Transferability
Stringency Dependability
The scoring is inspired by the AGREE collaboration, the
Neutrality Confirmability instrument used for evaluation of clinical guidelines (The

Table 2 An example of a calculation of ‘sum score’ for the criterion dependability.

Totally disagree Totally agree
Dependability 1 2 3 4 Comments and arguments
19. A logical connection X Question nr. 23 is not used in this

between data and themes study (not relevant)
developed by the researcher is described.
20. The process of analysis is described. X
21. There is a clear description of the findings. X
22. The findings are credible. X
23. Any quotation is reasonable/supporting
the interpretation.
24. There is agreement between X
the findings of the study and
the conclusions.
Sum score 14/5 = 2.8
AGREE Collaboration, 2001). The total score includes Phase 2

a choice among three options: ‘Recommended’ (‡15),
In Phase 2, an inter-rater reliability test was performed. Forty
‘Recommended with reservations’ (‡10 <15) or ‘Not
health professionals validated the second version of VAKS on
recommended’ (<10).
a peer-reviewed article (Table 3), and a professional article
(Table 4). For most questions, the respondents agreed in their
Ethical considerations assessment. Disagreement was seen for questions 7, 9, 12 and
25 with respect to the first paper (Table 3) and for questions 3,
In Denmark, only studies assessing bio-medical research must
16 and 28 for the second paper (Table 4). There were no
be reported to the ethical committee. However, this study
coincidences between the disagreements on the questions in
was approved by the hospital managers and all the respon-
the two articles. Both articles have a total score higher than 15
dents were informed of the purpose of the study verbally and
points, and were therefore ‘Recommended’ by the group.
in writing; they were also informed that participation was
voluntary and that their anonymity would be maintained.
Phase 3
Results In Phase 3, five nurses (the authors) assessed five peer-

reviewed articles using different qualitative methods. From
Phase 1 the description of ‘Total Score’, the overall evaluation of the
group is shown in Table 5. The article by Hughes et al.
The respondents found VAKS simple to use, and helpful in
(2007) is recommended with reservation (‡ 10 <15) by two
assessing the scientific findings from the article. One nurse
readers, three readers recommended the article with no
said; ‘The assessment tool ensures an overview of the article’.
reservation (‡15 points). The article by Werntoft et al. (2007)
Another nurse added; ‘The assessment tool helps me to see
was ‘recommended’ by four readers, and ‘recommended with
some other angles I wouldn’t have seen’, ‘…it gives me a
reservation’ by one reader.
systematic view of the studies’ strengths and weaknesses’.
The participants also wished us to build a manual. One
said; ‘To support the assessment process, it is important to Discussion
use a manual where the questions are given more explicit
The main result of the study is that our assessment tool seems
formulation’.
able to assess reliably qualitative research articles with
Furthermore, they suggested that we changed the questions
different designs when used by Danish nurses with a range
from ‘Yes’ and ‘No’ to a four-point rating scale, and they
of educational backgrounds.
argued for hierarchy of the qualitative studies. A colleague
A limitation of the study was the sample size in Phase 2,
suggested; ‘I could use the possibility of making an overall
where only 40 respondents completed the validation of the
recommendation. When we are using an assessment tool
assessment tool. The reliability is also difficult to investigate.
there must be a rating scale’.

L. Schou et al.
Table 3 Reliability, phase 2 (median, IQR), Zoffmann & Kirkevold (2009).

(Minimum–Maximum)
1 2 3 4 Median (IQR)
Formal requirements
1. Background of the study is described 5 15 4Æ00 (3Æ25–4Æ00)
through existing literature
2. It appears why the study is relevant 1 2 17 4Æ00 (4Æ00–4Æ00)
3. It is described how demands to informed 1 3 16 4Æ00 (4Æ00–4Æ00)
consent have been met (Helsinki Declaration)
4. It is described if there are relevant approvals 1 1 15 4Æ00 (4Æ00–4Æ00)
5. The researcher has described whether the 10 3 3 1 1Æ00 (1Æ00–2Æ50)
study can affect the informants
6. The researcher has described what will be 10 2 2 1 1Æ00 (1Æ00–2Æ00)
done if the study affects the participants
Credibility
7. The purpose is described clearly 2 2 6 10 3Æ50 (3Æ00–4Æ00)
8. The method is described 1 2 17 4Æ00 (4Æ00–4Æ00)
9. Arguments for choice of method have been made 6 3 7 4 3Æ00 (1Æ00–3Æ00)
10. The method suits the purpose 1 8 11 4Æ00 (3Æ00–4Æ00)
11. There is a description of how data were registered 5 15 4Æ00 (3Æ25–4Æ00)
12. Triangulation has been applied 3 1 4 11 4Æ00 (3Æ00–4Æ00)
13. The research process is described 1 4 14 4Æ00 (3Æ00–4Æ00)
Transferability
14. Selection of informants or sources is described 6 13 4Æ00 (3Æ00–4Æ00)
15. There is a description of the informants 2 7 11 4Æ00 (3Æ00–4Æ00)
16. It is argued why these informants are selected 1 5 14 4Æ00 (3Æ00–4Æ00)
17. The context (place and connection of research) is described 10 9 3Æ00 (3Æ00–4Æ00)
18. The relationship between the researcher(s) 8 6 5 2Æ00 (1Æ00–3Æ00)
and the context (in which the research takes
place) and the informants
Dependability
19. A logical connection between data and themes is described 1 9 10 3Æ50 (3Æ00–4Æ00)
20. The process of analysis is described 6 14 4Æ00 (3Æ00–4Æ00)
21. There is a clear description of the findings 7 13 4Æ00 (3Æ00–4Æ00)
22. The findings are credible 5 15 4Æ00 (3Æ25–4Æ00)
23. Any quotation is reasonable/supporting the interpretation 6 14 4Æ00 (3Æ00–4Æ00)
24. There is agreement between the findings of the 4 16 4Æ00 (4Æ00–4Æ00)
study and the conclusions
Confirmability
25. The researcher has described his background 7 3 5 4 2Æ00 (1Æ00–3Æ00)
and perceptions or pre-understanding
26. There are references to theorists (clear who has 1 4 13 4Æ00 (3Æ00–4Æ00)
inspired the analysis)
27. Description of whether themes were identified 4 16 4Æ00 (3Æ00–4Æ00)
from data or formulated in advance
28. It is described who conducted the study 4 9 7 3Æ00 (3Æ00–4Æ00)
29. It is described how the researcher participated in the process 2 10 8 3Æ00 (3Æ00–4Æ00)
30. The researcher has described whether his/her 9 2 4 1Æ00 (1Æ00–3Æ00)
position is important in relation to the findings
The nature of the tool did not allow a test-retest as, if the assessed the single questions differently, we did not know if it
respondents read and appraised the same qualitative article was due to differences in the understanding of the question or
twice, they would probably remember their answers from the in their opinions of the article’s ability to describe each
first test. Another limitation is that, when the respondents subject.

Table 4 Reliability, phase 2 (Median, IQR), Norlyk (2008).

(Minimum–Maximum)
1 2 3 4 Median (IQR)
Formal requirements
1. Background of the study is described through existing literature 5 15 4Æ00 (3Æ25–4Æ00)
2. It appears why the study is relevant 1 2 18 4Æ00 (4Æ00–4Æ00)
3. It is described how demands to informed consent 2 2 1 15 4Æ00 (3Æ25–4Æ00)
have been met (Helsinki Declaration)
4. It is described if there are relevant approvals 1 19 4Æ00 (4Æ00–4Æ00)
5. The researcher has described whether the study can affect the informants 16 3 1 1Æ00 (1Æ00–1Æ00)
6. The researcher has described what will be 18 2 1Æ00 (1Æ00–1Æ00)
done if the study affects the participants
Credibility
7. The purpose is described clearly 2 4 14 4Æ00 (3Æ00–4Æ00)
8. The method is described 6 14 4Æ00 (3Æ00–4Æ00)
9. Arguments for choice of method have been made 7 5 8 3Æ00 (2Æ00–4Æ00)
10. The method suits the purpose 4 16 4Æ00 (4Æ00–4Æ00)
11. There is a description of how data were registered 1 1 18 4Æ00 (4Æ00–4Æ00)
12. Triangulation has been applied 12 6 2 1Æ00 (1Æ00–2Æ00)
13. The research process is described. 1 7 12 4Æ00 (3Æ00 – 4Æ00)
Transferability
14. Selection of informants or sources is described 3 8 9 3Æ00 (3Æ00–4Æ00)
15. There is a description of the informants 3 12 5 3Æ00 (3Æ00–3Æ75)
16. It is argued why these informants are selected 3 9 8 2Æ00 (2Æ00–4Æ00)
17. The context (place and connection of research) is described 3 3 14 4Æ00 (3Æ00–4Æ00)
18. The relationship between the researcher(s) and the context 17 2 1 1Æ00 (1Æ00 – 1Æ00)
(in which the research takes place) and the informants
Dependability
19. A logical connection between data and themes is described 10 10 3Æ50 (3Æ00–4Æ00)
20. The process of analysis is described 2 18 4Æ00 (4Æ00–4Æ00)
21. There is a clear description of the findings 5 15 4Æ00 (3Æ25–4Æ00)
22. The findings are credible 5 5 10 3Æ50 (2Æ25–4Æ00)
23. Any quotation is reasonable/supporting the interpretation 1 19 4Æ00 (4Æ00–4Æ00)
24. There is agreement between the findings of the study and the conclusions 2 18 4Æ00 (4Æ00–4Æ00)
Confirmability
25. The researcher has described his background 17 2 1 1Æ00 (1Æ00–1Æ00)
and perceptions or pre-understanding
26. There are references to theorists (clear who has inspired the analysis) 2 18 4Æ00 (4Æ00–4Æ00)
27. Description of whether themes were identified 1 19 4Æ00 (4Æ00–4Æ00)
from data or formulated in advance
28. It is described who conducted the study 9 1 4 6 2Æ50 (1Æ00–4Æ00)
29. It is described how the researcher participated in the process 2 1 3 14 4Æ00 (3Æ00–4Æ00)
30. The researcher has described whether his/her position is 16 3 1 1Æ00 (1Æ00–1Æ00)
important in relation to the findings
We judge the sample to be representative because of the agree about the author’s ability to describe a subject properly,
broad level of education. perhaps because of differences in opinion about whether an
In spite of the limitations, we believe that this is the first item has been properly explained. That may also happen when
assessment tool for evaluating qualitative articles on which a professionals discuss or assess findings from quantitative
validation has been attempted. research, even if they use guiding assessment tools.
There seemed to be no questions in the reliability test, which One question has caused some difficulties in the develop-
the respondents did not understand. They did not always ment of our assessment tool. The question concerning

L. Schou et al.
Our respondents also argued for introducing a hierarchy of

What is already known about this topic the qualitative studies and they asked for the possibility of
• Qualitative research is recognized as making important producing a rating scale as the overall recommendation.
contributions to the quality of evidence-based practice. We chose not to introduce a hierarchy among our four
• Several tools for the assessment of qualitative articles quality criteria, credibility, transferability, dependability and
exist, but we are not aware of an assessment tool that is confirmability, because it might require us to define some
validated and applicable, not only for researchers but designs as inferior to others, as existing in the classical
also for clinicians with different levels of training and hierarchy introduced by Cochrane.
experience in reading research articles. With this assessment tool, we try to be methodology-
neutral to different qualitative methodological approaches.
However, the authors appraised five peer-reviewed articles
What this paper adds with different qualitative methods such as, a narrative study,
• An assessment tool has been developed and tested based a phenomenological study based on the writings of Heidegger,
on Guba’s four criteria for trustworthiness. and a hermeneutic phenomenological study as described by
• The assessment tool has been shown to be relatively van Manen (Table 5). Even though the five studies had
reliable when used by nurses and researchers in different methodologies, we did not find it difficult to use the
Denmark. assessment tool in evaluating the articles. In two of the
articles (Werntoft and Hughes), the evaluation resulted in
different categories: ‘recommended’ or ‘recommended with
Implications for practice and/or policy reservation’. However, no article was assessed to the category
• The assessment tool is recommended for use in ‘not recommended’. Indeed, even though we understand the
discussions among nurses in clinical practice. questions in our tool in the same way, because we developed
• The assessment tool is recommended when appraising them together, still we do not agree in all details.
qualitative articles for evidence-based clinical We are aware of the needs for appraisal criteria suited to
guidelines. different methodological approaches, as assessed by
Dixon-Woods et al. (2004). We recognize that we, in the
future, must focus on the distinctive study designs and
‘triangulation’ was difficult to understand for some of our
theoretical perspectives that qualitative research can adopt
respondents in Phase 1. However, in Phase 2, after the
(Dixon-Woods et al. 2004). However, due to our experience,
development of a manual, the problem seemed to be reduced.
we find that a broader assessment tool will be more practical
Our assessment tool is guiding the reader to assess the article
and easier to use for clinicians. Last but not least, it may be
in a more systematic way, but, as nurses, we still need to
noted that using a quantitative method in the evaluation of a
discuss our findings in a professional group before incorpo-
qualitative tool as we have done in this study i.e. inter-rater
rating the findings into clinical practice or guidelines.
reliability, may led to wonder and criticism. However, in our
In the first version of the assessment tool, we provided only
opinion, the inter-rater reliability can, as in other studies
‘Yes’ or ‘No’ as possible answers. After Phase 1, because our
where instruments are developed and tested, contribute to
respondents found it difficult to assess the questions in this
answer whether the tool is sufficiently robust to be used by
way, we changed the responses to a four-point rating scale.
nurses with various educational backgrounds.
Using this rating scale made the results of the evaluation less
consistent, but also more differentiated.
Conclusion
Table 5 Reliability test in phase 3.
We have developed an assessment tool for the appraisal of
Author – Methods*/Readers total score I II III IV V qualitative research studies. As far as we are aware, our study
is the first to test an assessment tool for qualitative research
Blomqvist – Typology 17 17 16 16 16
Hughes-Secondary analysis with 18 16 13 17 13 studies. We have shown that nurses with a broad range of
narrative analysis education and experience in reading research articles are able
Montgomery – Interpretive phenomenology 16 17 15 15 16 to appraise articles based on different qualitative research
Thomé – Hermeneutic phenomenology 18 18 16 17 20 designs in a relatively consistent manner.
Werntoft – Content analysis 12 15 16 15 17
However, the assessment of qualitative research articles
*The authors use the wordings design, methodology and or methods. should be discussed in a forum with specialist nurse colleagues

before the results can be implemented in clinical practice. We Guba E.G. (1981) Criteria for assessing the trustworthiness of
hope that VAKS will be used and further developed. naturalistic inquiries. Educational Communication and Technol-
ogy 29, 75–91.
Henderson R. (2004) Appraising and incorporating qualitative
Acknowledgements research in evidence-based practice. Journal of Physical Therapy
Education 2004, 1–8.
We acknowledge The Danish Nursing Society for the Hope K.W. & Waterman H.A. (2003) Praiseworthy pragmatism?
opportunity to present our assessment tool at a Danish Validity and action research Journal of Advanced Nursing 44,
nursing conference, and all the nurses who took part in the 120–127.
Hughes A., Gudmundsdottir M. & Davies B. (2007) Everyday
validation test.
struggling to survive: experience of the urban poor living with
advanced cancer. Oncology Nursing Forum, 34, 1113–1118.
Law M., Steward D., Letts L., Pollock N., Bosch J. & Westmorland
Funding
M. (1998) Guidelines for Critical Review Form - Qualitative
This research received no grants from funding agencies in the studies. McMaster University, Hamilton, ON. Retrieved from
http://www.srs-mcmaster.ca/portals/20/pdf/ebp/quanguidelines.
public, commercial, or not-for-profit sectors. We did, how-
pdf. on 21 November 2011.
ever, receive funding from the Danish Nurses’ Organization Lincoln Y.S. & Guba E.G. (1985) Naturalistic Incuiry. Sage pub-
to cover travel expenses during Phase 1. lications, Newbury Park, CA.
Malterud K. (2001) Qualitative research: standards, challenges, and
guidelines. Lancet 358, 483–488.
Conflicts of interest Maneesriwongul W. & Dixon J.K. (2004) Instrument translation
process: a methods review. Journal of Advanced Nursing 48, 175–
No conflict of interest has been declared by the authors.
186.
Montgomery P., Bailey P., Purdon S.J., Snelling S.J. & Kauppi C.
(2009) Women with postpartum depression: ‘‘my husband’’
Author contributions
stories. BMC Nursing 8, 1–14.
LS, HH, EEL, SL and IP were responsible for the study Morse J.M. (2006) Reconceptualizing qualitative evidence. Qualita-
tive Health Research 16, 415–422.
conception and design, performed the data collection and
Norlyk A. (2008) At balancere på en knivsæg. Klinisk Sygepleje 22,
were responsible for the drafting the manuscript. LS and IP 53–64.
performed the data analysis. Sandelowski M. & Barroso J. (2002) Reading qualitative studies.
International Journal of Qualitative Methods 1, 1–47. Article 5.
Spencer L., Ritchie J., Lewis J. & Dillon L. (2003) Quality in
References Qualitative Evaluation: A Framework for Assessing Research
Evidence. National Center for Social Research, London. ISBN
Ailinger R.L. (2003) Contributions of qualitative research to evi-
07715 044658 8.
dence-based practice in nursing. Revista Latino-Americana de
The AGREE Collaboration (2001)Appraisalof Guidelinesfor Research
Enfermagem 11, 275–9.
& Evaluation (AGREE) Instrument. The AGREE Collaboration,
Blomqvist K. & Edberg A.K. (2002) Living with persistent pain:
London. Retrieved from http://www.agreecollaboration.org on 21
experiences of older people receiving home care. Journal of
November 2011.
Advanced Nursing 40, 297–306.
Thome B., Esbensen B.A., Dykes A.K. & Hallberg I.R. (2004) The
Bromley H., Dockery G., Fenton C., Nhlema B., Smith H., Tolhurst
meaning of having to live with cancer in old age. European Journal
R. & Theobald S. (2002) Criteria for Evaluating Qualitative
of Cancer Care (England) 13, 399–408.
Studies.The Qualitative Research and Health Working Group,
Werntoft E., Hallberg I.R. & Edberg A.K. (2007) Older people’s
Liverpool School of Tropical Medicine, Liverpool.
reasoning about age-related prioritization in health care. Nursing
Daly J., Willis K., Small R., Green J., Welch N., Kealy M. & Hughes E.
Ethics 14, 399–412.
(2007) A hierarchy of evidence for assessing qualitative health
Zoffmann V. & Kirkevold M. (2009) Liv versus sygdom i vanskelig
research. Journal of Clinical Epidemiology 60, 43–9.
diabetesomsorg. Klinisk Sygepleje 23, 57–70.
Dixon-Woods M., Shaw R.L., Agarwal S. & Smith J.A. (2004) The
problem of appraising qualitative research. Quality & Safety in
Health Care 13, 223–225.

L. Schou et al.
The Journal of Advanced Nursing (JAN) is an international, peer-reviewed, scientific journal. JAN contributes to the advancement of
evidence-based nursing, midwifery and health care by disseminating high quality research and scholarship of contemporary relevance
and with potential to advance knowledge for practice, education, management or policy. JAN publishes research reviews, original
research reports and methodological and theoretical papers.
For further information, please visit JAN on the Wiley Online Library website: www.wileyonlinelibrary.com/journal/jan
Reasons to publish your work in JAN:
• High-impact forum: the world’s most cited nursing journal and with an Impact Factor of 1Æ540 – ranked 9th of 85 in the 2010
Thomson Reuters Journal Citation Report (Social Science – Nursing). JAN has been in the top ten every year for a decade.
• Most read nursing journal in the world: over 3 million articles downloaded online per year and accessible in over 10,000 libraries
worldwide (including over 3,500 in developing countries with free or low cost access).
• Fast and easy online submission: online submission at http://mc.manuscriptcentral.com/jan.
• Positive publishing experience: rapid double-blind peer review with constructive feedback.
• Rapid online publication in five weeks: average time from final manuscript arriving in production to online publication.
• Online Open: the option to pay to make your article freely and openly accessible to non-subscribers upon publication on Wiley
Online Library, as well as the option to deposit the article in your own or your funding agency’s preferred archive (e.g. PubMed).

Validation of A New Assessment Tool For Qualitative Research Articles

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Validation of A New Assessment Tool For Qualitative Research Articles

Uploaded by

Copyright:

Available Formats

JAN JOURNAL OF ADVANCED NURSING

Validation of a new assessment tool for qualitative research articles

Accepted for publication 22 October 2011

Keywords: assessment tool, evidence-based practice, instrument development,

(Dixon-Woods et al. 2004). Ailinger (2003) found that

2086 2011 Blackwell Publishing Ltd

2011 Blackwell Publishing Ltd 2087

cles (Blomqvist & Edberg 2002, Thome et al. 2004, Hughes

Truth value Credibility

2088 2011 Blackwell Publishing Ltd

Table 2 An example of a calculation of ‘sum score’ for the criterion dependability.

Dependability 1 2 3 4 Comments and arguments

19. A logical connection X Question nr. 23 is not used in this

AGREE Collaboration, 2001). The total score includes Phase 2

Results In Phase 3, five nurses (the authors) assessed five peer-

2011 Blackwell Publishing Ltd 2089

Table 3 Reliability, phase 2 (median, IQR), Zoffmann & Kirkevold (2009).

Totally disagree Totally agree

2090 2011 Blackwell Publishing Ltd

Table 4 Reliability, phase 2 (Median, IQR), Norlyk (2008).

Totally disagree Totally agree

2011 Blackwell Publishing Ltd 2091

Our respondents also argued for introducing a hierarchy of

2092 2011 Blackwell Publishing Ltd

2011 Blackwell Publishing Ltd 2093

Reasons to publish your work in JAN:

2094 2011 Blackwell Publishing Ltd

You might also like