You are on page 1of 15






NO .

3, 215 228

Going the distance: closeness in

qualitative data analysis software
(Received 3 October 2001; accepted 18 March 2002)
This study about researchers transitions to using qualitative data analysis (QDA) software
identified three stages of closeness to the data: the tactile-digital divide; the coding trap;
and the metacognitive shift. The tactile-digital divide involves adapting to working on
screen instead of paper, an initially distancing process. As users gain comfort with the
software, they experience the coding trapan issue of too much closeness to the data.
Users warned that there was a tendency to become bogged down in coding, and developed
strategies to provide analytical distance. The metacognitive shift involves learning to think
about software processes with the same level of reflectivity that should accompany
qualitative research processes in general, including developing strategies for error
recognition. These transitions invite reflections on the nature of cognitive tools and
expertise with them, which lead to implications for evaluating research and considering

Concerns about losing closeness to the data pervade conversations about
qualitative data analysis (QDA) software (Agar 1991, Mangabeira 1996,
Weitzman and Miles 1995). In their 1998 report on focus groups conducted
with QDA users, Fielding and Lee (1998) attempted to examine these
concerns in more detail, and began to differentiate between aspects of
One form of closeness involved a living knowledge of the content:
being able to recover the sights, sounds, and experiences of being in the
field (Fielding and Lee 1998: 74). However, they observed, sometimes,
when researchers refer to being close to their data, what they have in mind
are the tactile and perceptual aspects associated with data handling
(Fielding and Lee 1998: 75). Closeness was essentially equated with
positive sensations associated with handling fieldnotes, and distance
equated with the discomfort associated with the limitations of early
software screens. Thus, the term closeness to data conflates two different
constructs: knowledge of content and pleasure in handling data.
Lyn Richards, one of the developers of NUD*IST, also recently examined
the concept of closeness to the data (Richards 1998). She identifies aspects
Linda S. Gilbert, BFA, MA, PhD. Lindas research interests involve the use of computers for higherlevel creative and intellectual tasks. She is also interested in research methodology. She is currently
employed at the University of Georgia College of Education as an evaluator for two grant-funded
projects, and can be reached at: or through the web at http://www.arches.*gilbertl/
International Journal of Social Research Methodolog y
ISSN 1364-5579 print/ISSN 1464-530 0 online # 2002 Taylor & Francis Ltd
DOI: 10.1080 /1364557021014627 6



of closeness as full knowledge of the content and full access to the data
itself. She further differentiates between closeness to the original data and
closeness to contextual information about those data. According to Richards,
access to the original text is strongly supported by most QDA software,
including NUD*IST. However, such access represents an unrecognized skew
in methods: previously, most qualitative methodologies emphasize[d] the
creation of interpretive records from which to work (p. 322).
Richards also makes a critical observation: as much as qualitative
researchers openly value closeness to the data, they also need distance from
the data. Qualitative research requires an in-out process: researchers have
to achieve and manage both ways of zooming in and ways of achieving a
wide-angle view (Richards 1998: 324). She argues that closeness promotes
familiarity and appreciation for subtle differences, but distance allows
abstraction and synthesis.
Distance issues were also apparent in a recent study I conducted on
qualitative researchers who transitioned from manual practices to using
NUD*IST (Gilbert 1999). The purpose of the study was to describe how
individual qualitative researchers perceive that their research procedures
and perspectives have been influenced by the adoption of computerassisted qualitative data analysis software (QDA software). The primary
source of data was in-depth interviews with qualitative researchers who had
experience conducting qualitative analyses both manually and with
NUD*IST software. (NUD*IST was selected because of its long history of
development and its wide distribution. Participants in this study were using
versions 3 and 4 of NUD*IST.) Participants compared their two working
methods and reflected on the process of transition between them. The data
were analysed through individual case profiles and cross-case comparisons,
both informed by phenomenological perspectives. The study was further
informed by activity theory, a socio-cultural theoretical perspective that
regards individuals and tools as mutually influencing one another.
In that study, I saw three levels of distance issues, which seemed to
surface somewhat sequentially as a user became familiar with the program.
The first related to the perceptual aspects of handling data, as noted by
Fielding and Lee (Fielding and Lee 1998). The second related to closeness
to the data, in terms of knowledge and access, similar to that discussed by
Richards (Richards 1998). However, a third aspect of distance has
developed in conjunction with the use of QDA programs. This form of
distance relates to understanding and monitoring operations on the data
performed with the assistance of these programs, and requires users to
extend their metacognitive awareness to software processes as well as their
own cognitive processes. In this paper, I will discuss the first two stages of
distance before focusing on the third stage and its implications. As in the
original study, all names used in this paper are pseudonyms.
Stage 1: the tactile-digital divide
The tactile-digital divide involves adapting to working on screen instead of
paper. This transition can be difficult, and I suspect that some people never



learn NUD*IST (or other programs) in part because they find this stage too
uncomfortable. A number of the participants in this study compared
learning to work with data on screen to learning to write on the computer:
initially, they found the process cumbersome, but gradually adjusted.
Several of the participants described how they learned to work on the
computer. Betty coded on the hard copy at first, then entered the coding
into NUD*IST in a second step; later, as she became more comfortable with
the program, she read from the hard copy but coded directly on screen.
Both Frances and Diane recalled that they didnt feel close to their data at
first, then got used to working with data in the program. Frances
acknowledged that when she analysed her early feelings, she realized that
she simply felt a greater sense of ownership about the data when it was on
paper. I, Ive sort of had to think through what it is about this closeness to
the data, the ownership of the, of the data is about. . . At this point, she
said I can feel just as close to the data on the screen, as I can, um, on the
paper, it doesnt, you know, it doesnt have the same distancing effect on
me. But it did for a little while. Some of that was due to the difference
between pages that could be spread out, and scrolling back through text.
Now, she actually finds it easier to link data on the computer.
For users who persevere, the tactile-digital divide seems a temporary
period of discomfort, followed by a synthesis of paper and computer
methods. Most participants eventually established a combination of
working on paper and working on the screen that they found comfortable,
mingling paper-based and program-based coding. Alice said there come
times when I have to print it out...I cant work solely in the computer.
Diane also printed out sections to code on paper, though she seemed to feel
that she was unusual in doing so: And I know other people do everything
on the screen, but I, I dont. Based on her experience as a consultant and
trainer, one participant theorized that coding on paper and then entering
the codes was a transitional practice that faded as users became
accustomed to working on screen, but the practice seemed persistent
among the participants in this study. This finding corresponds to Fielding
and Lees (1998) observation that most of their participants continued to
examine their data on hard copy as well as on screen.
I have two theories concerning the tactile-digital divide. First, it may be
exacerbated by the mechanics of handling the text. NUD*IST 4, like other
QDA programs of its generation, required that the data be formatted in
plain text (ASCII). This format does not permit the use of bold, italics,
colour, or even variations in font or text size. Researchers already
accustomed to using such text characteristics to enhance meaning (for
example, indicating changes in the speakers intonation) may find it
especially difficult to adjust to working with their data with such programs.
Initial data preparation requirements also seem to be a barrier to new users.
The second theory is that a genuine need for different analytical
strategies may be conflated with tactile preferences, particularly for new
users. The participants sometimes worked offline because they wanted
analytical strategies that they didnt see in the software. In some cases, the
strategies were not sufficiently supported by the current version of the
program; in others, the users simply didnt recognize the softwares



capabilities. Chris, who considered coding on paper a transitional

strategy, herself used some non-computer analytical strategies. Diane
liked to print because for some things, I want to put them together in ways
that I cant put them together, the way NUD*IST gives them to me. In my
own experience, I worked on paper at times because NUD*IST does not
support concept maps for data exploration, and I did not own the conceptmapping software it exported to. However, Frances observed that her team
members labouriously created data representations that NUD*IST could
have generated directly from the coding, because they were unaware of its
capabilities to create tables and matrixes. Thus, the temporary discomfort
of working on-screen could easily be confused with real or perceived
limitations of the software, prompting some users to conclude I just cant
work that way before theyve given themselves sufficient time to adapt.
Since my study involved users who did learn to work on screen, both these
theories are preliminary.
Stage 2: the coding trap
The second aspect of distance from the data is the coding trap.
Surprisingly, the issue involves too much closeness to the data, not too
much distance. It seems to become noticeable after the user has developed
some facility with the basics of the program, after overcoming the tactiledigital divide.
As Fielding and Lee (1998) noted, though non-users express concern
about a distancing effect, QDA users actually feel that the software makes
them more knowledgeable about their data. This general feeling of
closeness was confirmed in this study: participants felt NUD*ISTs ability
to retrieve both data and context was invaluable, and fostered familiarity
with the data. In fact, capabilities like jump to source were seen as
promoting a level of closeness difficult to achieve manually. (Jump to
source allows the user to view a coded excerpt within its original context.)
Even newer users like Betty felt that non-users worries about distance
from the data were completely unjustified. She found closeness to the data
to be an advantage of the software: the beauty of it, I think, is that it lets
you be closer... have access to where your findings came from.
However, some participants specifically identified the need to balance
closeness to and distance from data. Alice acknowledged you can get
close to the data, and NUD*IST gives you many, many, many ways to do
that, but getting away from it (pause) to abstract up (pause) I have to print.
Chris observed I know people talk about being close to the data, um
(pause) but there are times when you need to be away from the data... And
then all of a sudden it, it comes together.
Participants felt they were able to analyse their data more thoroughly
using NUD*IST, in part because they could access it more readily than in
manual methods. At the same time, except for the two beginners, all the
users warned that there was a tendency to become bogged down in coding
with NUD*IST. Most of them didnt offer specific reasons for this
phenomenon, just theres something about coding in NUD*IST... One



strongly related concern was that coding could become mechanical or

unthinking. According to Chris, if youre just in the browser and youre
coding....I think its very easy to lose sight of where youre going, what
youre analysing. Others offered similar warnings.
Another related topic involved the search tools. As several participants
observed, the index search tools in NUD*IST depend upon the underlying
codes. If the user wants to take advantage of these advanced features, finegrained coding is necessary. Since NUD*IST manages multiple codes, all
potentially relevant distinctions can be tracked. From using NUD*IST
myself, I can see that it would be easy to fall into the unconscious
assumption that because NUD*IST can manage complex coding systems
and can track demographics and other information that might be relevant,
the user should track all the information available.
Extensive coding can also provide an excuse to delay other steps of the
analysis process. Gwen commented that one could go on doing something
with NUD*IST to avoid making decisions about what your datas saying.
Individuals who are uncertain about the processes of qualitative research
may be especially prone to such a problem. Alternatively, the program
interface itself may somehow over-emphasize the coding process: Diane, a
highly-experienced qualitative researcher, initially fell into the coding trap
despite her well-established manual coding methods. Of course, these two
theories are not mutually exclusive.
For the most part, participants became highly aware of the tendency to
get sucked in to coding. Though they did not necessarily define the coding
trap as an aspect of too much closeness to the data, the strategies they
developed to mitigate the problem generally provided analytical distance.
Gwen simply noticed that NUD*IST created an expectation of thorough
coding, and deliberately chose not to code at that level. Other strategies
included: (1) alternating working on the computer with working on paper;
(2) reflecting on the index tree categories; (3) writing memos; (4)
maintaining a focus on the research questions; (5) coding systematically
for specific research-related themes; (6) using other software programs with
different strengths and weaknesses; and (7) using complementary manual
methods. Thus, the majority of the participants in this study found ways to
compensate for the coding trap, just as they managed to negotiate the
tactile-digital divide.
Though coding trap may be a manifestation of too much closeness
that is particularly apparent in computer-assisted analysis, it highlights a
general issue that pervades qualitative research. General texts on analysis
address the importance of distance as well as closeness. One advises ...keep
drawing back in order to think about the total picture. Descend into detail,
to be sure, but balance that descent with self-conscious efforts to perceive a
general design, overall structure... (Lofland and Lofland 1995: 203;
emphasis in the original). Bogdan and Biklen (1992), urging researchers to
expand their analytic horizons, are even more emphatic: Nearsightedness
plagues most research. We get involved... and become so captured by the
particulars, the details, that we cannot make connections... (p. 162).
Closeness to the data has been prized in qualitative research in part
because it was difficult to achieve with manual methods. Now that



closeness is so much more achievable, the need for a broader, balancing

perspective becomes more apparent.
Stage 3: the metacognitive shift
The final stage of distance lies at a different level from the first two. It
involves learning to think about software processes with the same degree of
reflectivity that should accompany qualitative research processes in
general. In this stage, metacognitive processes, which are concerned with
the regulation of thinking and learning, extend to include software use.
Though beginners often displayed aspects of this form of awareness, it
seemed to develop in conjunction with expertise with the program.
Self-monitoring is one aspect of metacognition: thinking about how and
why one works in a particular way. For example, when users consciously
recognized the coding trap and developed strategies for dealing with it, they
displayed a metacognitive awareness of their working patterns. Experienced users developed a highly reflective attitude toward software use that
they strove to impart to novices.
Error recognition demonstrates another aspect of metacognition. The
users with the greatest level of expertise were also most aware of the
potential to make mistakes with NUD* IST. Frances described the
possibility of setting up a complex search without realizing that this is
not what I really want to ask. Chris ran checks to make sure that the codes
in her command files had been properly applied. These experienced users
actively looked for indications that their intentions and results correspondedanalytical feedback that assured them that they had accomplished their goals.
These participants obviously sought to avoid constructing a search
incorrectly, or unwittingly miscoding data. I have characterized these types
of potential errors as unmindful transformations: that is, doing something
with the data, but being unaware that the results were not what was
intended. Though these users developed strategies to confirm their results,
inexperienced or unaware users could conceivably make mistakes they
never even recognized as such, distancing them from their data in a
profound way. In processes such as applying codes with a command file,
small errors can have broad effects. I consider unmindful transformations
to be simultaneously a consequence of NUD*ISTs power, and the most
worrisome aspect of distance from the data.
A personal experience reinforced my own awareness of unmindful
transformations. I used a command file in NUD*IST to apply automatic
coding to separate the interview questions and to code documents
according to document type. At one point, I realized that the number of
documents coded to interviews was incorrect: there should have been three
more. After investigating further, I found that a one-digit typo in my
command file had miscoded the last three documents entered. Once I
discovered the error, it was easy to correct. But I found the implications of
that mistake terrifying: if I had not caught it, searches through interviews
would have missed all the data in those documents. After that experience,



the twin questions what exactly am I trying to do? and did that do what I
wanted? bracketed my every use of the program.
Though I see unmindful transformations as the most critical form of
distance from data, I do not see this problem as unique to either NUD*IST
or to qualitative data analysis programs in general. Users of most complex
computer programs need to be alert to subtle signals in order to detect
problems. Results that violate some expected condition are one common
type of indicator. For example, SPSS users who notice that the average of a
group of numbers exceeds the maximum value (as in a class average of 112
for a test that has only 100 points) can be fairly sure that some of the data
has been entered incorrectly. On a reassuring note, since qualitative data
analysis requires intimate knowledge of the data, a QDA user would have
more opportunities to find anomalies than users of many other complex
Chris (and other advanced users) developed specific strategies for
verifying that results were in line with expectations. These strategies may
be identified and taught to newer users. However, the strategies themselves
are less important than their underlying motivation: the awareness that
complex software like NUD*IST needs to be monitored.
Another way in which metacognition was demonstrated was in
identifying trade-offs: consciously assessing desirable and undesirable
effects of different ways of working with the software. Examples include
one participants speculation about whether using NUD*IST earlier in her
project might have focused the data collection morean advantageat the
cost of allowing the earlier interviews to be too significanta disadvantage.
As she put it, a bit of win-lose in that. Betty, a relative novice, showed this
kind of reflectivity when talking about coding online or on paper. She
observed that the advantages of seeing the coding structure might be offset
by the tendency to fit codes into existing strategies, limiting the
development of categories too quickly.
Reflective users also monitored how the use of a program affected their
working methods over time, sometimes in subtle ways. Participants who
had used NUD*IST over an extended period usually adapted their working
methods and use of the software to accommodate one another. For
example, Chris identified changes in her initial working processes that she
ascribed to the capabilities of QDA software. She now begins by coding
large chunks, organizing her data so that she can reflect on it and develop
more subtle categories. In contrast, she had previously begun her manual
analysis by assigning fine codes, from which she built up her larger
categories. Gwen recognized the issue of setting boundaries so that the user
doesnt end up doing something with NUD*IST to avoid having to make a
decision about what your datas saying. Frances acknowledged that her
data sources had shifted slightly: she tended to use more textual documents
that could be easily handled in NUD*IST, and to rely less on field notes
because they required typing. Reflective users were aware of these changes
as conscious choices.
Of course, there were also instances in which lack of reflectivity could
be identified, generally in terms of unexamined assumptions about the
program. Prior comparative experiences helped users revisit such ideas.



Diane had initial problems that revealed preconceptions she had about
NUD*IST: I thought that...I really had to have all this tree all figured out
before I started...I thought NUD*IST was imposing much more on my
(pause) way of thinking than it actually ended up imposing. A user with
less manual experience might never have questioned that initial assumption, and would have been more influenced by their incorrect understanding of the softwares requirements.
Users who fail to reflect on their use of software are at risk for becoming
distant from their data-handling through unmindful transformations of
the data or through unconscious trade-offs in their working methods. To
avoid this aspect of distance, metacognitive awareness needs to be fostered
in users of NUD*IST and similar cognitive tools.
On the other hand, users who learned to use the software effectively
found that they not only felt close to their data, but that they felt close to
their software. One participant referred to being one with the machine,
and several described how different programs fit with their thinking. At
this level, the program operates as a functional organ, extending the
capabilities of the user (Kaptelinin 1996). Several users felt strongly that
using NUD*IST increased the validity of their work, because they could use
it to explore their data more thoughtfully and to check for errors at levels
that would be impossible manually.
The participants in this study consistently used the metaphor of
NUD*IST as a tool, a metaphor also found in literature surrounding QDA
programs and computers in general (Tesch 1990, Jonassen and Reeves
1996, Hannafin 1999). This metaphor merits further examination with
respect to its implications for developing metacognitive awareness.
Exploring the tool metaphor
Tools extend and qualitatively change human capabilities. Some activities,
like sawing, cannot be performed without tools; others can. Even activities
that can be performed without tools can become transformed with their
use. Transportation tools provide an obvious example: people have always
been able to walk, and thus could travel prior to the invention of the
carriage, the locomotive, the automobile, or the airplane. Yet each new
transportation tool changed the scope of travel, simultaneously transforming the landscape and setting new expectations. Though walking has
not been abandoned, it is generally used for shorter distances or for
pleasure, not as a serious form of regular transportationat least in
developed countries. The effects of these changes have been far-reaching.
Users generally have goals in using a tool, and the sophistication of
their goals and the levels of their skill influence the results of their work.
However, the tools also affect the results, and further, affect the users.
Kuuti writes The tool is at the same time both enabling and limiting: it
empowers. . . with the historically collected experience and skill crystallized to it, but it also restricts the interaction to be from the perspective of
that particular tool or instrument only; other potential features. . . remain
invisible. . . (Kuutti 1996: 27). Kaptelinin adds There are implicit goals



that usually are built into the tools by their developers. The goals
achieved by people equipped with a tool are often influenced by the tools
goal, and the final results differs from both goals, being a compromise
between them (Kaptelinin 1996: 53). Designers speak of affordances of
the tool, which invite certain kinds of interaction (Norman 1988, Carroll
1997). Experienced users in this study seemed more aware of the tools
goals than beginners. Novices tended to assume that the tools were neutral,
whereas experienced users were aware of influences, but expected a good
researcher to be able to control them.
As previously stated, the participants consistently used the metaphor of
QDA software as a tool. Their examplesfile cabinets, calendars,
checkbooks, scissors and gluedemonstrate a general tendency that I
have noticed: when the word tool is used in conjunction with software,
small tools seem to come to mind first. More general metaphors include
hammers, saws, and other woodworking equipmenthand tools that the
user holds and guides. However, not all tools are so small and easily
controlled, and the more power that tools have, the more damage they can
do if misused. A table saw can cut more wood than a handsaw (and do so
with great accuracy), but fingers are also at risk.
A table saw represents a mid-sized tool: no longer a hand tool, but still
on human scale. At the next level are bulldozers and cranes: huge tools that
surround the operator, and that can wreak havoc if misused. In these
examples, each tool is not only larger than the last: it is also more powerful,
requires more skill and safety training on the part of the operator, and can
do more damage if misdirected. (Note that this categorization does not
imply that powerful tools are only appropriate for larger projectstable
saws can be used for very fine cutsmerely that their effects are broader.)
In thinking of cognitive tools such as QDA programs, we need to
consider the scale of the tool in terms of its power. This form of scale affects
both the tools utility in the hands of a skilled user and its danger in the
hands of the unskilled. Most of the advanced users of NUD*IST in this
study approached the program with confidence tempered by respect, which
suggests a relatively high level of power.
Tools vary not only in terms of power, but in their range of functions.
Some tools have only one or two functions, which are readily apparent: a
hammer is for pounding, a saw for cutting. Other tools are more multipurpose. The wider the scope of the tool, the less obvious any one use tends
to be. A table saw that converts to a router offers flexibilityif the user
knows what a router does, and how to configure the saw to provide that
function. Thus, one aspect of tool use involves identifying what functions
are available, when to use themand when to choose a different tool
entirely. The more experienced users in the study actually tended to speak
of NUD*IST as a collection of tools, rather than as a single tool, recognizing
the multiple functions within the program. However, user experience is not
the only factor in recognition of capabilities; there is a tremendous
interaction between the design of any given tool and the users ability to
understand its function (Norman 1988, Norman 1993, Carroll 1997).
Developments in the interface of NUD*IST have made resources within the
program clearer to the user: for example, the establishment of free nodes



created an explicit invitation to code in a non-hierarchical manner, though

some experienced users had recognized that possibility prior to their
As mentioned earlier, the skill of the user is another aspect of tool use.
Development of skill usually requires practice with a specific tool. A master
carpenter can do more with a lathe than a new user, because he or she
knows how to work effectively with it. Similarly, some NUD*IST users
created very sophisticated command files and searches as their understanding of those tools developed. Such development of skill also requires
ongoing learning. The users who were least satisfied with their use of the
program were those who had reached a plateau in terms of their skill level,
and didnt have time to explore the program further.
Finallyand perhaps most importantlythe goals of the user affect
tool use. One reason master carpenters achieve more than the average user
is simply that they understand what good work looks like. Experienced
researchers, like those in this study, are more likely to have pre-existing
goals based in qualitative research standards, and to be able to set new goals
using opportunities offered by the software. The participants generally
realized that their previous experience aided them in this respect. Several
were very worried about novice users who lacked previous research
experience because they dont know how to approach it. Such novice users
clearly are at a disadvantage in terms of establishing high-level goals, and
may be especially prone to being led by the softwareor by their own
misunderstandings of what theyre doing.
In relation to computer-based learning environments, Hannafin (1999)
offers a preliminary typology of functions related to cognitive tools: seeking,
collecting, organizing, generating, processing, and communicating. Note
that features and functions are not necessarily congruent: one feature may
be employed for various functions, and different features can serve the same
function. For example, in NUD*IST, the search function may be used for
exploring themes (seeking), or for collecting and organizing related
instances. Conversely, multiple features can be used for organizing
command files, searches, cut-and-paste within the tree index.
Users must determine appropriate configurations and uses of tools
based on their goals, recognizing that some functions will be superfluous in
any given situation. Hannafins preliminary typology may help users
articulate their immediate goals more clearly. However, high-level goal
articulation must be the function of discussion within the qualitative
community. Software developers and users of QDA programs have
complained that the programs have been condemned for flaws that more
properly apply to the methods themselves, and called for a more vigorous
and general discussion of methods (Fielding and Lee 1996, Kelle 1997,
Fielding and Lee 1998, Richards 1998, 1999). QDA programs should not
be censured for crystallizing standard practices or for explicating existing
problems. Neither should software developers be in the position of driving
methodological development because other qualitative researchers do not
sufficiently articulate analytical procedures and goals.
Based on this exploration of the tool metaphor, figure 1 offers a model
describing expertise with cognitive tools. Three intersecting requirements



Figure 1. Expertise with cognitive tools.

are resource recognition (awareness of the tools capabilities), ability to use

those resources (skill), and goal identification. All three areas are fostered
within a reflective attitude.
No particular sequence is attached to these three requirements. Goals
often drive tool selection and skill development, but sometimes the
capabilities of the tool suggest new goals, or new ways of meeting existing
goals. The level of natural alignment between user goals and identifiable
resources may define the fit with your thinking that participants described
as perceiving in different programs.
Tool advocates frequently stress that tools merely carry out peoples
intentions. This view is most clearly typified in its extreme by slogans such
as guns dont kill people, people kill people. However, a tool generally
does not correct for misdirected intentions or incompetent use. The more
complex and powerful the tool, the more the skill and intent of the user
become an issue. These considerations have implications for the
trustworthiness of qualitative research.
Issues of trustworthiness
In qualitative research, the researcher has historically been considered the
primary tool for research (Merriam 1998). In some texts, the implication
seems almost that using tools taints the research in some way. However,
tools pervade qualitative research, whether practical tools such as tape
recorders and word processors or theoretical tools for analysis. Though
some are now taken for granted, they do have an impact on research. (See
Fielding and Lee (1998) and Seidel (1991) for interesting comments on
changes in the practice of ethnography occasioned by the introduction of
tape recording.) With the advent of QDA software, the researchers ability
to use tools thoughtfully may need to be considered more carefully and
weighed more heavily.



In this study, some participants evaluated other researchers credibility

in part on how well they understood the software they were using. Even the
users who felt that their own work was actually more valid with the
software were unwilling to extend that assumption to other research that
used the same software, unless they had some evidence of researcher
competence with the software.
Participants in this study were also concerned about people attempting
complex studies with insufficient skillsboth technical and conceptual
because their expectations of the software were unrealistic. For example,
they observed that some researchers trained in quantitative methods
learned the software without first familiarizing themselves with qualitative
concepts, transferring their quantitative standards (for example, about
sample sizes) to their qualitative projects.
Thus, goal orientation and ability to use resources (see figure 1) seem
to be aspects of expertise especially related to trustworthinessessentially,
are they doing the right thing? and are they doing that thing right?
(Resource recognition appears to be less of an issue, though it certainly
affects working efficiency.) Advanced participants insisted that research
conducted with software should be held to the same standards of
methodological description required of manual studies: clear goal articulation, supported by a description of methods used, the justification for those
choices, and explicit links between the data and the findings. This suggests
the importance of methodological description may be increased rather than
diminished by the use of softwareand analyzed with program x does not
constitute a description.
In addition to the repeated concerns about novice users who didnt
understand qualitative research, participants believed that research
evaluators (editors, grant providers, and so on) should guard against
assumptions that software use automatically confers credibilityan
assumption that they felt non-users or novices were especially prone to
make. They also suggested that misconceptions of non-users and new users
in positions of power (for example, grant providers) were a cause for
concern. Such misconceptions can create unrealistic or unreasonable
expectations that may deeply influence the practice of qualitative research.
This paper outlines three levels of distance highlighted when working
with qualitative data programs. First, the tactile-digital divide describes
the initial shift between working with paper-based materials and electronic
files. This stage tends to be a temporary, though uncomfortable, transition.
Second, the coding-trap illustrates the issue of finding appropriate
analytical distance. In manual methods, closeness was prized in part
because it was difficult to achieve. Since QDA software makes closeness
easier, the need to balance closeness with a broader perspective becomes
more apparent. Articulating these stages, and identifying strategies that
others have used to work through them, may help new users make the
transition to qualitative data analysis software.



Third, the metacognitive shift addresses the issue of distance from the
mechanics and implications of data handling methods. Skilled users
displayed a strong metacognitive awareness of their own program use,
and seemed to feel the software extended their own thinking. However,
their very awareness highlighted the potential for errors caused by
unthinking or inept application of program functions. Such errors are a
concern for new or inexperienced users of the software, especially those
that lack prior experience that would assist them in goal orientation.
Reflections on the tool metaphor were then used to highlight aspects
of expertise with cognitive tools such as QDA software: goal articulation,
resource recognition, and ability to use resources. The final section
contained a brief discussion on issues of trustworthiness associated with the
use of tools in qualitative research. Experienced users were most aware of
potential influences of the software. They expected a reflective user to be
able to control those influences, and were concerned over unthinking
application of software functions. These findings have implications both
for helping new users and for evaluating research conducted with QDA
software: researchers at all levels must guard against the assumption that
learning qualitative software is equivalent to learning qualitative research.
At all three levels, the balance between closeness and distance is
instructive. The tactile-digital divide forces consideration of what does
closeness to data mean? The coding trap highlights the value of analytical
distance to balance closeness to data, and the value of purposefully moving
between closeness and distance. The metacognitive shift achieves
closeness to the powerful ways that software can transform data, but it
achieves that closeness through reflective self-monitoringa form of
distance, in that researchers step back to look at processes and decisions.
As QDA software makes aspects of closeness more manageable, it
challenges us to re-consider where and when closeness is of value, and
where and when we need a little distance to provide perspective.
Agar, M. (1991) The right brain strikes back. In N.G. Fielding and R.M. Lee (eds) Using Computers in
Qualitative Research (London: Sage Publications), pp. 181 194.
Bogdan, R.C. and Biklen, S.K. (1992) Qualitative Research for Education: An Introduction to Theory and
Methods, second edition (Boston: Allyn and Bacon).
Carroll, J.M. (1997) Human-computer interaction: psychology as a science of design. International
Journal of Human-Computer Studies, 46, 501 522.
Fielding, N. and Lee, R. (1996) Diffusion of a methodological innovation: CAQDAS in the UK.
Current Sociology, 44, 242 258.
Fielding, N.G. and Lee, R.M. (1998) Computer Analysis and Qualitative Research (London: Sage).
Gilbert, L. (1999) Reflections of Qualitative Researchers on the Use of Qualitative Data Analysis
Software: An Activity Theory Perspective (Athens, Georgia: University of Georgia).
Hannafin, M. (1999) Learning in open-ended environments: tools and technologies for the next
millenium. ITFORUM 1999.
Jonassen, D. and Reeves, T.C. (1996) Learning with technology: using computers as cognitive tools. In
D.H. Jonassen (ed.) Handbook of Research for Educational Communications and Technology (New
York: Simon and Schuster Macmillan), pp. 693 719.



Kaptelinin, V. (1996) Computer mediated activity: Functional organs in social and developmental
contexts. In B.A. Nardi (ed.) Context and Consciousness: Activity Theory and Human-Computer
Interaction (Cambridge, MA: MIT Press), pp. 45 68.
Kelle, U. (1997) Theory building in qualitative research and computer programs for the management of
textual data. Sociological Research Online, 2, 5
Kuutti, K. (1996) Activity theory as a potential framework for human-computer interaction research. In
B.A. Nardi (ed.) Context and Consciousness: Activity Theory and Human-Computer Interaction
(Cambridge, MA: MIT Press), pp. 17 44.
Lofland, J. and Lofland, L.H. (1995) Analysing Social Settings: A Guide to Qualitative Observation and
Analysis, third edition (San Francisco: Wadsworth Publishing Co).
Mangabeira, W. (1996) CAQDAS and its diffusion across four countries: national specificities and
common themes. Current Sociology, 44, 191 205.
Merriam, S.B. (1998) Qualitative Research and Case Study Applications in Education, second edition
(San Francisco: Jossey-Bass).
Norman, D. (1988) The Design of Everyday Things (New York, NY: Currency Doubleday).
Norman, D.A. (1993) Things that Make us Smart: Defending Human Attributes in the Age of the Machine
(Reading, Mass: Addison-Wesley Publishing Company).
Richards, L. (1998) Closeness to data: the changing goals of qualitative data handling. Qualitative
Health Research, 8, 319 328.
Richards, L. (1999) Data alive! The thinking behind NVivo. Qualitative Health Research, 9(3), 412
Seidel, J. (1991). Method and madness in the application of computer technology to qualitative data
analysis. In N.G. Fielding and R.M. Lee (eds) Using Computers in Qualitative Research (London:
Sage Publications), pp. 107 116.
Tesch, R. (1990) Qualitative Research: Analysis Types and Software Tools (London: Falmer Press).
Weitzman, E.A. and Miles, M.B. (1995) Computer Programs for Qualitative Data Analysis: A Software
Sourcebook (Newbury Park: Sage Publications).