See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/250869376

Study of User Experiences on Electronic and Paper-Based Reading

Article · July 2008

CITATIONS READS

4 83

2 authors:

Ching Chiuan Yen Ming-Hsu Wang

National University of Singapore Chang Gung University
47 PUBLICATIONS   548 CITATIONS    17 PUBLICATIONS   127 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Creating Unique Technologies for Everyone View project

All content following this page was uploaded by Ming-Hsu Wang on 15 August 2016.

The user has requested enhancement of the downloaded file.

 Study of User Experiences on Electronic- and Paper-Based
Reading

Ching-Chiuan YEN*, Ming-Hsu WANG**

* Department of Architecture, National University of Singapore,

akiyc@nus.edu.sg
** Department of Industrial Design, Chang Gung University, Taiwan
m9042005@stmail.cgu.edu.tw

Abstract: This paper reports the result of an experiment which tested the users’ experience regarding electronic- or
paper-based reading. The use of electronic means for reading is widely spread and utilized for a wider variety of
tasks, including on-line or electronic reading. In addition, the development of “E-Book” has become fashion in IT
industry and even been predicted to replace the conventional paper-based reading. Nevertheless, little research has
been done to understand the association of user preferences between these means, in particular, with regard to Chinese
environment. A qualitative performance measurement applied to 20 university students was carried out, in which
recorded the user behavior and critical incidents in different reading modes based on an analysis of a videotape of
reading process, in order to understand the usage of different means, as well as to monitor how people responded to
various reading environments. The results showed that most people did not cope with e-based reading well. The
disadvantages of paper-based reading, however, did not yet converted to advantages for e-based reading. The main
barriers identified are the lower level of man-machine interactions and self-control during e-reading process. Other
technical issues such as the glaze of screen, and layout and reading position were also reported. A list of
recommendations was made in order to design an e-based reading device which can provide a more comfortable
reading experience. Additional implications and future research directions were also discussed.
Key words: HCI performance, Video display terminal (VDT), Reading model, E-Book, Usability

1. Introduction
With the development of information technology (IT) and the widespread of the Internet applications, the use
of electronic information is increasing within our daily life for recent years. Music, for example, is one of the
first and most successful products undergone the digitalized transformation. Books and papers are also obvious
candidates to go for digitalization. Several scholars [1,5,8] claimed that e-Books are more efficient than
paper-based books from various perspectives such as storage, transfer, delivery and accessibility. Nevertheless,
paper-based documents’ reading has its long tradition. Although the increasing popularity of electronic reading
seems an inevitable fact, everybody being ready for an electronic life is somewhat doubted.
Recently, Dr. Murphy and her colleagues [4] from Ohio State University published an article on American
Psychological Association Annul Meeting and states that “texts on computer screen harder to understand, less
persuasive”. In their research, 131 university students read 2 articles from Time magazine. The students had been
divided into two groups: one group read the printed materials, and the other read the exact same text after it has
been scanned into a computer. The result showed that those who read on paper would have a better comprehension
of the content. A US Department of Labor’s report [9] also indicated that printed papers have numerous
advantages, such as higher resolution, less fatigue, faster reading speed, etc. The report, however, further
identified that electronic information is able to provide more dynamic and higher accessibility to relevant
information on Web. These results do suggest that the electronic reading model is promising. As Dr Murphy [4]
suggested that “students needs to learn different processing abilities when they are attempting to read
computerized texts,” and “ a lot of questions have to be answered before we continue further into making
computers part of the curriculum.” A study based on the comparison between those reading types is therefore
required, in order to identify the strength and weakness of the application to such e-materializations movements.

2. Method
2.1 Objectives
The purpose of this research is to investigate the usages of various reading models, in order to understand the
usability of such modes. Readers’ usage, operation status, satisfaction rate and needs will be brought together as
a reference for future development in related to electronic products.
2.2 Research Contents
Fig. 1 shows the framework of the reading modes for experiment. Reading and searching for texts were the
main focus. 3 reading modes were chosen for experiment according to the different contexts of user-tool-task
model [7] (Fig. 2):
y Mode 1: traditional reading model – a piece of printed out paper for reading and answering, together with a
pen for note-taking and highlighting, as well as answering.
y Mode 2: combination reading model, i.e. reading on an electronic paper (PDF format) and answering on a
piece of printed test paper.
y Mode 3: full electronic reading model – both reading and answering an electronic paper (Word format) on an
electronic device.

Fig. 1 Reading model Fig. 2 User-tool-task model

A performance measurement [6] was applied which recorded the user behaviors and critical incidents in
different reading modes in-depth based on the analysis of real interactions on types. In addition, a “retrospective
testing” method, a variation of the think-aloud method which was introduced by Nielsen [11] for usability test,
was also applied, as which allows the subjects watch the just-completed experiment session on videotapes and
make additional comments while watching. The benefit of using this method is that rich and detailed information
related to how the users think can be obtained. More information is received by reviewing the videotape
together with the participants and asked them questions related to their behaviors during the experiment when
required. Using this technique, however, means that each experiment takes at least two times more than normal
testing.
2.3 Subjects
Twenty subjects participated in the experiment, i.e. 10 males and 10 females. All the participants are the
students of Chang Gung University, as the experiment materials were focused on university-standard students. An
ambiguity in terms of language ability is therefore able to eliminate. Subject details were collected via a
questionnaire after the completion of the experiment. The age of the subjects were 20-25 and average of 22.4
years old.
2.4 Equipments and Research Setting
Fig. 3 shows the environment for experiments which was set up within an enclosed laboratory in Chang Gung
University, in order to avoid distraction from outer environment. A notebook was used to present the electronic
materials (for both Mode 2 and 3) which were displayed on a 14” color LCD screen set to a resolution of
1024x768 pixels. A pen and a piece of answering sheet were provided for Mode 1 and 2, besides the notebook.
A piece of reading paper and answering sheet, and a pen were provided for Mode 1. Behind the participant, a
hidden camera (Sony DV) was set up to record the entire process.

Fig.3 Research environment setting Fig. 4 Mode 1

Fig. 5 Mode 2
Fig. 6 Mode 3
2.5 Materials
To avoid the memorization effects, an alternate-form test were therefore undertaken in the experiment. 15
documents of similar length and level of difficulty were selected from the past reading tests papers of the National
University Entrance Examination. The goal of such an Examination is to assess the standards of students which
are similar to “A” levels in the British system. To have the same level of difficulty, the questions coming with
the original testing materials were edited and in the end 5 question left for each testing paper. A pilot test was
subsequently undertaken in order to evaluate the consistency of the selected documents. 10 university students
were invited to do the reading test. The documents which can not reach 100% correct answering rate were
eliminated. In the end, 10 documents were chosen and transferred into electronic paper (Word and PDF format). A
12-point “Ming” font with single line spacing was applied, as it is one of the most commonly used font in both
Chinese electronic and printed format to avoid ambiguity.
2.6 Procedure
Prior to the experiment, a brief introduction about the testing procedures was undertaken. 3 testing materials
were randomly drawn by the subjects after the introduction. In the experiment, the subject stayed in the laboratory
alone, and started the test from Mode 1 to 3 accordingly and videotaping was undertaken in parallel with the each
test. During the test, no interfering to the participants until he/she said that he/she is done, and he/she started the
next mode. Until 3 modes had been completed, the experiment as whole is considered finished. Right after the test,
a video review accompanied with questionnaire was undertaken to understand how the participants felt during the
process and whether he/she satisfied the specific mode of reading. In total, the whole experiment process for a
subject is around 30-50 minutes.

3. Results and Findings

3.1 User & Task analysis
A user & task analysis was applied to identify the problems encountered in the testing process and to produce a
clear understanding of what the user wants under such a special environment. To analyze the task, an
input-transformation-out mode suggested by Faulkner [3] was applied, the testing process was divided into
“reading”, “information processing”, and “question answering”. Table 1-3 shows the results of user and task
analysis according to these 3 processes.

Table 1 Task analysis for Mode 1
Reading Participants can directly move the
position of the testing paper to the most
comfortable place for reading and
answering.
Processing Can use pen to circle the keywords and
record down in any places within the
paper and anytime of the reading. In
addition, the participants wish, or use
pen as a guide for the eyes.
Answering Can use pen to write the answer on paper
and modify the answer easily.
Table 2 Task analysis for Mode 2
Reading Participants must adjust their heads
toward to the screen, whilst moving the
mouse or stroking on the keyboards to
browse the information.
Processing May use fingers or pen to touch the
screen, and body languages showed are
increasing (e.g. scratch the heads, spin
the pen, and hands on the chins).
Answering Hold the pen whilst answering and may
also use the pen pointed to the screen as a
guide. The pen may be placed down
whilst reading a new question.
Table 3 Task analysis for Mode 3
Reading Participants move their positions toward
the screen
Processing They would place their hands on the Table 4 Mean reading time for each mode
mouse to mark the questions and also on Minimum Maximum Time SD
the keyboard and therefore ready to Mode 1 3:21 5:24 4:12 0:36
answer Mode 2 3:15 6:38 4:33 0:53
Answering They can use the mouse or stroke on the Mode 3 3:00 5:58 4:23 0:43
keyboard whilst answering and
subsequently look back to the screen for
reading a new question and his/her hand
would be rest on the keyboard or the
mouse.

3.2 Reading Time

Table 4 and Fig. 7 demonstrate the time spent between different subjects within different modes. The results
demonstrate that Mode 1 (4:12 SD=0:36) required the shortest time in average and followed by Mode 3 (4:23
SD=0:43) and Mode 2 (4:33 SD=0:53). The use of latter two modes is required at least 10 seconds more than that
of mode 1 in average. In total, the reading time of 12 subjects spent in mode 1 were shorter than that of in both
Mode 2 and 3. 13 subjects spent a relatively longer time whilst using Mode 3.
3.3 Satisfaction
Fig. 8 shows the comparison of user satisfaction between the 3 reading modes. The satisfaction rate of mode 1
is significant higher than that of other two modes. 16 subjects (80%) expressed that they were in favor of using
Mode 1 and only 3 people (15%) preferred to use Mode 3. Ironically, Mode 2 was the most dissatisfied mode as
no one expressed positive experience and 2 subjects however nominated Mode 2 as the most dissatisfied item
(which include one subject who indicated that mode 1 is the most satisfied experience between 3 modes).

07:12 Mode 1 Mode 2 Mode 3

16 Mode 1

06:00
14
12
10
04:48
8
Favoriate

6
03:36
4 Mode 3
2
02:24
0
-2
01:12 -4 Mode 2

00:00 Fig. 8 Comparisons between subjects satisfaction rate in

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
different Modes
Fig. 7 Reading and answering time for individuals

3.4 Subjective evaluation

Table 5 demonstrates the participants’ subjective opinions about the use of the 3 reading modes and the pros
and cons of individual mode were also identified.

4. Discussion
This research applied a combination of a “retrospective testing” technique and “performance measurement” to
identify the problems which users encountered during electronic or paper-based reading. Significant differences
had been identified during the research process as follows:

Table 5 Pros and cons of individual testing mode

Mode Pros Cons
Mode 1. is able to mark directly on the reading 1. is wasting paper and also wasting storage
1 materials, read the highlight whilst answering space.
in order to save time. 2. is easily to catch the ink if bad printing
2. is easy to flip through. quality.
3. is comfortable in terms of sitting position. 3. is easily to doodling on the paper which may
4. is comfortable in terms of less eye strain. affect the cleanliness
5. is more freedom of thoughts while reading. 4. is troublesome as it needs to turn to the
6. is more realistic. previous or next page.
7. is easier to browse a piece of information. 5. is to restrict the mode of thinking
6. is required a liquid paper for modification.
Mode 1. is able to save the resource, i.e. paper 1. is less comfortable as user’s eyes tend to be
2 2. is easier to store the document. tired easily.
3. is able to keep the origin of the document, as it 2. is not convenient and easy to read when a full
is impossible to modify. page mode is required.
4. is able to control time. 3. is tired as the eyesight must move all the
5. is able to let everybody reading the same time, consequently, it is easily to make a
materials on a big screen and answer mistake.
individually or in different sets of questions. 4. is difficult to control the page up and down to
the most appropriate place.
5. is distracting as it needs to look after both
screen and answering sheet.
Mode 1. is easily to make any modification. 1. is not able to jot down anywhere whilst
3 2. is not required to bring a pen. testing.
3. is able to do a distance testing. 2. is more tired as the eyes may feel
4. is convenient to make any query uncomfortable when the screen is too bright
5. is saving resource and space or too dark.
6. is easier to store the document. 3. is limited by the understanding of the
7. is easier to find a piece of specific information software.
as a searching tool is provided. 4. is more tired in terms of sitting position.
5. is easily to lose whilst reading.
6. is too easily to modification the question
sheet.

1. Most participants preferred the conventional reading mode, i.e. paper-based. This may be contributed the
unfamiliarity of e-based reading, and paper-based reading has more freedom in terms of reading position.
2. While reading a piece of paper, most participants would like to place their attention on the pen and be ready
for making marks or answers. While reading on electronic device, more vivid body languages may be
involved (e.g. scratching the head or spinning the pen). The reasons may be linked to the longer reading time
required and uncomfortable reading position.
3. On the time spent, reading on an electronic device spent more time than that of printed paper. This result is
similar to previous relevant study [10]. This may be caused by the format and layout of the text as a few
participants interrupted their reading as they misread to other lines during the experiment. The emit lighting of
screen was also affected to participants’ eyes, as eye strain may be occurred easily as result of reading on too
strong emitted lighting.
4. From the research, the advantage of paper-based reading mode is people used to this mode and a better degree
of freedom provided. On the other hand, electronic device provides people with multimedia and
personalized options. However, the degree of free activity and interactions between electronic papers and the
users are lower than that of printed papers. This may be the reason why the electronic paper can not fully
replace the conventional paper format, even though most of IT industry would prefer the conventional paper
 print out can be replaced by electronic media.
5. In comparison between Mode 2 and Mode 3, the degree of flexibility and the ability of modification between
Word and PDF format may contribute respectively to the levels of user satisfaction. It is obvious that there
is a need to take a note during reading process, however, a security measure kept the paper in the original
contents is also required.
6. In terms of reading position, it is more comfortable to change the position of object, i.e. paper (or ebook)
itself, than change the subject’s position. In this case, a notebook may be too heavy and too big to move
toward to the place where reader wants. The size and weigh of the e-book is therefore important too keep
reader in a comfortable position.
From the participants’ preferences, most people still prefer the usage of conventional reading and testing model,
mostly because they can flip through, make remarks and highlight the texts. The realistic feeling from the paper
text is another important issue which is not easily replaced by electronics media. The radiation and
emitted-lighting from the screen and scrolling up down of the pages causing eye strain. Electronic device is not
entirely inconvenience, for instance, saving resource, easily to perform search are also quoted by the subjects in
the research.
4.1 Research limitation
.The limitations of this study are as follows:
y An alternate form of testing materials were applied, a consideration effort was put though in order to make a
similar levels of difficulties a slightly difference may still exist and consequently a difference may occur
between the time spent for reading.
y The different formats between Mode 2 and 3 may affect the reading performance in terms of time as the
association between software and user’s experience.
y It is difficult to test the comprehensions between various modes’ as the method of alternate form was applied.
y The use of retrospective testing method is very time consuming, which is easily to collect rich qualitative
information but less quantitative information.

5. CONCLUSION
This experiment is a pilot study which attempts to understand the difference between the reading modes. The
use of the combination between performance measurement and retrospective testing provided rich and in-depth
information reading user needs related to different reading modes. The limited sample size and diversity of testing
materials may cause the experiment results less quantitatively significant. A few key influential points were
collected as mentioned in last section. According to the findings mentioned in previous sections, several
recommendations can be made for the design of an electronic device (may be an e-book, e-paper or any other
e-form media) as follows:
y The size and weight of ebook should be considered. The user prefers the paper can be approached to the user
rather than that of vise versa.
y The interaction between e-paper and user is crucial. Most of user prefers a higher degree of freedom in terms
of note-making, highlight, etc, but without changing the original contents. The way of allocation the
information is also vital in particular for which larger than a full page screen.
y The legibility of screen formats is vital as user may misread.
 y The software used should be similar to that of the user used to, as user may need to take some more time to
learn.
y The design of emitted-lighting system for the screen is also crucial, as the current system may cause users’ eye
strain.
Further research directions should explore the idea of applying objective measurements to comprehension,
reading speed, and tiredness. The study focus on different target user groups may also useful as which can build
the user spectrum as a whole. To sum up, the importance of man-machine interactions had been identified for
the design of an electronic information device. It is hoped the result of the study can serve as a useful reference
for the designer whilst designing an e-form of reading and testing device.

Acknowledge
This research was supported by a grant from National Science Council. Special thanks also need to go to WY
Liaw, YW Chan for their help to complete this research.
References
1. Bryant, J. M., “The paperless book”, IEEE Review, November, 1995, pp.245-247.
2. Dyson, M and Kipping, GJ, “The legibility of screen formats: are three columns better than one?” Computer&
Graphics, 21(6), 1997, pp.703-712.
3. Faulkner, X. Usability Engineering, Macmillian, Hampshire, 2000.
4. Grabmeier, J, “Texts on computer screens harder to understand, less persuasive” Research News, Ohio State
University, available: http://www.osu.edu/units/research/archive/comptext.htm [12.20.2002].
5. Harrison, B. L., “E-Books and the future of reading”, IEEE Computer Graphics and Applications, May/June,
2000, pp.32-39.
6. INUSE, Usability Assessment, Version 1.2, Information Engineering Usability Support Centres, 1996.
7. Kreifeldt, J.G., Ergonomic product design course format. HFES 2000 Congress, 2000, p 2-117~120
8. Lee, K. H., Guttenberg, G. & McCrary, V., “Standardization aspects of eBook content formats”, Computer
Standards & Interfaces, 24, 2002, pp.227-239.
9. Levi, M, Literature at the Human-Compute Seam, US Department of Labor, 2002 available:
http://www.bls.gov/ore/htm_papers/st000100.htm [12.20.2002].
10. Mayes, D. K., Sims, V. K. & Koonce, J. M., “Comprehension and workload differences for VDT and
paper-based reading”, International Journal of Industrial Ergonomics, 28, 2001, pp.367-378.
11. Nielsen J, Usability Engineering, Academic Press, Boston, 1993

View publication stats