Compare the conventional one-to-one assessment to the

computerized assessment approach. Critically evaluate

their use in different settings.

The term assessment has various meaning relative to its setting. Psychologically, it is the

compilation and integration of psychological related data obtained from tools such as multiple

administration of test and etc in order to perform a psychological evaluation and subsequently a

clinical diagnosis (Cohen-Swerdlik, 2009). Although assessments are extremely useful and

practical, there are numerous intrinsic drawbacks that comes along side with it such as the high

cost of testing and complications such as inconsistent scoring, administration & interpretation of

test data (Lukin, 1985).Traditionally, conventional one-to-one assessment such as paper & pencil

testing were used. However, with the rapid development of Information & Communication

Technology such as the Internet, a steady shift in paradigm from paper based to computer-based

testing has been employed by almost all sectors (Kuzu & Uysel, 2009).Computer-based

assessments (CBA)/ testing were originally implemented as a green IT strategy to reduce paper

consumption (Chua, 2011). The term computer-based assessment is defined as the utilization of

digital technology to compile, process & report the results obtained from a psychological

assessment (British Psychological Society, 1999) while the conventional Paper & Pencil Testing

(PPT) involves manually administering and interpreting the test. One of the main advantages of

computer-based assessments is the efficiency of the computer which completes all the “tedious”

steps in the procedure of an assessment which includes administering items, recording responses

& scoring results. Therefore in comparison to conventional assessment which is dependent on

human personnel, computer-based assessments can provide significant cost,labor & time savings.

Similarly, in the conventional assessment it is almost impossible to provide standardization for

every test administration and etc therefore variation is inevitable. It is however possible

otherwise when computer-based assessments are used, consequently, improving the reliability of

the test. Likewise, when using Computer-based assessments more precision is acquired as the

timing and delivery of test items & the response measurement is much more consistent.

(Singleton, 2004). In sum, the preference of CBA over conventional assessments can be ascribed

to the automation of test administration & the rapid scoring & interpretation of data. Since data

entry is automated, it is unlikely for data to go missing (Cronk & West, 2002). Other advantages

that CBA can offer are as such; - random item selection, the ability to automatically tailor an

equivalent, alternate test form based on the available items in the test bank (Millsap, 2000)

Because of the ample amount of advantages using CBA such as those outlined, test

administrators typically convert the traditional paper & pencil test for administration through a

computer (Joubert & Kreik, 2009). However, CBA does also come with a number of drawbacks

such as the extremely high initial setup cost of CBA which may not be compatible for certain

types of assessment such as performance & extended response questions. Content errors,

computer glitches, data security lapses and server crashes are some of the other disadvantages in

CBA (Jawaid et al., 2014). Likewise, although CBA has many advantages in contrast to the

conventional methods, it must maintain an equivalency of results & comparability with its

conventional paper & pencil testing counterpart (Lukin,1985) in order to be considered valid and

reliable .
 In the athletic & sports setting, there has been an increase in the use of computerized

neurocognitive assessment tools (NCAT) exclusively or as a complement to the traditional

neuropsychological testing batteries (NP) as screening tools for cognitive deficits obtained from

mild traumatic brain injury (mTBI) (Cole et al., 2018). NCAT typically used in this setting are

the CogState & ImPACT. It is especially crucial to determine the psychometric properties of

these tools to fully delineate their clinical use as such subclinical impairment (mTBI) is only

usually detected upon careful neuro-psychological testing. Because of the severe long-term

neurological and cognitive consequences that an athlete can obtain if he/she resumes

participation in sports without full recovery upon succumbing to traumatic brain injury, it is vital

to accurately assess the cognitive abilities of the athlete via neuropsychological testing instead of

opting for the subjective rating of his/her symptoms as an indicator of recovery (Dicker &

Maddox, 1993). Because of this, many sports physicians have opted for the NCAT due to its

sensitivity in detecting mild changes in cognition over a period of time as many athletes

generally suffer from increased deteriorating cognition with repeated exposures to head injuries

which occurs often in sports (Cantu,1998). NCAT usually measures reaction time (milliseconds)

which minimizes psychometric drawbacks as this enables the possibility of a wide range of

performance level & reduces practice effects which contributes to a stronger reliability in

comparison to NP. A study by Collie et al., (2001) proposed that this detection (cognitive

changes) can only be detected using NCAT as it possesses good psychometric properties in

comparison to NP, chiefly for task that requires higher complex cognition. NP has poor test-

retest reliability, limited range of possible scores & is highly prone to floor & ceiling effects.

Other advantages of using NCAT includes that it is sensitive to variability therefore it can detect

inconsistencies in performance which is thought to be a better indicator of cognitive dysfunction

as proposed by Bleiberg et al., (1997) whereby patients with TBI had varied performance in

response time in a day & across four consecutive days. Similarly, further advantages comprises

of the practicality of NCAT compared to NP as aforementioned such as automated data storage

& scoring in milliseconds which allows immediate interpretation by the physician, the ability to

assess an entire sporting team all at once given computing resources are available & the ability

to maintain the standardization of administration protocols (Collie et al, 2001). However, NCAT

does come with a number of drawbacks, computerized cognitive tests like CogScreen requires

additional distinct hardware like a touch sensitive screen which is rather expensive and

inconvenient in terms of portability. On the other hand, there are numerous free paper & pencil

neuropsychological tests in scientific literature that can be administered independent of a

computer. Furthermore, recent findings by Cole et al., (2018) pointed out that the criterion

validity ( NCAT scores in accurately predicting a clinical condition) and construct

validity(Correlation of NCAT scores when compared with the NP test scores in measuring a

similar cognitive domain were medium at best) were not promising as established.

Clinically, computerized psychological assessments (CPA)are used more extensively as

they are used in clinical assisted interviews, written tests & an aid in the diagnosis of a

psychological condition. The use of computers as an essential component of a clinical

assessment & intervention has been rapidly growing and has been shown to be beneficial for

clinical practices particularly for treating adolescents & children as previous studies have

exhibited that the preference for CPA are contingent on familiarisation and exposure. Similarly,

most of the available applications in testing are suited solely for youths (Berger, 2006).In child

psychiatry, Computerized assessments is fairly attractive due to the fact that it can be tailored in

a way to be engaging & suited to its audience, therefore it is useful for children as it facilitates
communication which enables a child to convey his/her self-perceptions and etc which is

essential for a diagnosis (Parkin, 2000). Through Computer Assisted Interview (CAI), Steward

& Steward (1996) provided evidence that children gave disclosure & more detailed information

about molestation compared to a verbal interview. A similar finding was obtained for adolescents

whereby CAI led to a substantial increase in the disclosure of sensitive information compared to

the paper & pencil procedures (Supple et al., 1999). therefore increasing the overall reliability.

Because the diagnosis of a mental disorder, learning difficulties, behavioural complications is

contingent upon informant’s accurate reporting of his/ her own symptoms, reliability of the

information is pivotal. Although CAI can help increase the reliability of information obtained,

structured interviews administered by clinicians are still considered the gold standard which as

Nurcombe (1992) highlighted may cause inaccuracy in diagnosis which primarily arises from

leading questions and etc which tends to lead to overdiagnosis of a mental condition. A study by

Cawthorpe (2001) provided evidence that computer-based interviews such as the Computer-

Based Diagnostic Inventory Schedule for Children - Revised (CDISC-R) is relatively accurate in

diagnosing depression compared to compared to structured interviews conducted by clinicians.

The Achenbach Child Behaviour Checklist which has been computerised for parents’ completion

has shown to be more reliable as parents agreeably gave more spontaneous written answers as

opposed to the paper version (Parkin, 2000)Therefore, it can be concluded that the utilization of

CAI will lead to more objective and accurate results. Some Computerized Tests uses certain

programs for children which includes “virtual reality” & computer games which facilitates

clinical assessments more efficiently, mainly in accurately measuring a child’s impulsivity,

attention and vigilance in children with behavioural, emotional & learning difficulties which
could be otherwise hard to measure in a traditional test due to its non-engaging, and potentially

anxiety-provoking nature.

There are researches that maintain the question of equivalency of CBA vs PPT. Schatz,

Ballantyne & Trauner (2001) suggested that both the test modes had low construct validity when

it comes to measuring a similar construct. Another study by Schatz et al., (2001) supported this

claim whereby when comparing the construct of attention in the CBA version (Tests of Variable

of Attention - TOVA) & the PPT version (Connor’s Parent Rating Scales) on children with

ADHD, both procedures correctly identified 85% of the children as having problems in attention,

however, only the CBA version identified 30% of the control children as having such problems

which was not detectable via the PPT version suggesting important difference between the two

modes of testing. Overall, it seems the primary advantage of CBA over PPT is probably the

richer set of data that can be obtained such as reaction time which is of high clinical value. The

standardization of administration which substantially reduces variation is instrumental in clinical

assessment that requires repetitious procedures such as memory testing. Other advantages

include the accuracy of scoring and the saving of costs and time (Berger, 2006). Likewise, data

collection in CAI is also known to be more convenient and is subjected to less errors. The most

important limitation of CBA is the loss of rapport between the clinician and the testee which is

usually built in the traditional clinical interview. Consequently, some data that could be of

significant clinical utility like response style, interacting style, managing difficulties are

dismissed. Essentially, these observations are an important aspect in understanding the results.

Similarly, many have argued that CAI is unethical especially when the testees are undergoing

psychological distress.
 In sum, many literatures have indicated that direct comparison particularly for NCAT &

NP can’t be made as adaptation of the NP to a computerized platform ultimately alters the test in

terms of its construct (Bauer et al., 2012)which may be attributed to the “method variances” in

the procedure of both the tests. For instance, when testing verbal memory, NP usually opt for

auditory presentation of the stimuli. In contrast, NCAT usually presents the stimuli visually.

Similarly, what is tested usually varies between NCAT & NP. For example, in memory testing,

NP usually employs free recall of words, meanwhile, NCAT typically employs a recognition task

The contrast of the presentation of stimuli & method of responding (i.e: what is measured)

between both NCAT & NP further suggest that these variabilities will affect the reliability of a

NCAT adapted from a NP. It is no doubt that the use of computerized assessment is highly

practical in the athletic and clinical settings. However, computerized assessment should not

completely replace its traditional paper & pencil counterpart as issues of reliability and validity

still exists. In the clinical setting, it is vital to not replace a professional clinician with a

computer, computers should be seen solely as a tool to aid in a diagnosis procedure. As

aforementioned, because the use of computers is contingent upon familiarity, computerized

assessment should be made optional to the traditional paper & pencil testing, particularly for

older people as previous research (Dimock & Cormier, 1991) have established that the

performance in computerized assessment is affected by computer anxiety.

 References

1. Al-Amri, Saad. (2008). Computer-based testing vs. paper-based testing: A comprehensive

approach to examining the comparability of testing modes. Essex Graduate Student Papers in

Language & Linguistics. 10.

2. Calam, R., Cox, A., Glasgow, D., Jimmieson, P. and Larsen, S. (2000). Assessment and Therapy
with Children: Can Computers Help?. Clinical Child Psychology and Psychiatry, 5(3), 329-343.
3. Cawthorpe, D. (2001). An Evaluation of a Computer-Based Psychiatric Assessment: Evidence for

Expanded Use. Cyberpsychology & Behavior, 4(4), 503-510.

https://doi.org/10.1089/109493101750527060

4. Chua, Yan Piaw. (2011). Comparisons Between Computer-Based Testing And Paper-Pencil

Testing: Testing Effect, Test Scores, Testing Time And Testing Motivation.

5. Cole, W., Arrieux, J., Ivins, B., Schwab, K. and Qashu, F. (2017). A Comparison of Four

Computerized Neurocognitive Assessment Tools to a Traditional Neuropsychological Test Battery

in Service Members with and without Mild Traumatic Brain Injury. Archives of Clinical

Neuropsychology, 33(1), 102-119.

6. Collie, A. (2001). Computerised cognitive assessment of athletes with sports related head injury.

British Journal Of Sports Medicine, 35(5), 297-302. https://doi.org/10.1136/bjsm.35.5.297

7. Florian, L., & Hegarty, J. (2007). ICT and Special Educational Needs (pp. 1-6). Maidenhead:

McGraw-Hill International (UK) Ltd.

8. Lukin, M., Dowd, E., Plake, B., & Kraft, R. (1985). Comparing computerized versus traditional

psychological assessment. Computers In Human Behavior, 1(1), 49-58.

https://doi.org/10.1016/0747-5632(85)90006-8

9. Millsap, C. (2000). Comparison Of Computer Testing vs Traditional Paper & Pencil Testing

(Doctor Of Philosophy). University Of North Texas.

10. 9 Parkin A. Computers in clinical practice: applying experience from child psychiatry. BMJ

2000;321:615-618. https://doi.org/10.1136/bmj.321.7261.615

11. Rubenzer, S. (1992). A comparison of traditional and computer-generated psychological reports

in an adolescent inpatient setting. Journal Of Clinical Psychology, 48(6), 817-827.https://doi.org/

10.1002/1097-4679(199211)48:6<817::aid-jclp2270480617>3.0.co;2-5

12. Schulenberg, S., & Yutrzenka, B. (1999). The equivalence of computerized and paper-and-pencil

psychological instruments: Implications for measures of negative affect. Behavior Research

Methods, Instruments, & Computers, 31(2), 315-321. https://doi.org/10.3758/bf03207726