THE AMERICAN JOURNAL OF DISTANCE EDUCATION, 19(3), 163181 Copyright 2005, Lawrence Erlbaum Associates, Inc.

Using Socratic Questioning to Promote Critical Thinking Skills Through Asynchronous Discussion Forums in Distance Learning Environments
Ya-Ting C. Yang
Institute of Education National Cheng-Kung University

Timothy J. Newby
College of Education Purdue University

Robert L. Bill
School of Veterinary Medicine Purdue University This study investigated the effects of using Socratic questioning to enhance students critical thinking (CT) skills in asynchronous discussion forums (ADF) in university-level distance learning courses. The research effort empirically examined two coherent subjects: (a) the efficacy of teaching and modeling Socratic questioning for developing students CT skills in ADF and (b) the persistence of students CT skills following the teaching and modeling of Socratic questioning in the ADF. The results indicate (a) teaching and modeling of Socratic questioning helped students demonstrate a higher level of CT skills and (b) students maintained their CT skills after exposure to and modeling of Socratic questioning in the ADF.

Encouraging students to develop critical thinking (CT) skills has become an important issue in higher education. Several teaching strategies, such as classroom assessment techniques (Angelo 1995), cooperative learning
Correspondence should be sent to Timothy J. Newby, Purdue University, BRNG, Room 3138, 100 North University Street, West Lafayette, IN 47907. E-mail: newby@purdue.edu

163

USING SOCRATIC QUESTIONING

strategies (Cooper 1995), and case study pedagogy (McDade 1995) have been proposed to help promote CT. King (1995) and Taba (1966) suggested that the level of thinking that occurs is influenced by the level of questions asked. Asking thoughtful questions plays an important role in inducing students higher-level cognitive processes, such as self-reflection, revision, social negotiation, and conceptual change of student misconceptions, all of which are integral to CT. In addition, when students are asked to generate questions on their own, factual rather than thought-provoking questions are generally posed (Dillon 1988; Flammer 1981; Kerry 1987; King 1990). Socratic questioning is one of the most popular and powerful teaching approaches that can be used to guide students in generating thoughtful questions, thus fostering their CT skills (Maiorana 199091; Paul 1993). Instead of providing direct answers, the Socratic questioning approach stimulates studentsminds by continually probing into the subject with thought-stimulating questions (Paul 1993). As a result, through active interactions between instructors and students and among students, Socratic questioning can facilitate studentsCT skills by the exchange of ideas and viewpoints, giving new meaning to content, exploring applications to problems, and providing implications for real-life situations (Maiorana 199091). In the context of distance education, however, interactions are often hindered because learners and instructors rarely meet face-to-face. Due to the lack of effective two-way communication tools to facilitate interactions for distance educators, relatively few researchers have explored the presence of CT within distance education. Emerging technologies such as asynchronous discussion forums (ADFs) (i.e., text-based computer-mediated communication tools) have enabled educators to make distance education a truly interactive experience (Anderson and Garrison 1995; Garrison 1993). Since ADFs are promising tools that offer an opportunity for interactive discussions and also provide an opportunity for the instructor to moderate student discussions (Duffy, Dueber, and Hawley 1998; Walker 2004), it appears feasible to integrate Socratic questioning as a part of distance instruction. This has the potential to improve students CT skills through an ADF (Hettinger 1995; Newman, Webb, and Cochrane 1995). Gunawardena, Lowe, and Anderson (1997) studied critical discussions in electronic mail and were also able to provide evidence that CT can occur in computer conferencing and that such processes occur as a direct result of exchanges among participants. Using case study and qualitative research methods, Bullen (1998) concluded that CT can be demonstrated through computer conferencing. Jeong (2000) showed that a structured bulletin board discussion can support CT and critical discussions.
164

YANG, NEWBY, BILL

The aforementioned studies have provided valuable information indicating that students CT skills can be fostered and demonstrated through structured online discussions. Nevertheless, some of the research results, such as Jeongs, were merely based on self-reported student surveys or surveys used to gain descriptive information. To truly ascertain the effectiveness of structured online discussions in improving students CT skills, there is a need to conduct more true and quasi-experiments. Thus, the first goal of this study was to conduct empirical research to ascertain the effectiveness of structured online discussions in improving students CT skills. In this study, online discussion exercises, including debates and case studies facilitated during the semester, were used in an ADF to monitor students mastery of CT skills. In particular, we were interested in realizing whether students can maintain their CT skills after the instructor concluded the facilitation of Socratic questioning. Thus, the second goal of this study was to investigate the persistence of studentsCT skills after the application of teaching and modeling Socratic questioning in an ADF. Specifically, based on these two goals, an interrupted time-series quasi-experimental design was employed to test the following research questions: Will students CT skills improve after participating in Socratic dialogues, as modeled and facilitated by the instructor during structured ADF discussions? After exposure to and modeling of Socratic questioning, will students maintain their higher levels of CT skills without the instructors further facilitation? Method The experimental research was conducted for two consecutive sixteen-week semesters. As shown in Figure 1, planned research procedures (Treatment I and Treatment II), consisting of several observations administered at appropriate times, were performed to measure and collect the required data for the research analysis. These two treatments were conducted using the same course, instructor, and course format. The independent variable (X1) was the teaching and modeling of Socratic questioning within the ADF. The dependent variables were the students CT skills, measured via the California Critical Thinking Skills Test (CCTST) (Facione 1990a, 1992), and class discussions on the ADF.
165

USING SOCRATIC QUESTIONING

Figure 1. Interrupted Time-Series Quasi-Experimental Style Note: X1 = independent variable (the teaching and modeling of Socratic questioning in ADFs); O1 = scores on CCTST and results of precourse demographic survey; O2, O3, O4, and O5 = CT skills demonstrated in online discussions 1, 2, 3, and 4, respectively; O6 = scores on CCTST at the end of the semester.

The CCTST is aimed at college/graduate students and adult professionals (Facione 1990b). The test consists of thirty-four multiple-choice items and targets the core CT skills regarded to be essential elements in a college education. The test reports an overall score on ones CT skills and five subscales: analysis, evaluation, inference, deductive reasoning, and inductive reasoning. There were four online discussions within each treatment. The instructor began the online class discussion by launching into two case studies and two controversial issues for debates that were important and relevant to the course content and reading assignments. During each discussion, the instructor asked the students to identify and post at least one argument or strong example to support their comments about the discussion issue. Each student was required to respond to at least one other students posting to explore the issue at hand and widen the discussion. Finally, at the end of the discussion, students were asked to either summarize the points that were made during the discussion or to write a short reflection about the discussion. The content of the class discussion posted on the ADF was analyzed qualitatively by using the coding scheme for evaluating critical thinking in computer conferencing, which is discussed in detail in the Data Analysis section. Participants Sixteen undergraduate veterinary distance learning students (thirteen females and three males) at a large university in the midwestern United States participated in the study. Within Treatment I there were eleven participants, while Treatment II had five participants.
166

YANG, NEWBY, BILL

Procedures During each semester, the instructor posted four specific discussion topics. Discussions 1 and 2 (O2 and O3) took place during the first half of the semester. Discussions 3 and 4 (O4 and O5) occurred during the second half of the semester. Of these four discussion exercises, there were two case studies (Discussions 1 and 3) and two debates (Discussions 2 and 4). Each discussion lasted for two weeks. The independent variable (X1) in Treatments I and II was administered at different times (see Figure 1). For Treatment I, during the second half of the semester, the instructor taught and modeled Socratic questioning explicitly and students practiced these questioning techniques by composing comments as well as challenging others in the ADF. The following exemplifies some Socratic questions the instructor modeled and asked to help students examine their thinking:
(Questions of clarification) Student A: Cardiovect, in all honesty, is probably not the best drug for any of these cases. It has no real intrinsic activity on the receptor for angiotensin II and aldosteronewhich would help dogs 1 & 2. The instructor: Im unclear about your point here. Are you saying that no intrinsic activity would help dogs 1 & 2 or that having intrinsic activity for angiotensin II would be beneficial? I can read the comment either way. (Questions that probe assumptions) Student B: I would tell the owner that at this time Immodium A-D should not be given. The reason for that is this type of medication decreases gut motility and slows down the intestinal movement. This would prolong the time the bacteria or enterotoxins are in the intestinal tract and might cause future damage. The instructor: You seem to be assuming that the cause of Clarences sickness is the bacteria or enterotoxins in his intestinal tract. By what reasoning did you come to this assumption? (Questions that probe reasons and evidence) Student C: Clarences abdominal radiographs revealed evidence of possible hypersecretion of the intestines. The instructor: What evidence would this be? Student D: No, the insecticide would not cause the weight loss and the chronic diarrhea. Instead it can cause dyspnea because it would inhibit the activity of acetylcholinesterase, resulting in bronchoconstriction and dyspnea.

167

USING SOCRATIC QUESTIONING

The instructor: Specifically, why wouldnt it cause weight loss? Any ideas of how to back your statement? Think about your statement concerning dyspnea. What compound (neurotransmitter) stimulates the smooth muscles in the respiratory tract to constrict? What effect would this insecticide have on that neurotransmitter? Does this support your argument?

For Treatment II, Socratic questioning was taught and practiced during the first half of the semester, and students practiced these questioning techniques themselves through online discussions during the second half of the semester without the instructors further facilitation. Treatment I was designed to investigate the efficacy of the manipulation of X1 in promoting students CT skills, while Treatment II was designed to analyze whether students would maintain their CT skills after the manipulation of X1. Data Analysis Both quantitative and qualitative methods were used to analyze the data collected in this study. The quantitative data, scores on the CCTST, were analyzed using the Generalized Linear Model analysis of variance procedure. The qualitative data, online class discussions on the ADF, were analyzed via content analysis. Online class discussions were analyzed by the coding scheme for evaluating critical thinking in computer conferencing (see Table 1), which was created by combining the Interaction Analysis Model for examining CT in computer conferencing, developed by Gunawardena, Lowe, and Anderson (1997) (see Table 1, Part A) and a modified coding scheme with indicators of critical (+) and uncritical () thinking developed by Newman, Webb, and Cochrane (1995) (see Table 1, Part B). The coded qualitative data were then further analyzed using a chi-square test to investigate whether students demonstrated CT skills from the online class discussions. Coding scheme for evaluating critical thinking in computer conferencing. Gunawardena, Lowe, and Anderson (1997) developed the Interaction Analysis Model for the evaluation of the process of knowledge construction that occurs through social negotiation in computer-mediated communication (CMC) (400). They focused on assessing the quality of interactions in a CMC environment from both the interactive and cognitive dimensions. As shown in Table 1, Part A, their model includes five phases of interactions and twenty-one operations (subcategories) to study critical discussions in electronic mail. This model was designed to assess
168

Table 1. Coding Scheme for Evaluating Critical Thinking in Computer Conferencing

Part A: Interaction Analysis Model Phase I: Sharing/comparing of information A. A statement of observation or opinion B. A statement of agreement from one or more other participants C. Corroborating examples provided by one or more participants D. Asking and answering questions to clarify details of statements E. Definition, description, or identification of a problem Phase II: The discovery and exploration of dissonance or inconsistency among ideas, concepts, or statements A. Identifying and stating areas of disagreement B. Asking and answering questions to clarify the source and extent of disagreement C. Restating the participants position, and possibly advancing arguments or consideration in its support by references to the participants experience, literature, formal data collected, or proposal of relevant metaphor or analogy to illustrate point of view Phase III: Negotiation of meaning/co-construction of knowledge A. Negotiation or clarification of the meaning of terms B. Negotiation of the relative weight to be assigned to types of argument C. Identification of areas of agreement or overlap among conflicting concepts D. Proposal and negotiation of new statements embodying compromise, co-construction E. Proposal of integrating or accommodating metaphors or analogies Phase IV: Testing and modification of proposed synthesis or co-construction A. Testing the proposed synthesis against received fact as shared by the participants and/or their culture B. Testing against existing cognitive schema C. Testing against personal experience D. Testing against formal data collected E. Testing against contradictory testimony in the literature Phase V: Agreement statements/applications of newly constructed meaning A. Summarization of agreements B. Applications of new knowledge C. Metacognitive statements by the participants illustrating their understanding that their knowledge or ways of thinking (cognitive schema) have changed as a result of the conference interaction

[IA] [IB] [IC] [ID] [IE]

[IIA] [IIB] [IIC]

[IIIA] [IIIB] [IIIC] [IIID] [IIIE]

[IVA] [IVB] [IVC] [IVD] [IVE]

[VA] [VB] [VC]

(continued)

169

USING SOCRATIC QUESTIONING

Table 1 (Continued)
Part B: Analysis Model for Analyzing Depth of Critical Thinking R+ Relevance R+ Relevant statements to the issue discussed R Totally irrelevant statements to the issue discussed I+ Importance/Significance I+ Important/significant points/issues I Totally unimportant, trivial points/issue N+ Novelty N+ Provide new information, ideas, or solutions that have never been mentioned (even they are not important or useful) N Repeat what has been already said without any further exploration A+ Accuracy A+ The references/literature used or information/data collected to support the participants position are accurate and true A The references/literature used or information/data collected to support the participants position are clearly false J+ Justification J+L+ Provide a logical statement of opinion, agreement, or disagreement with supporting reasons/examples/justifications/proof J+L Provide an illogical statement of opinion, agreement, or disagreement with supporting reasons/examples/justifications/proof J Statement with simple agreement, disagreement, or alternative opinions without elaboration C+ Critical Assessment C+L+ Critical assessment/evaluation of ones own previous statements/reflection or others contributions toward the issue discussed with logical thinking process C+L Critical assessment/evaluation of ones own previous statements/reflection or others contributions toward the issue discussed with illogical thinking process C Uncritical or unreasoned acceptance/reject

the exchanges made among class members and how these exchanges moved from the lower to the higher phases of CT. According to Gunawardena, Lowe, and Anderson (1997), movement from one phase to the next shows that knowledge is constructed by the process of social negotiation. The transcript analysis procedure involves reading and coding the messages to one or more of the five phases. Newman, Webb, and Cochrane (1995), as shown in Table 1, Part B, provided a content analysis method to measure CT in computer-mediated
170

YANG, NEWBY, BILL

group learning. To measure the frequencies of specific critical (+) and noncritical () skills demonstrated in the discussions, they developed a set of paired CT indicators (x+ ), such as R+ and N+ , where x are CT indicators such as relevance (R) and novelty (N). x+ is the count of positive statements of a CT indicator and x is the count of negative statements in a transcript. Statements from the discussion transcripts were analyzed and scored across the list of indicators. Once the scripts were marked, the totals for each + or indicator were counted, and a depth of CT ratio (x ratio) was calculated for each of CT indicators: x ratio = (x+ x)/(x+ + x), converting the counts to a 1 (all uncritical) to +1 (all critical) scale. For example, 48 positive statements of relevance (R+) and three negative statements of relevance (R) were found in a transcript. The CT ratio of relevance (x ratio) is 0.88 ((483)/(48+3)). This was done to produce a measure that was independent of the quantity of participation, reflecting only the quality of the messages. While the aim of Newman, Webb, and Cochranes (1995) coding scheme was to measure the amount and type of CT taking place in group learning to check on the possibilities of using computer conferencing to promote deep learning, Newman, Webb, and Cochrane did not attempt to measure the interactive dimension taking place in the discussion. Gunawardena, Lowe, and Anderson (1997) and Newman, Webb, and Cochrane (1995) both focused on the problem of assessing the quality of interactions in a CMC environment. While Newman, Webb, and Cochrane aimed at assessing how much and what kind of CT takes place, Gunawardena, Lowe, and Anderson aimed at evaluating the processes of critical discussions from the interactive and cognitive dimensions. Therefore, the tag codes with critical (+) and uncritical () valences used in Newman, Webb, and Cochranes coding scheme were incorporated into Gunawardena, Lowe, and Andersons interaction model to form a new coding system, the coding scheme for evaluating critical thinking in computer conferencing, for our study (see Table 1). The combined coding system evaluated both whether students demonstrated CT skills in their online discussions and whether studentsdiscussion moved from the lower to the higher phases of CT. Training session for establishing interrater reliability. To establish adequate interrater reliability for this study, the data from two online discussions in the pilot study were evaluated by two ratersthe course instructor and the researcherusing the final coding scheme (Table 1). In the training session, the researcher first provided the article, Critical Thinking: What It Is and Why It Counts (Facione 1998) and a coding
171

USING SOCRATIC QUESTIONING

manual detailing the exact criteria and procedure for assigning a code for different units. After reading the article and the manual, the raters discussed the concept of critical thinking together, and then reviewed as well as negotiated the definition of twenty-one CT categories and fourteen different tags in the coding theme one by one. Consequently, the raters separately evaluated and assigned ratings to the first online discussion in the pilot study. The coding results were compared by the raters to reach an overall agreement on the definitions and criteria for the selection of the codes or tag codes. Using Miles and Hubermans (1994) interrater reliability formula (reliability = number of agreements / (total number of agreements + disagreements)), the interrater reliability rate for this initial coding was determined to be 83.27% agreement. Of the total 245 coding decisions made, there were 41 differences. When differences occurred, the researcher and the instructor discussed the discrepancies in the coding results until a consensus was reached through mutual conversation. After the first coding, the instructor and the researcher independently coded the other online discussion in the pilot study and then discussed the results before actually coding the online discussion postings for the study. The interrater reliability rate for the second coding results reached 93.12%. Based on this rating, both the accuracy and reliability of using this coding instrument meet the general check-coding standard that is required in the 90% range (Miles and Huberman 1994). This check-coding process was a time-consuming task. However, real rewards were gained by bringing an unequivocal, common vision of what the codes meant and which blocks of data best fit which codes and by establishing consistency of judgment among the raters. Coding procedures. One of the first tasks in the coding procedure was to parse the discussion transcripts into a unit of analysisthat is, what portion of communication will be used as the smallest unit to analyze. A unit of analysis can be a phrase, sentence, paragraph, or message that illustrates any one of the indicators. The instructor and researcher discussed, negotiated, and then parsed the discussion transcripts into units of analysis together. Then, they independently rated each unit across the Gunawardena, Lowe, and Anderson (1997) category of interactions and all six of Newman, Webb, and Cochranes (1995) criteria, if applicable. The following examples are used to explain how each unit of the online postings was coded. For example, if Student A started a new (N+) discussion (IA), which is relevant (R+) and important (I+) to the discussion issue, but the statement was inaccurate (A) and without supporting arguments (J), the coding was [IA/R+/I+/N+/A/J]. Please note that the indicator
172

YANG, NEWBY, BILL

critical assessment (C) was not suitable in this case because Student A was a starter to the discussion and no other statements/opinions were available for him to evaluate. In a second example, Student B critically evaluated Student As message with logical thinking process (C+J+) and replied by stating a contradictory viewpoint (IIA) that no one mentioned before (N+). In addition, Student Bs statement was accurate (A+), relevant (R+), and important (I+) with logical supporting arguments (J+L+), therefore it was coded [IIA/R+/I+/N+/A+/J+L+/C+J+]. After coding, we counted the total number of units of analysis that occurred in each of the phases and operations according to the coding scheme for evaluating critical thinking in computer conferencing (see Table 2). In addition, similar to what Newman, Webb, and Cochrane (1995) did, the totals for each + or indicator were counted, and a depth of CT ratio (x ratio) was calculated for each criterion (see Table 3). Both the researcher and the instructor coded all of the four online discussions in Treatments I and II and calculated the interrater reliabilities for each discussion separately. Using Miles and Hubermans (1994) formula, the interrater reliabilities for the eight coded online discussions ranged from 90.31% to 95.36%. Results and Discussion CCTST The mean scores on the precourse CCTST were 18.27 (SD = 3.88) and 18.40 (SD = 3.36) in Treatments I and II, respectively. The mean scores on the postcourse CCTST were 19.73 (SD = 4.76) and 21.00 (SD = 4.42) in Treatments I and II, respectively. The p values reveal that both the Treatment I group (t(10) = 3.07, p = .0059) and the Treatment II group (t(4) = 5.10, p = .0035) showed significant gains on the results of postcourse CCTST. These results indicate that both Treatment I and II groups improved their CT skills by the end of the semester. Asynchronous Online Discussions Table 2 consists of two partsPart A and Part Blisting the number of units of analysis which were coded in each phase and its operations for the two treatments. There were a total of 228 and 298 units of analysis in the first half of the discussion in Treatments I and II, respectively. There were a total of 216 and 205 units of analysis in the second half of the discussion in Treatments I and II, respectively. Students tended to discuss more in the first half of the semester than in the second half. This is
173

Table 2. Results of Asynchronous Online Discussions by Interaction Category for Treatment I Group and Treatment II Group
First Half of the Semester (Discussions 1 and 2) Interaction Category # of Unit of Analysis Total (%) Second Half of the Semester (Discussions 3 and 4) # of Unit of Analysis Total (%)

Part A: Treatment I Group IA IB IC ID IE IIA IIB IIC IIIA IIIB IIIC IIID IIIE IVA IVB IVC IVD IVE VA VB VC Total Part B: Treatment II Group IA IB IC ID IE 158 10 2 20 0 118 10 4 5 3 133 10 7 15 7 17 6 0 2 0 3 1 0 0 0 0 0 0 2 7 18 228 132 1 1 4 0 12 20 2 3 0 8 3 0 1 0 0 0 0 3 8 18 216

Phase I

172 (75.44%)

138 (63.89%)

Phase II

23 (10.09%)

34 (15.74%)

Phase III

6 (2.63%)

14 (6.48%)

Phase IV

0 (0.00%)

1 (0.46%)

Phase V

27 (11.84%)

29 (13.43%)

228 (100%)

216 (100%)

Phase I

190 (63.76%)

140 (68.29%)

(continued)

174

YANG, NEWBY, BILL

Table 2 (Continued)
Phase II IIA IIB IIC IIIA IIIB IIIC IIID IIIE IVA IVB IVC IVD IVE VA VB VC Total 17 30 4 5 0 7 4 0 0 0 0 0 0 6 1 34 298 51 (17.11%) 15 14 0 5 0 4 0 0 0 0 0 0 0 1 0 26 205 29 (14.15%)

Phase III

16 (5.37%)

9 (4.39%)

Phase IV

0 (0.00%)

0 (0.00%)

Phase V

41 (13.76%)

27 (13.17%)

298 (100%)

205 (100%)

likely because the students were busy studying and completing their course work toward the end of the semester, so they spent less time discussing online. On average, while each Treatment I participant contributed 40.36 units of analysis ((228 + 216) units/11 participants), each participant in Treatment II generated 100.60 units ((298 + 205) units/5 participants), respectively. The number of units of analysis, in general, is approximately proportionally equivalent to the number of messages. Therefore, these results imply that if the instructor started to teach and model CT skills at the outset of the online discussion (Treatment II) rather than in the middle of the semester (Treatment I), students seemed to be more motivated to participate, and the discussion tended to be more dynamic (100.60 vs. 40.36 units per participant in Treatments II and I, respectively).

Efficacy of teaching and modeling Socratic questioning for developing students CT skills in ADF. In Treatment I, the total units of analysis in the first half of the semester (228) were approximately the same as those in the second half (216) (Table 2, Part A). In the first half of the ADF, there
175

USING SOCRATIC QUESTIONING

were 75%, 10%, 3%, 0%, and 12% of units in Phases I, II, III, IV, and V, respectively. In the second half of the ADF, 64%, 16%, 6%, 0.5%, and 13% of the units were coded in Phases I, II, III, IV, and V, respectively (see Table 2, Part A). These data show that during the second half of the semester, the discussion tended to move toward higher phases. That is, students seemed to more critically ask and answer questions to clarify ideas and to negotiate meaning and identify areas of agreement or overlap among conflicting concepts, 2(4, N = 444) = 9.80, p = .0439. Likewise, a statistically significant difference was found when comparing the quality of interaction between the first half of the discussion in Treatment II, where Socratic questioning was also facilitated and modeled, and that in Treatment I, where no Socratic questioning was facilitated, 2(4, N = 526) = 9.77, p = .0444. These two chi-square results indicate that students who participated in Socratic dialogues, as modeled and facilitated by the instructor during ADF (the second half and the first half of the semester in Treatments I and II, respectively), had a higher quality of interactions than students who participated in an ADF where Socratic dialogues were not fostered (the first half of the semester in Treatment I). Persistence of students CT skills. In Treatment II, 64%, 17%, 5%, 0%, and 14% of units occurred in Phases I, II, III, IV, and V, respectively, in the first half of the ADF. In the second half of the ADF, 68%, 14%, 4%, 0%, and 13% of the units were coded in Phases I, II, III, IV, and V, respectively (see Table 2, Part B). The data indicate that the first half of the discussion had slightly higher percentages of units of analysis at higher phases than the second half of the discussion. This implies that when the instructor discontinued asking and modeling Socratic questions during the second half of the semester in Treatment II, the students discussion moved slightly back to Phase I. However, according to the results of the chi-square test, 2(4, N = 503) = 1.32, p = .8583, the quality of interactions in terms of the interactive and cognitive dimensions was not significantly different. That is, the quality of interaction in the second half of the discussion, where the instructor discontinued modeling and facilitating Socratic dialogues, was not significantly lower than that in the first half of the discussion in Treatment II. In addition, the percentages in higher phases in the second half of the Treatment II discussion were still higher than those in the first half of the Treatment I discussion, where students were not taught to think critically (see Table 2, Parts A and B). Table 3 also consists of Parts A and B. The number of codes and the depth of CT ratio for each of the indicators are presented.
176

Table 3. Depth of Critical Thinking by Indicator for Treatment I Group and Treatment II Group
First Half of the Semester (Discussions 1 and 2) Tag Code Number of Codes Depth of CT Ratio Second Half of the Semester (Discussions 3 and 4) Number of Codes Depth of CT Ratio

Part A: Treatment I Group R (relevance) I (importance) N (novelty) A (accuracy) J (justification) R+ R I+ I N+ N A+ A J+L+ J+L J C+L+ C+L C Total Part B: Treatment II Group R (relevance) I (importance) N (novelty) A (accuracy) J (justification) R+ R I+ I N+ N A+ A J+L+ J+L J C+L+ C+L C Total 180 2 152 3 126 3 158 8 122 1 8 28 0 1 792 97.80% 96.13% 95.35% 90.36% 115 2 118 1 66 2 62 3 49 1 2 17 0 1 439 96.58% 98.32% 94.12% 90.77% 163 7 142 6 101 24 90 22 82 12 19 12 0 4 684 91.76% 91.89% 61.60% 60.71% 155 3 151 2 96 1 93 5 88 1 4 22 0 1 622 96.20% 97.39% 97.94% 89.80%

45.13%

89.25%

C (critical assessment)

50.00%

91.30%

86.26%

88.46%

C (critical assessment)

93.10%

88.89%

Note: Depth of CT (critical thinking) ratio = (positive indicator negative indicator) / (positive indicator + negative indicator).

177

USING SOCRATIC QUESTIONING

Efficacy of teaching and modeling Socratic questioning for developing students CT skills in ADF. The depth of CT ratios in Treatment I for each indicator in the second half of the ADF discussion (see Table 3, Part A) were all higher than those in the first half of the discussion. Both the first half and the second half of the discussion had high R (relevance) (92% vs. 96%) and I (importance) (92% vs. 97%) ratios. The participants, in general, brought in relevant materials and seemed to have adopted a serious style when taking part in the discussion. There were notable differences for N (novelty) (62% vs. 98%), A (accuracy) (61% vs. 90%), J (justification) (45% vs. 89%), and C (critical assessment) (50% vs. 91%). This is possibly because of the Socratic questioning that was introduced and taught during the second half of the discussion. Students seemed to take more thought before posting their messages. As a result, the number of negative statements for N, A, J, and C was reduced; thus, the depth of CT ratios for these indicators greatly increased. The chi-square results, 2(1, N = 1306) = 50.77, p < .0001, statistically affirm this claim. A similar result was found when comparing the depth of CT ratios between the first half of the discussion in Treatment II, where students participated in Socratic dialogues, and that in Treatment I, where students did not participate in Socratic dialogues, 2(1, N = 1476) = 53.76, p < .0001. Therefore, it is concluded that students CT skills improved after participating in Socratic dialogues, as modeled and facilitated by the instructor during ADF.

Persistence of students CT skills. Upon inspecting the depth of CT ratios in the second half of the Treatment II, it is evident that the depth of CT ratios did not decline when compared with those for the first half of the Treatment II discussion, R (relevance) (98% vs. 97%), I (importance) (96% vs. 98%), N (novelty) (95% vs. 94%), A (accuracy) (90% vs. 91%), J (justification) (86% vs. 88%), and C (critical assessment) (93% vs. 89%). This claim was verified by the results of the chi-square test, 2(1, N = 1231) = .28, p = .5935. In addition, the depth of CT ratios in the second half of the Treatment II discussion were much higher than those in the first half of the Treatment I discussion, where students were not taught to think critically (see Table 3, Parts A and B). Therefore, the results imply that without the instructors further facilitation to improve students CT skills, the discussion maintained about the same quality as that facilitated by the instructor.
178

YANG, NEWBY, BILL

Conclusions CT is an important issue in higher education, and educators have continued to focus on the development of CT in students (Beyer 1995; Ennis 1985; Hilgenberg and Tolone 2000; Paul and Heaslip 1995; Taube 1997). This study focused on investigating the effects of using Socratic questioning to enhance students CT skills in an ADF environment. The empirical results of this study indicate that with appropriate course design and instructional interventions, CT skills can be cultivated and maintained in ADF. This is probably because an ADF affords students the time for thoughtful analysis, composition, negotiation, and reflection as their discussion of an issue evolves and allows instructors to model, foster, and evaluate the CT skills exhibited during the discussion. According to the experience from this study, CT skills are difficult to foster, but not impossible. If students are asked to experience, explore, and test their ways of thinking, they will find it to be substantial work; however, positive gains in students CT skills and attitudes provided evidence that teaching and learning CT is worth the effort. Therefore, it is encouraged that distance educators and courseware designers take the challenge to create an active learning environment where CT is valued and where students are motivated and supported in their attempts to think critically via an ADF.

References Anderson, T., and D. R. Garrison. 1995. Critical thinking in distance education: Developing critical communities in audio teleconferencing context. Higher Education 29 (2): 183199. Angelo, T. A. 1995. Beginning the dialogue: Thoughts on promoting critical thinking: Classroom assessment for critical thinking. Teaching of Psychology 22 (1): 67. Beyer, B. K. 1995. Critical thinking. Bloomington, IN: Phi Delta Kappa. Bullen, M. 1998. Participation and critical thinking in online university distance education. Journal of Distance Education 13 (2): 132. Cooper, J. L. 1995. Cooperative learning and critical thinking. Teaching of Psychology 22 (1): 78. Dillon, J. T. 1988. Questioning and teaching: A manual of practice. New York: Teachers College Press. Duffy, T. M., B. Dueber, and C. Hawley. 1998. Critical thinking in a distributed environment: A pedagogical base for the design of conferencing systems. In Electronic collaborators: Learner-centered
179

USING SOCRATIC QUESTIONING

technologies for literacy, apprenticeship, and discourse, ed. C. J. Bonk and K. S. King, 5178. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Ennis, R. H. 1985. A logical basis for measuring critical thinking skills. Educational Leadership 43 (2): 4448. Facione, P. A. 1990a. The California Critical Thinking Skills Test (CCTST): Form A (1990) (1992). Millbrae, CA: California Academic Press. . 1990b. Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Research findings and recommendations. ERIC, ED 315423. . 1992. The California Critical Thinking Skills Test (CCTST Form B (1992). Millbrae, CA: California Academic Press. . 1998. Critical thinking: What it is and why it counts. 1998 Update: California Academic Press. Available online at http://www.calpress.com/pdf_files/what&why.pdf Flammer, A. 1981. Towards a theory of question asking. Psychological Research 4:407420. Garrison, D. R. 1993. Quality and access in distance education theoretical considerations. In Theoretical principles of distance education, ed. D. J. Keegan. New York: Routledge. Gunawardena, C., C. Lowe, and T. Anderson. 1997. Analysis of global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research 17 (4): 395429. Hettinger, G. 1995. Raising the level of the debate: The effects of computer mediated communication on group dynamics and critical thinking skills. ERIC, ED 383300. Hilgenberg, C., and W. Tolone. 2000. Student perceptions of satisfaction and opportunities for critical thinking in distance education by interactive video. The American Journal of Distance Education 14 (3): 5973. Jeong, A. 2000. Bulletin boards support critical thinking in small group discussions. University of Wisconsin-Madison, Learning Technology and Distance Education. Available online at http://ltde.tripod.com/ groupware/bbcriticalthinkingsurvey.htm Kerry, T. 1987. Classroom questions in England. Questioning Exchange 1 (1): 3233. King, A. 1990. Enhancing peer interaction and learning in classroom through reciprocal questioning. American Educational Research Journal 27 (4): 664687.
180

YANG, NEWBY, BILL

. 1995. Designing the instructional process to enhance critical thinking across the curriculum: Inquiring minds really do want to know: Using questioning to teach critical thinking. Teaching of Psychology 22 (1): 1317. Maiorana, V. P. 199091. The road from rote to critical thinking. Community Review 11 (12): 5363. McDade, S. A. 1995. Case study pedagogy to advance critical thinking. Teaching of Psychology 22 (1): 910. Miles, M. B., and A. M. Huberman. 1994. Qualitative data analysis: An expanded sourcebook. Thousand Oaks, CA: Sage. Newman, D., B. Webb, and C. Cochrane. 1995. A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Interpersonal Computing and Technology: An Electronic Journal for the 21st Century 3 (2): 5677. Paul, R. W. 1993. Critical thinking: What every person needs to survive in a rapidly changing world. Santa Rosa, CA: Foundation for Critical Thinking. Paul, R. W., and P. Heaslip. 1995. Critical thinking and intuitive nursing practice. Journal of Advanced Nursing 22 (1): 4047. Taba, H. 1966. Teaching strategies and cognitive functioning in elementary school children. Cooperative Research Project, No. 2404. San Francisco: San Francisco State College. Taube, K. T. 1997. Critical thinking ability and disposition as factors of performance on a written critical thinking test. Journal of General Education 46 (2): 129164. Walker, S. A. 2004. Socratic strategies and devils advocacy in synchronous CMC debate. Journal of Computer-Assisted Learning 20 (3): 172182.

181