‘Tohoku J. Exp. Med., 2006, 210, 373-381 Development of a Questionnaire to Evaluate Patient Satisfaction with Medical Encounters Yousuke Takemura,’ Jia Liu,” Reiko Arsumt' and Tsukasa Tsuba! ‘Department of Family Medicine, Mie University School of Medicine, Mie, Japan >The Second Affiliated Hospital of Dalian Medical University, Dalian, China Taxemurs, ¥., Liv, J., Atsumt, R. and Tsups, T. Development of a Questionnaire to Evaluate Patient Satisfaction with Medical Encounters. Tohoku J. Exp. Med., 2006, 210 (4), 373-381 — Given that a medical practice exists for patients, it is worth deter- mining the degree of patient satisfaction with regard to the medical practice’s quality of care. Considering the importance of noticing patient satisfaction and its influence on clini- cal care, intense evaluation of a questionnaire’s validity and reliability is essential. The purpose of this study was to establish a valid and reliable self-administered scale to me sure patient satisfaction with fewer questions than previous scales applicable in medical settings in Japan, A qualitative method was used to develop and revise content-valid ques- tion items of the questionnaire. Factor analysis revealed five subscales among, 12 items: “overall satisfaction”, “complete examination”, “patient centeredness”, “examination time”, and “whole person care”. A test of internal consistency was also assessed. The concurrent validity was assessed to evaluate the association between the score of the cur- rent questionnaire and that of the visual analogue scale or other questionnaire. Agreement between two sets of score, scores just after consultation and 30-50 min after that, was assessed t0 evaluate the test-retest reliability of each question item. ‘The results revealed satisfactory validity, including the content and concurrent validity, internal consistency (Cronback alpha = 0.77-0.85), and the test-retest reliability of our questionnaire (Kappa score = 0.61-0.71). In conclusion, we have developed a short-form self-administered patient satisfaction questionnaire applicable in Japan, with acceptable validity and reliabil- ity. This questionnaire may contribute to conducting further studies related to patient sub- jective responses to encounters in Japanese medical settings, and evaluating and improving the clinical interview skills of medical students or trainees in medical education. interviews; patient satisfaction; questionnaire; Japan © 2006 Tohoku University Medical Press Given that a medical practice exists for satisfaction is important as a predictor of health patients, it is worth discovering the degree of patient outcomes. It is evident that higher patient satis- satisfaction in a medical practice to increase the faction is associated with improved health status quality of care provided. Furthermore, patient (Woolley et al. 1978; Linn and Greenfield 198: Received July 31, 2006; revision accepted for publication October 20, 2006. Correspondence: Yousuke Takemura, Department of Family Medicine, Mie University School of Medicine, 2174 Edobashi, Tsu, Mie 514-8507, Japan, ‘e-mail: yousuke@clin.medie.mie-u.ac.jp Drs. Takemura and Liu contributed equally to the article. 373374 Y. Takemura et al Fitzpatrick and Hopkins 1983; Patrick et al. 1983; Deyo and Diehl 1986; Headache Study Group of University of Western Ontario 1986; Fitzpatrick et al. 1987, Bradley aud Lewis 1990; Hall et al 1990). Questionnaires are the primary means of, measuring patient satisfaction in a medical set- ting. However, it is still difficult to measure a person's subjective satisfaction using these ques- tionnaires, as there are a limited number of such, scales (Hulka et al. 1970; Wolf et al. 1978; DiMatteo and Hays 1980; Ware et al. 1983; Roter et al. 1987; Hall and Dornan 1988; Baker 1990; Tamblyn et al. 1994). Further, not all question- naires have been evaluated from the standpoint of their validity or reliability. A questionnaire must be reliable; that is, the random error of response must be minimized so that a consistency of mea- surement is achieved. The questionnaire must also be valid; that is, it must be a true measure of what it purports to measure and must not be sub- ject to bias. Considering the importance of notic- ing patient satistaction and its intluence on clini- cal care, an intense evaluation of a questionnaire’s validity and reliability is essential. Otherwise, the quality of care based on questionnaires could lapse rather than improve. The translated version of the Medical Interview Satisfaction Scale (MISS) is the only published multi-item scale ‘measuring patient satisfaction in medical encoun- ters in Japan (Minowa et al. 1995). However, the scale has been examined only with regard to its internal consistency. The associations between the measured patient satisfaction and the physi- cian’s views of the encounter have been investi- gated, and a negative or insignificant positive association was found between them. The face validity of the questionnaire was assumed without its back translation, which is supposed to be done to confirm the face validity for a translated ques- tionnaire. Furthermore, the original scale was developed in the United States (Wolf et al. 1978). Since the Japanese culture is different from that of western countries, where most patient satisfaction questionnaires have been developed, a Japanese- derived questionnaire is needed to investigate patient satisfaction in Japan. It must also be con- sidered that a long-form questionnaire would decrease the response rate, as it would be too long for outpatients who visit the clinic for only a short time. ‘The majority of questionnaires have many ‘questions that require a relatively large amount of time to complete compared with the current ques- tionnaire. The short-form patient satisfaction questionnaire is more suited to medical communi- cation research, especially for research done in busy outpatient cl The purpose of this study was to establish a valid and reliable scale to measure patient satis- faction applicable in medical settings in Japan with fewer questions than the previous scales. This questionnaire development followed four steps: (1) a review of published questionnaires, (2) development and revision of question items by qualitative work, (3) selection of question items by principal components and factor analyses, and (4) the evaluation of the questionnaire’s validity and reliability. Sunrects ANp MetHons em development ‘A review of published satisfaction questionnaires ‘was undertaken to identify question items (Hulk et a. 1970; Wolf et al. 1978; DiMatteo and Hays 1980; Ware et al. 1983; Roter et al. 1987; Hall and Dornan 1988; Baker 1990; Tamblyn etal. 1994). We also invited five Japanese experts witha good deal of knowledge or expe- rience regarding communication or patient satisfaction rescarch or teaching to join the item development group. ‘These five persons included two family physicians, a general intemist, a registered nurse, and a patient. They were asked to suggest additional questions to identify patient satisfaction. We compared these items identified during our qualitative work with all those previously identified. We developed a bank of questions to enable us to produce multi-item scales, which are more reliable than single questions. Sixteen positively and negatively ‘worded question items in Japancce Were selected for the development of this questionnaire. ‘A Delphi process was then used to gain consensus regarding each question among the five experts (Jone and Hunter 1995). The Delphi process isa group facilitation technique that is interactive and multistage, designed to transform opinion into group consensus. This process allowed us to evaluate the content validity of the ques- tionnaire, The five experts were invited to each roundPatient Satisfaction Questionnaire 375 held in the Department of Family Medicine, Mie University School of Medicine on November 2003. These experts ranked their agreement with each item of the questionnaire in the first round. ‘The rankings were summarized and inchuded in repeat version ofthe ques- tionnaire. Inthe second round, the experts were asked t0 rewrite or remove questions that were confusing, ambig- uous, oF might produce skewed responses. ‘They then re-ranked their agreement with each statement in the questionnaire at this round, with the opportunity to change theie score in view of the group's response. ‘The re-rankings were summarized and assessed with regard to the degree of consensus. Since an acceptable degree of consensus had not yet been obtained, the third round ‘was repeated. In the third round, an acceptable degree of consensus was finally obtained and the process was deemed complete. Inthe draft of the patient satisfaction questionnaire, two items regarding general satisfaction and 16 items for specific aspects of satisfaction were distributed to 12 Patients; this draft was revised minimally to correct some wording to facilitate understanding. A balanced five- point Likert-type scale was used for the responses to the question items (Likert 1932), ‘These were labeled “strongly agree”, “agree”, “uncertain”, “disagree”, and “strongly disagree”. For analytical purposes, zero, one, two, three, and four were assigned to these responses if the questions were positively worded, and the reverse onder if negatively worded. This scaling method is rela- tively easy to complete and has been employed in many surveys. Ware etal. (1982) have explained the advantag- of its use fora patient satisfaction questionnaire. Validity and reliability Patients who visit the outpatient clinic of the Department of Family Medicine, Mie University School of Medicine Hospital in Japan, are primarily adults over the age of 15 years and with common diseases like hypertension, hypercholesterolemia, diabetes, depres- sion, and so on. ‘The patients come from a wide range of ages and social class, from both rural and urban areas. From March to November of 2004, 436 new patients (male ; female = 1 : 1, all Japanese) visiting the outpa- tient facility for the first time were recruited to the study with written informed consent to fill out this patient sat- isfaction questionnaire following consultations with their Physicians. A total of 43 fifth-grade medical students hhaving family medicine clinical clerkships, nine family medicine residents, and two faculty members of the Department of Family Medicine, Mie University School of Medicine Hospital, saw these patients. Medical stu- dents usually saw one patient, and residents or faculty members saw more than 30 of these palicuts. Patients who were too sick to complete the questionnaire, unable to read it, unwilling to stay after the doctor visit, or did not give consent to join the study were all excluded. The final response rate was 90%. The directions were explained to the patients by one of two research assis- tants; they answered patient questions such that all ques- tions about the questionnaire or study were answered. ‘The Research Ethical Committee of the Mie University ‘School of Medicine approved this stidy Fewer than 5% (n = 24) of the questionnaires were incomplete, resulting in a study population of 412. The selection of question items was guided by the findings of principal component analysis and factor analysis, with the use of a Varimax rotation. In this method, questions are picked out that tend to be answered in a similar fash- jon. In this way, the different factors or subscales that influence patient satisfaction can be identified, and the ‘homogeneity of the questions within each subscale deter- mined, This process contributes to finding subscales for the question items and therefore confirming the construct ‘alidity of the questionnaire. Principal component analy- sis was used to identify the number of factors or sub- scales underlying each patient satisfaction subscale, The eigenvalue limit for the principal component analysis ‘was set at one, We omitted from the factor analysis the ‘wo questions relating to overall satisfaction, as we antic- ‘pated that all questions would tend to load with these two factors underlying the general component. Principal factor loadings in the factor analysis were defined as those greater than 0.7. Score distribution characteristics provide an impor- tant indication of the variability in responses. Sufficient variability is necessary for scales to yield meaningful information for quality assessment. ‘The authors exam- ined the mean, standard deviation, skewness, kurtosis, and range of all question items, including the overall sat- isfaction seales, The reliability of the questionnaire containing the overall satisfaction subscales was assessed using a test of internal consistency. ‘This testis frequently employed in questionnaire development. Internal consistency was ‘evaluated for total items and items of each subscale of, the questionnaire found by factor analysis using Cronbach alphas. ‘The inter-correlation matrix of the subscales was analyzed by calculating the Pearson corre-316 lation coefficients. Where a Cronbach alpha exceeds its correlation with other subscales, there is evidence of ‘unique reliable variance measured by the subscale. ‘Three hundied aud forty paticuts were asked to complete the visual analogue scale and the Japanese ver- sion of the MISS in addition to our questionnaire (Wolf et al. 1978; Minowa et al. 1995). ‘The Japanese version of the MISS is the only published multi-item patient sat- isfaction scale used in a medical encounter setting in Japan, but it has limited validity and reliability, as men- tioned above (Minowa et al. 1995). The visual analogue scale is often used to measure patient satisfaction in research because of its convenience of use (Ahlsioe etal 1984). The concurrent validity was assessed to evaluate the association between the total score or each subscale score of the current questionnaire and that of the visual analogue seale of the Japanese version of the MISS using the Pearson correlation coefficient. ‘Among the subjects, 43 patients were asked by a research assistant to fill out the same questionnaire again ‘about 30 - 50 min after finishing their account of the visit in order to assess the test-retest reliability of this measur- ing instrument, ‘The test-retest method 1 also often used Y. Takemura etal to estimate reliability. The kappa value was used to eval- uate the test-retest reliability of each question item. The kappa statistic provides an indication of exact agreement between two sets of scores. It algo controle for chance agreement between ratings. Kappa values greater than 0.75 indicate strong agreement beyond chance and those between 0.40 and 0.74 reflect fair fo good agreement. All data were stored and analyzed using SAS/STAT version 9.1 software (SAS 2004) on a DELL computer (DELL Inc., Round Rock, TX, USA). Resutts ‘The Delphi process did not remove any of the 16 question items since all the question items had a median score of more than five among nine (1: disagree, 9: agree), with a consensus or range of three or less until the third round. However, all of the question items were revised to obtain con- sensus among the experts. To decide upon the number of factor(s), the ‘eigenvalue was calculated by principal component analysis, We designated the number of factors as ‘Taste 1, The selection of question items guided by the factor analysis (n = 412). Question item Factor! Factor2 Factor3 Factor4 a 0.79219 0.19452 0.21044 0.144 @ 0.82863 0.14726 0.14729 0.26461 Qs 0.52676 0.42185 0.20548 0.03328 ra 0.58212 09356 0.03127 0.37494 es 0.21747 0.13561 ~0,08007, 0.83035 26 0.18569 0.23817 0.09571 0.82738 q 0.43858 27145 0.05457 0.53048" eB 0.0607 0.14978 Os 0.01945 e 0.12206 0.04528 0.89353 0.0393 Quo 0.14982 0.07325 0.87565 0.01173 Qu 0.61432 0.41401 0.01353 0.17687 2 0.42426 0.63168 0.11503 0.36373 3 0.21779 O77405 0.15492 0.27299 ais 0.23915, 0.72842 0.1682 0.25864 ais 0.28726 0.7745 0.00818 0.04888 Que 0.62075 041631 0.12669 0.24019 Factor analysis with Varimax rotation was employed to reveal principal factor loadings. Factor loadings greater than 7.0 were defined as principal factor loadings and are circled with a dotted line, ‘Two question items for overall satisfaction (QG1, QG2) were omitted from this analysis.Patient Satisfaction Questionnaire 377 four since the fourth eigenvalue was around one. ‘The Varimax factor rotation identified four factors in the questions, as shown in Table 1. These four factors or subscales explain 68.9% of the variance in the 16 items, In this step, six question items were excluded that were not loaded highly onto any factor, and the two overall satisfaction ques- tion items that were omitted in the principal com- ponent analysis and factor analysis were added. This resulted in a 12-item questionnaire presented in Table 2. These 12 items were again evaluated for construct validity using factor analysis with a Varimax rotation, which led to the same subclass- es, The authors judged whether these subscales were coherent and indicated the issues important to patients, and also decided names for each sub- scale in qualitative fashion. ‘These subscales were named “complete examination” (two questions), “patient centeredness” (three questions), “exami- nation time” (three questions), and “whole person care” (two questions). Table 3 shows the deseri Most questions or subscales were negatively skewed, indicating distributions with more posi- tive ratings of the questionnaire. The full range of possible scores was observed for most questions or subscales, except for “patient centeredness”. able 4 shows the Cronbach alpha for total score, “overall satisfaction”, “complete examina- tion”, “patient centeredness”, “examination time”, and “whole person care”. These results suggest that the subscales of the questionnaire are inter- nally consistent under the conditions of this study, as they satisfy the minimum criteria of 0.7 for internal consistency. Table 5 shows the inter- correlation matrix of these subscales. On the whole, these subscales inter-correlate positively and significantly. “Patient centeredness” had the lowest inter-subscale correlations, although these were still significant. Furthermore, each subscale’s Cronbach alpha substantially exceeded its correlation with all other scales. This suggests that they represent fairly discrete aspects of patient satisfaction, though there is sufficient inter-correlation between the subscales to suggest that they are also aspects of a more global value. Table 6 identifies the significant association between the current 12-item questionnaire, the Taste 2. Items of each question, Questions: Items Overall satisfaction 1(QG1) Lam very satisfied with the medical consultation that I had today. 2(QG2)__ The medical consultation that I had today has better point(s) than those of other doctors. Other questions 1(Q1) This doctor examined me carefully and completely. 2(Q2)__ This doctor examined me perfectly. 3(Q5)__ This doctor knows almost everything about me, 4(Q6) Think that this doctor really knows how I think. 5(Q8)__ The ime forthe medical consultation with me was not long enough to deal wit everyting want 6(Q9) wonder if this doctor could have spent alittle longer time with me.” 7(QIO) The time for the medical consultation with me was @ litle bit t00 short.” 8(Q13) This doctor listened to my ideas. 9(QI4) This doctor listened to what I want him/her to do. 10 (QIS) __I think that this doctor is very honest ‘The original questionnaire was written in Japanese. ‘The above items were translated from the original questionnaire by the authors “These items are negatively worded, and each of them was scored in the reversed order.378. Y. Takemura etal. ‘Tante 3. Descriptive statistics for each question to indicate the variability in responses (n = 412). Question items Mean so. Skewness __Kurtosis Observed range Overall satisfaction ol 134 038 0.05 18 ai 3.20 on “071 on 04 a2 294 om 0.13 ~0.16 14 ‘Complete examination 6.00 132 029 02s 18 a 3.16 064 0.48 049 4 @ 2.84 078 -027 0.13 0-4 ‘Whole person care 3.90 LSI 0.17 095 08 Qs uw 0.86 027 0.58 04 06 219 082 0.12 039 04 Examination time 801 219 051 17 on es 2.60 0.88 062 0.63 04 @ 264 083 0.49 0.46 04 Qo 27 0.79 0.18 157 04 Patient centeredness 8.80 161 0.06 059 312 a3 3.00 059 028 oa 14 au 287 0.67 0.12 017 14 ais 2.93 0.64 0.10 0.15 4 sp, standard deviation, ‘Taste 4, Internal consistency evaluated by the Discussion value of Cronbach alphas (n = 412), Subscales Cronbach alpha Total score 0.85, Overall satisfaction 078 Complete examination 0.84 Patient centeredness om Examination time 0.85: Whole person care 0.80, visual analogue scale, and the Japanese version of the MISS, and this association indicates the satis- factory concurrent validity of the questionnaire. Table 7 shows that kappa values for each question item were greater than 0.6, and that the test and retest scores were highly correlated. It shows that the questionnaire is reliable and repro- ducible. ‘The short-form questionnaire appears to be a valid and reliable tool to measure patient satisfac- tion in a medical setting. The questionnaire attempts to measure satisfaction with a particular individual encounter as distinct from general att- tudes toward physicians or health care service. Because items in the questionnaire refer directly to a specific patient-physician interaction, it is likely to be more sensitive to actual differences in care of the encounter. The inter-subscale correlation was lower for “patient centeredness” than for the other sub- scales. This correlation is in contrast with the importance of patient centeredness in the overall satisfaction with a practice shown in this study. This contradiction might indicate that the subscale identifies a relatively distinetive aspect of patient satisfaction compared with the other subscales. The advantage of this patient satisfaction questionnaire is that the validity, including the content and concurrent validity, internal consis-Patient Satisfaction Questionnaire 379 ‘Tante 5. Inter-correlation matrix of the questionnaire subscales to show reliable variance measured by the subseale (n = 412), Fees Overall Complete Whole person Patient Examination satisfaction examination care ccenteredness time (Overall satisfactrion 1.00000 ‘Complete examination 0.69175 1.00000 Whole person care 0.52628 0.54269 1.00000 Patient centeredness 0.26151 0.29983, 0.25299 1.00000 Examination time 0.37960 0.45574 0.46329 0.06813 1.00000 ‘The inter-correlation matrix of these subscales was analyzed by calculating Pearson correlation coefticients, ‘All correlations in this table Were p < 0.0001 except for this correlation (p = 0.15) ‘Taste 6. The relationship between total score fr subscales and VAS or MISS to assess the concurrent validity (n = 340). Correlation Correlation Subscales with VAS* with MISS* Total score 0.69 079 Overall satisfaction 0.64 0.60 Complete examination 0.62 0.66 Patient centeredness 0.37 0.52 Examination time 0.40 0.43 Whole person care 01 0.69 VAS, visual analogue scale; MISS, Medical Interview Satisfuction Scale, All values were < 0.0001. *Correlation was assessed with the Pearson correlation coefficients tency, and the test-retest reliability have been established. The majority of questionnaires of this type have not been investigated with regard to all of these factors. In the current study, quali- tative methods like the Delphi method were used to evaluate the content validity. Furthermore, the large number of subjects used to evaluate the validity and reliability increases the power of the analyses. ‘The number of subjects used to assess the test-retest reliability of this questionnaire may not be thought to be sufficient. However, the sample ‘TaBLE 7. Kappa scores for each question to | assess the test-retest reliability (n = 43) Question tems Kappa score Overall satisfaction 073 Qi 0.75 Q 0.83 Complete examination 0.59 Qi 061 a 0.66 Whole person care 0.63 5 0.84 se 0.65 Examination time = en 068 go 0.96 uo 093 Patient centeredness 0.66 13 079 ais 083 ais on ‘It could not be calculated because the Tow number was not equal to the column number, size of 43 for this aspect of the work is compara- ble to that of other similar studies and was judged to be satisfactory. Since the questionnaire was developed by Japanese experts in a Japanese medical setting,2 380 Y. Takemura etal the generalizability of the questionnaire in Japan is greatly increased. Indeed, in this study, the fac- tor analysis with Varimax rotation of the answers, to the Japanese version of the MISS revealed five subscales: namely, “general satisfaction”, “physi- cian’s explanation”, “patient centeredness”, “hur- riedness of encounter”, and “physician’s attitude”, Both our questionnaire and the Japanese version of the MISS have a subscale for “patient centered- ness”, The “examination time” subscale in our ‘questionnaire is similar in content to the “hurried- ness of encounter” subscale in the MISS. However, while “complete examination” and “whole person care” subscales were found t0 be important in the current questionnaire, this impor- tance was not revealed by the analysis of the Japanese version of the MISS. Another advantage of this questionnaire is that it has fewer question items than previous questionnaires, which enhances the patient response and diminishes patient burden. As such, the questionnaire is practical. ‘The present results, however, should still be seen as preliminary and further study might be needed, since the study has some limitations. First, it is still difficult to assess patient satisfac- tion through questionnaires, although the validity and reliability of the questionnaire were evaluated and found to be acceptable. However, there is no way to reveal patient satisfaction except by ques- tionnaire, ‘The other limitation is that it cannot be. applied to a patient who does not speak Japanese since it is written in Japanese. This questionnaire is the first patient satisfaction questionnaire devel- oped in Japan for which the validity and reliability have been investigated. The distribution of some of the scores is moderately skewed, as shown in Table 3. The distribution of patient satisfaction may result from patients’ reluctance to use the lower points of the scales. Because scaling is rel- ative, however, a skewed distribution of scores may not have any real consequence for determin- ing reliable differences in satisfaction levels. In conclusion, a short-form self-administered patient satisfaction questionnaire with acceptable validity and reliability was developed. It appears to be a useful measure of patient satisfaction, eas- ily administered, and applicable in Japan. ‘This questionnaire may contribute to further research, related to patient’s subjective response to encoun- ters in a Japanese medical setting. Furthermore, it should be beneficial in evaluating and improving the clinical interview skills of medical students or trainees in medical education. These effects could potentially lead to greater patient satisfaction and ultimately improved health care. Acknowledgments ‘The authors are grateful to the patients who gave up their valuable time to participate in this study. ‘The authors also wish to express their grati- tude to the five experts for their great contribution to the Delphi method to validate the questionnaire. Finally, I would like to thank Prof. Debra Roter, the Department of Health, Behavior, and Society, Johns Hopkins University Bloomberg School of Public Health for her warm-hearted English editing and ‘comments to make this paper valuable, References Ablsioe, B., Briton, M., Murray, V. & Theorell, T. (1984) Disablement and quality of life after stroke. Stroke, 18, 886-890. Baker, R. (1990) Development of a questionnaire to assess tients” satisfaction with consultations in general practice. Br.J. Gen, Pract 40, 487-49, Bradley, C. & Lewis, K. (1990) Measures of psychologic well-being and treatment satisfaction developed from responses of people with tablet treated diabetes. Diaberc Med, 7, 445-451 Deyo, R.& Diehl, A. (1986) Patont satisfaction with medical ‘cate fo low back pain. Spine, 11, 28 30. DiMatteo, MLR. & Hays, R. (1980) The significance of patiens* perceptions of physician conduct A study of patient sti Faction in a family practice center. J. Community Health, 6, 18-34 Fiupatrick, R., Bury, M., Frank, A. & Donnelly, T. (1987) Problems inthe assessment of outcome in aback pan ci ie. Int Disabil Sud, 9, 165. Fitzpatrick, RM., Hopkins, A, & Harvard-Wats, 0, (1983) So- cial dimensions of healing: A longitudinal study of out comes of medical management of headache. Soc. Sci. Med.,17, 301-510. Hall, JA" & Dornan, M.C. (1988) Meta-analysis of satisfaction ‘with medical eare: Description of researeh domain and analysis of overall satisfaction levels. Soc. Sei. Med, 27, 637-644, Hall, J.A., Feldstein, M, Fretwell, M.D, Rowe, J.W.& Epstein, ‘AM. (1990) Older patients’ health status and satisfaction With medical care in an HMO population. Med. Care, 8, 261-210. Hasson, F, Keeney, 8 & McKenna, H. (2000) Research guide- lines forthe Delphi survey technique. J. Adv. Nur. 32, 1008-1018, Headache Study Group of Unversity of Westem Ontario (1986)Patient Satisfaction Questionnaire 381 Predictors of outcome in headache patients presenting t0 family physicians. Headache, 26, 285-294, Hulka, B.S., Zyeanski, §.J,, Cassel, J.C. & Thompson, SJ. (1970) Seale forthe measurement of sttudes toward phy= isians and primary medica care. Med. Care, 8, 428-436 Jone, J. & Hunter, D. (1995) Consensus methods for medical and services research, BMJ, 311, 376380. Likert, R (1932) A technique forthe measurement of atiudes ‘Arch. Psychol. 140, 44-53. Linn, L. & Greenfield, . (1982) Patient suffering and patient Satisfaction among the chronically il. Med. Care, 20, 425-431, Minowa, ¥, Kashi, A, & Watanabe, R (1995) The validity and reliability of the Medical Interview Satisfaction Scale Dilon-fj-Shinpo, 3736, 30-33. in Japanese) Patrick, D.L, Serivens,E. & Charlton, .RH. (1983) Disability ‘and patient satisfaction with medical care. Med. Care, 21, 1062-1075. Rote, DL, Hall, .A. & Katz, NR. (1987) Relations between physicians’ behaviors and analogue patients’ satisfaction, Fecall and impressions, Med, Care, 2, 437-481 SSAS Institute, Inc, (2008) SASISTAT 9.1 user's guide, volumes 1-7, Cary, NC, SAS Tnsiute, In. ‘Tumhlyn, R. Renareya, 8, Snell I. McLeod. P. Schnarch.B, ‘& Abrahamowicz, M. (1994) The feasibility and value of ‘using patient satisfaction ratings to evaluate internal medi- cine residents. J. Gen. Intern. Med.,9, 146-152, Ware, LE. Ie, Snyder, MK., Wright, WR. & Davies, AR, (1983) Defining and measuring patient satisfaction with redical care. Eval. Program Plann. 6, 247-263. Wolf, MAH, Puttam, SM, James, $.A. & Stiles, WB. (1978) ‘The medical interview satisfaction seale: Development of a scale to measure patient perceptions of physician behavior. J. Beha. Med, 1, 391-401 Wooley, FR., Kane, RL. Hughes, C.C. & Wright, D.D. (1978) "The effects of doctor-patient communication on satisfaction sind outcome of care. Soc. Si. Med. 12, 123-128.