1

Methods

Study Design and Setting

This was a cross-sectional survey-based study to gain an insight into the applicants’

experience with virtual interviews for the radiology residency program in Saudi Arabia and

Bahrain. The survey was designed using the QuestionPro survey software (Seattle, WA, USA).

The survey commenced on May 1, 2020 and was open to respondents for 14 days.

Study Participants

The study involved the applicants to the radiology residency program in different regions

of Saudi Arabia (including Riyadh, Eastern Province, Mecca, and Najran) and Bahrain. A

personalized e-mail with a link to the online survey was sent to a total of 183 applicants. Each

applicant received a unique link so that the survey could not be filled more than one time from the

same link. This ensured that the survey would not be compromised by duplicate responses. A

reminder message was sent 1 week later.

Content of the Questionnaire

A cover letter describing the purpose of the study and informing the applicants of their

voluntary nature of participation was provided along with the survey. Participants were encouraged

to contact the research investigator for any queries, using the contact information provided. There

was no personal identification data requested.

The survey was comprised of 35 questions and was completed in approximately 5 minutes.

These questions covered the following areas: (1) demographic information, (2) application and

nomination information, and (3) applicants’ experience with virtual interviews. A 5-point Likert-

scale (strongly disagree, disagree, neutral, agree, and strongly agree) was used to record data about
 2

the applicants’ experience with the virtual interviews, including: (1) potential benefits, (2)

potential difficulties, (3) general impression, and (4) preferences for the type of interview.

The face and content validity of the questionnaire was verified by a panel of experts who

assessed the relevance and appropriateness of questions. A pilot study was then conducted, with a

group of applicants to different residency programs, to assess the clarity of the questions and the

time to complete the survey. No changes were made after the pilot study. The internal consistency

of the questionnaire was evaluated using Cronbach alpha reliability test. The coefficient for the

overall questionnaire was 0.89 indicating a good internal consistency. The coefficients for the

benefits (0.73), difficulties (0.39), impression (0.88), and preferences (0.77) subscales were also

estimated. The relatively lower coefficient for the difficulty’s subscale could be attributed to the

heterogeneity of its items.

Statistical Analysis

The data obtained were compiled using QuestionPro platform and analyzed using IBM

SPSS for Windows, Version 25 (IBM Corp., Armonk, NY, USA). Categorical variables are

presented as percentages and frequency distributions. Tableau Desktop (Tableau, WA, USA) was

used to create the diverging stacked bar charts for the Likert-type questions. The overall score for

the Likert-type questions was calculated after reverse scoring of all negatively worded questions.

The overall score was compared based on different demographic and application information using

Student’s t-test or ANOVA test, as appropriate. The significance level was set to α = 0.05.

Ethical Considerations

The study was approved by the Institutional Review Board at the Imam Abdulrahman Bin

Faisal University. All information pertaining to the survey was provided in the cover letter.

Therefore, the completion of the survey was considered as acknowledgment of informed consent.
 3

Results

Characteristics of Applicants

A total of 143 applicants to the Diagnostic Radiology Training Program have completed

the survey (an overall response rate of 78.1%), including 72 men (50.3%) and 71 (49.7%) women.

Overall, 92 (64.3%) applicants were graduates of the class of 2018, and only 19 (13.3%) applicants

graduated in 2017 or before. The majority (92.3%) of applicants were graduates from medical

schools in Saudi Arabia. Moreover, only 11 applicants were graduates from other countries,

including Bahrain, China, Egypt, Jordan, Netherlands, Poland, Turkey, and the United States of

Emirates (Table 1).

Application and Nomination Information

Most (72.7%) respondents were first-time applicants to the postgraduate training

programs of the Saudi Commission for Health Specialties. In addition, the majority (89.5%) of

applicants had the radiology residency program as their first choice in the rank order list in the

application process, while only five (3.5%) applicants it as their third choice or more.

Approximately half (48.3%) of applicants were nominated to the training centers in Riyadh, while

other applicants were nominated to Jeddah (23.1%), Eastern Province and Bahrain (18.9%), Mecca

(7.7%), or Najran (2.1%). Overall, 54.5% of applicants attended 6–10 interviews while others

attended 1–5 (18.2%) or >10 interviews (27.3%). In addition, only 10 (7.0%) reported having

previous experience with virtual interviews (Table 1).

 4

Virtual Interview Experience

The applicants’ responses regarding their experience with virtual interviews were very

positive and are summarized in (Figure 1) and (Table 2). The majority of applicants agreed on the

potential benefits of virtual interviews. For example, most (67.9%) applicants agreed or strongly

agreed that the virtual interview saved them the expenses of traveling to the interview centers. In

addition, most applicants agreed or strongly agreed that the virtual interview helped them in

avoiding taking time off from work (64.4%) or made it easier for them to attend all the invitations

(79.1%). Only 6 (4.2%) applicants disagreed that the virtual interviews helped to avoid conflicts

in the interviewing dates.

Regarding the potential difficulties of the virtual interviews, nearly one-third of applicants

showed agreement or strong agreement with having suboptimal audio or video quality. Moreover,

only 13 (9.1%) applicants disagreed or strongly disagreed that they have received good technical

support prior to the virtual interviews. Around 40% of applicants indicated that they agreed or

strongly agreed that they had difficulty maintaining eye-contact or were concerned about the lack

of physical contact (e.g., handshaking) during the interview.

Overall, 29% of applicants disagreed or strongly disagreed that the time per interview was

appropriate and the interview allowed them to express their abilities to their satisfaction. Around

half (47.6%) of applicants agreed or strongly agreed that the number of questions they had during

the interview was appropriate. Nearly 70% of applicants had agreement or strong agreement that

the virtual interviews are less stressful and logistically easier than the physical interviews.

Moreover, approximately one-third of applicants agreed or strongly agreed that they would have

done better in physical interviews. Similarly, nearly 30% of applicants disagreed or strongly

disagreed that the virtual interview has increased their chance of being accepted. Despite these
 5

results, a majority (63%) of applicants agreed or strongly agreed that they were satisfied with their

experience with the virtual interviews. In addition, only 18.9% of applicants disagreed or strongly

disagreed that the virtual interviews were fair. Almost half of the applicants reported agreement or

strong agreement that the virtual interviews were more convenient than the physical interviews.

When it came to their preference for the type of interview, almost half of the applicants

agreed or strongly agreed that the virtual interviews should be continued next year while only one-

fourth disagreed or strongly disagreed. Moreover, there was almost no disagreement (99.3%) that

the virtual interviews were the best option during the COVID-19 pandemic. Overall, there were

no significant differences in the overall experience of applicants with virtual interviews based on

any of the demographic or application and nomination information (p > 0.05).