Test design questions

Q1. What is the test designed to measure?

Ans: The higher authorities and institutions globally recognize student engagement and active
learning. Engaging students in various educational activities will undoubtedly enhance their
learning experience. A range of factors affecting student engagement have been enumerated in
studies and have been mentioned in theory. However, a few have been implemented. The role
of student engagement in quality education is a new concept in Saudi Arabia. Even though
numerous studies in Saudi Arabia have focused on students' performance and engagement,
none have focused on validating findings. Moreover, no valid, reliable instrument was studied or
developed to evaluate student performance via active learning. Hence, the present thesis aims
to evaluate and measure the student's engagement in active learning classrooms and
interaction with the faculty members within various cultural dimensions via a validated and
reliable scale.

Q2. What is the objective of the test?

Ans: The questionnaire will be built according to the previously designed validated and reliable
questionnaires. Develop a scale to measure student engagement, entailing the process of
conceptualizing the construct of 'student engagement' (Behavioral, Emotional, and
psychological). Construct items to measure 'student engagement' (academic preparation,
perseverance, academic difficulty, academic help-seeking, collaborative learning, student-faculty
interaction, hours studying and working, campus support, and predicted graduation from their
current institution). With this questionnaire we will calculate the student’s engagement and
interaction in an active learning environment which will eventually help the the administrators,
curriculum designing committee, professors, and authorities at the university level. Designing
and scaling a valid and reliable instrument to measure student engagement will eventually help
higher authorities measure these outcomes for other universities. Furthermore, this will help
universities design curricula that benefit students and faculties. Authorities at the university will
get a pathway to plan the semester according to the requirements of students and faculty. They
were, overall, aiming to engage students in active learning. This approach will help universities
in Saudi Arabia achieve a desirable outcome and serve as a win-win situation for faculty and
students.

Q3. Is there a need for this test?

Universities worldwide have developed various systems to encourage students' engagement in

every sector (Trowel, 2010). For example, In Japan and China, the School engagement survey
(Fredricks et al.2005) and student engagement scale (Gunuc and Kuzu, 2015) are used to
survey students' engagement. These instruments focus on university course performance
outcomes at macro levels. Similarly, the "National Survey of Student Engagement" (NSSE,
2001) was developed by American universities to evaluate the performance of students. The
outcome of this survey focused on the learning experience and students' participation at the
university level. It also measured the cognitive behavior of students and positive development
throughout the university years. A similar instrument, the "Australian Survey of Student
Engagement" (AUSSE), was developed by Australian and New Zealand universities in the year
2007 (Coates,2010, Coates 2015). This instrument was developed to promote student
engagement at the university level by collecting data on students' perception of learning
experience and satisfaction with university courses and services (Coates, 2010; Coates 2015).
In European Universities, "The ASPIRE-to-Excellence" initiative was launched by the
International Association of Medical Universities (AMEE) (AMEE 2017). "Student Engagement
Questionnaire (SEQ)," developed by Kember et al. (2009), was utilized in a study conducted
among the final year medical students at King Saud University Riyadh.

This questionnaire will be designed utilizing the validated and reliable questionnaires for
analyzing student engagement. This will be the first designed questionnaire to access the
performance of students in Saudi Arabia. It will be comprised and will be designed to be filled up
in no more than 25 mins. In this study, a pilot test will be employed to assure survey instrument
reliability and stability (Johnson & Christensen 2014). Using Cronbach’s alpha, the study
measured internal consistency (Field 2018; Hair et al. 2010; Johnson & Christensen 2014).
Normally a value of 0.7 or over is an indicator of internal consistency of an instrument (Field
2018; Hair et al. 2010; Tabachnick & Fidell 2007). In this study, this reliability test will be
performed on each key construct. Further, this study also employed the composite reliability
technique to assure the validity and reliability of the survey instrument (Bagozzi & Yi 1988;
Nunnally & Bernstein 1994)

Q4. Who will use this test?

Ans: Authorities at the university will get a pathway to plan the semester according to the
requirements of students and faculty. They were, overall, aiming to engage students in active
learning. This approach will help universities in Saudi Arabia achieve a desirable outcome and
serve as a win-win situation for faculty and students.

Q5. Who will take this test?

Ans: This questionnaire will be designed for the students and educators to evaluate the factors
responsible for direct and indirect issues that influence student engagement and interaction.
This test will provide an insight for developing curriculum that supports students and educators
needs. Students and educators in the university can take this test. This will be a self-respond
questionnaire; hence the chances of bias will be more. Students with low academic
performance might ask their colleagues to help them fill the questionnaire which might change
the response.

Q6. What content will the test cover?

Ans: Educational institutions should use engagement measures to motivate students in the
subject, and to achieve high levels of motivation should focus on teaching resources,
academics’ ability to use technology efficiently, development of relevant curriculum resources
that meet the need and developing strong support systems to guide students to engage them in
various curriculum activities. Student engagement is an important factor associated with the
performance of a university. Hence, the acknowledge the growth of university and student
developing a proper guideline to examine their performance is necessary. This questionnaire will
be designed according to the educational outcomes in university of Saudi Arabia.
Q7. How will the test be administered? Individually or in groups? Is it amenable to both group
and individual administration? What differences will exist between individual and group
administrations of this test? Will the test be designed for or amenable to computer
administration? How might differences between versions of the test be reflected in test scores?

Ans: this test will be administered by the students but will be explained to them in the form of
informational sheet. This information sheet will be pre-approved by the staff and administration
of the universities. Individual administration of questionnaire chances of bias will be less
compared to the group as more answers will be marked according to the high achiever at the
university.

Q8. What is the ideal format of the test? Should it be true–false, essay, multiple-choice, or in
some other format? Why is the format selected for this test the best format?

Ans: this will be a closed ended and open-ended questionnaire. questions regarding current
curriculum will be mostly closed ended with “yes” and “no”. While the questions regarding the
improvement will be open ended and themes will be developed. This will help the administration
of universities to design curriculum accordingly.

Q9. Should more than one form of the test be developed? On the basis of a cost–benefit
analysis, should alternate or parallel forms of this test be created?

Ans: Initially only one questionnaire form will be developed as it will be cost-effective, and time
constrain. If required, then the questions might be increased. As study will procced the answer
to this question will be developed.

Q10. What special training will be required of test users for administering or interpreting the
test?

Ans: No special training is required. Educators at university and secondary education level can
utilize this questionnaire in their institution. As this is a developing study no restrictions have
been mentioned. To analyze the answers to this questionnaire statistical analysis will be
required.

Q11. What types of responses will be required of test takers? What kind of disability might
preclude someone from being able to take this test? What adaptations or accommodations are
recommended for persons with disabilities?

Ans: As this is an online questionnaire form any student can fill the answers.

Q12 Who benefits from an administration of this test? What would the test taker learn, or how
might the test taker benefit, from an administration of this test? What would the test user learn,
or how might the test user benefit? What social benefit, if any, derives from an administration of
this test?

Q13. Is there any potential for harm as the result of the administration of this test? What
safeguards are built into the recommended testing procedure to prevent any sort of harm to any
of the parties involved in the use of this test?
Q14. How will meaning be attributed to scores on this test? Will a test taker’s score be
compared to others taking the test at the same time? To others in a criterion group? Will the test
evaluate mastery of a particular content area?