“THE IMPACT OF

LEARNING OUTCOMES
ASSESSMENT ON
ACCREDITATION SYSTEMS
IN LATIN AMERICA”
Nadia Paola Mireles Torres
University of Calgary 
GDER 700 A & 700 B
Dr. T. Armstrong
Winter, 2010
Background

 Globalization - Global forces - ICT,

Knowledge society, global market
 New challenges
Global market
○ More competitive graduates: new role of HE
New skills and competencies
 Quality Assurance
Increased attention
Important global trend
Quality Assurance (QA)

 Dimensions
QA for accountability – quantity focus
QA for improvement – quality focus
 Mechanisms for QA
Evaluation
Assessment
Accreditation
Accreditation

 Most common approach to measure QA

 Accreditation focus turned indicators from
quantity inputs to quality outcomes
 New challenge: the development of evidence
and assessment of student learning
outcomes (J. S. Eaton, 2002).
Learning outcomes (LO)

 Accreditation has gradually shifted from

encouraging to requiring LO. Kuh & Ikenberry (2009)
 Who?:
USA (CHEA)
○ Award for Institutional Progress in Student LO
EU (Tuning)
Australia
OECD (AHELO)
Literature review

 Quality assurance
approaches
QA for improvement Developed countries

 Accreditation
Current status Latin America … ???
Trends to LO assessment
Impact in accreditation criteria
○ Case: USA / CHEA (award)
○ Cases: Mexico, Argentina,
RIACES
 Globalization
Increase in HE New skills

Trend on QA
For accountability For improvement

Approaches
Accreditation LO assesment
Incorporate LO in
Developed accreditation
countries

Latin Accreditation ?? ??
America

An interpretation of the future impact that LO assessment will have in accreditation

systems of Latin America
Research question

 How is the trend to assess LO in other countries

impacting accreditation in Latin America?
How was the process to integrate the assessment of
LO in the criteria and certification processes of
accreditation agencies in the USA?
To what extent the CHEA award has influenced
institutions in the USA to start assessing LO?
Accreditation is a transferred process. How is the LO
assessment going to be transferred to Latin American
accreditation systems?
Studies that have addressed the problem

 Related to accreditation and the trend to focus

in LO:
Brittingham, 2009; Christy, McNeal, & Lewis, 2002;
Mori, 2009; Beno, 2004; Volkwein et al., 2006;
Whittlesey, 2005
 Related to HEIs experiences on LO
assessment:
Anagnos, 2008; Bresciani, 2009; Coates, 2009;
Peach, Brian E.; Mukherjee, Arup & Hornyak,
Martin; 2007; Schaeper; 2009; Stivers; Bonnie &
Phillips, Jeffrey; 2009
Studies that have addressed the problem.

 Whittlesey’s (2005): Criteria used by some

accreditation agencies to determine to what extent
the LO assessment is incorporated in their
certification processes
 Volkwein et al. (2006): Impact of accreditation on
LO experiences of 40 HEIs.
 Nusche (2008): Analyzed different instruments used
to assess LO in different countries
 AHELO feasibility study (2011): Assess LO in
different HEIs around the world with a unique
instrument
Studies that have addressed the problem.

 LO assessment is uncommon in Latin America

(either in HEIs or in accreditation agencies)
 Few or no studies addressing the LO
assessment in LA
(Nuñez, 2008): compared the LO assessment in
different years. Mentions one specific accreditation
agency to be including the LO assessment in its
certification criteria
 Studies related to accreditation do not
contemplate very deeply the LO assessment
Research framework

 Interpretivism “is an attempt to understand and

explain human and social reality… looks for
culturally and historically situated interpretations
of the social life-world”.
 This study seeks to:
Understand evolution of QA in a specific country
Explain how the trend to QA for improvement (LO
assessment) impacted accreditation
Interpret how this trend will make an impact in an
accreditation system that yet does not completely
focus on QA for improvement
Methodology

 Case study: “the researcher explores in depth

a program, an event, an activity, a process, or
individuals” Creswell (2003)
 Type of case study:
Historical organizational
Multi sites
 “The more that your question seek to explain
how or why some phenomenon works, the
more case study will be relevant” Yin (2008)
Prcedure for Data Collection

 Interviews and document analysis

Interviews with accreditation agencies' authorities
in: USA, Mexico, Argentina and RIACES
CHEA case: Review of documents and records
regarding the implementation of LO assessment in
certification criteria
Data analysis

 The case study “is designed to elicit

discussion and analysis of a particular
situation… based on their analyses, students
can make predictions about future events”.
Naumes & Naumes (2006)
Reliability of the study

 CHEA case: leader in requiring HEIs so to

include the assessment of LO in their practices.
 Mexican case: only OECD Latin country that
will participate in the AHELO study
 Argentinean case: Latin-American accreditation
agency requiring a proof of LO assessment in
its certification criteria
 RIACES as a general Latin American case: to
predict how accreditation in Latin America may
gradually focus on the assessment of LO.
Limitations

 Interpretations may not be completely accurate

 Quality assurance faces other challenges that
may not be contemplated in this study and may
affect the interpretations
 Hard to get the whole Latin perspective by
researching only two countries as case studies
 Lack of experience of the author in the
accreditation field may be a limitation to
understand critical issues
 Language/databases, context/studies
Anticipation of results

 Will there actually be an impact?

 I dont really know
 Very new topic
 Maybe a copy or adaptation from?
USA
EU
 Very slow pace
 International accreditation – more important
than inclusion of LO assesment
References
Altbach, P. G., & Knight, J. (2006). Visión panorámica de la internacionalización en la educación superior: motivaciones y realidades. Perfiles
Educativos, XXVIII, pp. 13-39.
Beno, B. A. (2004). The role of student learning outcomes in accreditation quality review. New Directions for Community Colleges, 2004(126), 65-72.
Brittingham, B. (2009). Accreditation in the United States: How did we get to where we are? New Directions for Higher Education, (145), 7-27.
Brunner, J. J. (2000). Globalización y el futuro de la educación: Tendencias, desafíos, estrategias.
Seminario sobre Prospectiva de la Educación en la Región de América Latina y el Caribe.
Carot, J. M. (2010). Debate semanal del Portal Enlaces. Retrieved 3/22/2010, 2010, from
http://www.iesalc.unesco.org.ve/index.php?option=com_content&view=article&id=1565:&Itemid=494&lang=es&limitstart=3
Christy, W. K., McNeal, L., & Lewis, R. (2002). Accreditation: New directions and new reports.
Coates, H. (2009). What's the difference? A model for measuring the value added by higher education in Australia. Higher Education Management &
Policy, 21(1), 77-95.
Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches (2nd ed.). Thousand Oaks, Calif.: Sage
Publications.
Crotty, M. (1998). The foundations of social research: Meaning and perspective in the research process. London; Thousand Oaks, Calif.: Sage
Publications.
de Wit, H., Jaramillo, C., & Knight, J. (2005). Higher education in Latin America: The international dimension. Washington, D.C.: World Bank.
de Wit, H., & Knight, J. (1999). Quality and internationalisation in higher education. Paris: Organisation for Economic Co-operation and Development;
Programme on Institutional Management in Higher Education.
Diaz Barriga, Á. (2009). La acreditación de programas (planes de estudio). Entre el formalismo y los procesos educativos. RISEU
Didou Aupetit, S. (2006). Internacionalización de la educación superior: entre el entusiasmo y el desencanto. Perfiles Educativos, XXVIII, pp. 56-70.
Eaton, J. S. (2002). Assuring quality and accountability in postsecondary education: Assessing the role of accreditation. Washington D.C.: Council for
Higher Education Accreditation.
Eaton, J. S. (2008). ATTENDING to student learning. Change, 40(4), 22.
Eaton, J. S. (2009). Accreditation in the United States. New Directions for Higher Education, (145), 79-86.
European Commission. TUNING educational structures in Europe. Retrieved 3/22/2010, 2010, from http://tuning.unideusto.org/tuningeu/
Gazzola, A. L., & Pires, S. (2008). Hacia una política regional de aseguramiento de la calidad en educación superior para América Latina y el Caribe.
Venezuela: IESALC-UNESCO.
Global University Network for Innovation. (2007). La educación superior en el mundo 2007. acreditación para la garantía de la calidad: ¿Que está en
juego?. Madrid: Mundiprensa.
Kehm, B. M., & Teichler, U. (2007). Research on internationalisation in higher education. Journal of Studies in International Education, 11(3-4), 260-273.
References
Kristoffersen, D. (2010). Advice on learning outcomes resarch.
Kuh, G., & Ikenberry, S. (2009). More than you think, less than we need: Learning outcomes assessment in American higher education. National
Institute for Learning Outcomes Assessment.
Lamarra, N. F. (2003). Higher education, quality evaluation and accreditation in latin america and MERCOSUR. European Journal of Education,
38(3), 253.
Liu, O. L. (2009). Measuring learning outcomes in higher education No. RDC-10) Educational Testing Service.
Lopez Segrera, F. (2003). El impacto de la globalización y las políticas educativas en los sistemas de educación superior de América Latina y el
Caribe. Buenos Aires, Argentina: CLACSO.
López Segrera, F., Brock, C., & Dias Sobrinho, J. (2009). Higher education in Latin American and the Caribbean 2008. Venezuela: IESALC-
UNESCO.
Marijk van, d. W. (2007). Internationalization of higher education in the OECD countries: Challenges and opportunities for the coming decade.
Journal of Studies in International Education, 11(3), 274-289.
McMillan, J. H. (2008). Educational research: Fundamentals for the consumer Pearson/Allyn and Bacon.
Mori, R. (2009). Accreditation systems in Japan and the United States: A comparative perspective on governmental involvement. New Directions
for Higher Education, (145), 69-77.
Naumes, W., & Naumes, M. J. (2006). Art and craft of case writing. M.E. Sharpe.
Nuñez, A. M. (2008). Evaluación de los niveles de adquisición e integración de conceptos y competencias en ciencias básicas en la Universidad
de Mendoza. Universidad de Granada).
Nusche, D. (2008). Assessment of learning outcomes in higher education: A comparative review of selected practices OECD, Directorate for
Education.
Organization for Economic Co-operation and Development. OECD feasibility study for the international assessment of higher education learning
outcomes (AHELO). Retrieved 3/22/2010, 2010, from
http://www.oecd.org/document/22/0,3343,en_2649_35961291_40624662_1_1_1_1,00.html
Santiago, P., & Organisation for Economic Co-operation and Development. (2008). Tertiary education for the knowledge society. Paris: OECD.
Volkwein, J. F., Lattuca, L. R., Harper, B. J., & Domingo, R. J. (2006). Getting in sync: The impact of accreditation on student experiences and
learning outcomes. Chicago: Paper presented at the Annual Meeting of the Association for Institutional Research.
Whittlesey, V. (2005). Student learning outcomes assessment and the disciplinary accrediting organizations. Assessment Update, 17(4), 10-12.
World Bank. (2002). Constructing knowledge societies. World Bank.
Yin, R. K. (2008). Case study research: Design and methods (4th ed.)