(D2) Competency-Based Approach (Gross - Perwitasari)

Competency-based
Approach: Impact on Student

Learning Outcomes
Evelyne Gross
Head of Accreditations, Rankings and Social Responsibility
Burgundy School of Business - Dijon - France
Arum Perwitasari, Ph.D.

Academic Relations Specialist
ETS Global Amsterdam
Learning Objectives
 To understand how assessment of learning contributes to improving student

learning outcomes
 To know how to make sure students are aware of their competencies
 To be able to implement some of the assessment tools developed for students
2
Building up Skills for
Business® - BSB®
Evelyne Gross
Head of Accreditations, Rankings and Social Responsibility
Burgundy School of Business - Dijon - France
Building up Skills for Business® - BSB®: the Project
 A project begun in 2009
 A strategic project for Burgundy School of Business
 Structuring and federative project (concerning all programs)
 Assuring the coherence of each program in terms of skills
 Deepening the partnership with companies
 Meeting quality requirements of the accreditation standards
 Objectives
 To serve as the basis of the Assurance of Learning Process
 To help students to build up their competencies and to promote them
 To improve the pedagogical process
Building up Skills for Business® - BSB®: the Project
 Stakeholders of the Project

 Students
 Program Directors
 Faculty
 Companies
 Dedicated Resources and Tools
 Two project managers
 A portfolio of competencies
 Self-assessment and assessment tools (Skills Book)
 Promotion process of competencies (Career Booster)
Milestones of the Project
6
1. Development of Portfolios of Competencies
 A portfolio of competencies is a tool which enables the identification of the competencies used
and required in a working activity or in a job.
 For an educational program, a portfolio of competencies identifies and lists the competencies
targeted by the program, i.e., the competencies that the students who follow the program may
potentially develop.
BSB’s portfolios of competencies:
 Learning Goals deriving from the School’s Mission (2008)
 Learning Objectives developed crossing companies expectations (interview / focus groups
with companies) and faculty expertise (2009 revised in 2019)
 Learning objectives expressed in terms of competencies
 Knowledge, generic skills and professional skills
7
1. Portfolios of Competencies
 Learning Goals Common to All Programs
INTENDED LEARNING OUTCOMES (ILO) / LEARNING GOALS (LG)

ILO/LG1
To understand how organizations work; to identify and mobilize the relevant tools in an
organizational and/or operational context
ILO/LG2
To acquire the skills and attitudes of entrepreneurial management
ILO/LG3
To act as a socially responsible, internationally minded manager
ILO/LG4
To acquire the knowledge and to master the use of the techniques and tools required to
exercise a profession.
8
1. Portfolios of Competencies
 MSc Wine Management as an Illustration

Learning Goals Learning Objectives / Competencies
WM LG 01 WM LO 01 - To be able to identify the needs of an organisation
To understand how organisations work; to WM LO 02 - To be able to identify potential risks stand by organisation
identify and mobilise the relevant tools in an
WM LO 03 - To be able to choose the best organisation to achieve a goal
organisational and/or operational context
WM LG 02 WM LO 04 - To be able to react at unexpected events
WM LO 05 - To be able to use critical thinking
To acquire the skills and attitudes of
entrepreneurial management WM LO 06 - To be able to optimize business under constraints
WM LO 07 - To be able to complete a project
WM LO 08 - To be able to manage people with diverse skills sets
9
2. Pedagogical Experiences
 Identification of Pedagogical Experiences Developing Competencies
Key Pedagogical Experiences
Courses
Professional experiences : Internship / Apprenticeship / Gap year
International Experiences (Semester and Double Degree)
Project and team Experiences (team work, learning by carrying out community initiatives,
associations)
Professional thesis
 An assessment for each experience

3. Self-Assessment and Assessment Tools
 Assessment of Competencies (from sample of students (2012) to all students of all
programs (2016)
 Course modules (assessment by Faculty)
 Internship, apprenticeship and professional experiences (assessment by company tutor)
 Professional thesis (assessment by thesis tutor)
 Self-Assessment Tools (progressive implementation since 2010)

 At different time of the curriculum
 At the beginning/end of programs
 At the beginning/end of concentration track
 During the curriculum through the Skills Book
 Team and project experiences
 International experiences
 Student Clubs & Societies experiences
11
3. Self-Assessment and Assessment Tools
 Example of Assessment Grid For Internship (MGE2 et MGE3)
 Generic Skills (GS)
Being Acquired more Non-applicable
Acquired (2)
acquired than expected (1)
MGE GS 02 : To position one’s action
within an organisation
To understand the organizational situation in its complexity
To be capable of measuring the impact of decisions and actions
To know how to inform and to report back to one’s hierarchy or clients
(1) These criteria fit with the accreditation norms criteria "Below, Meets, Above".
(2) "Non-applicable" signifies that the student did not have the opportunity to develop this competency during the internship.
 Professional skills (PS)

Internship in Entrepreneurship (EMP) Being Acquired more Non-
Acquired applicable
acquired than expected
MGE PS EMP 01 : To be capable of analysing a
situation within a given framework (risks analysis of
entrepreneurial activity) and making decisions
accordingly
4. Feedback from Students Self-Assessment
What are the main skills developed Self-evaluation – Association Experience 2014
during association experience?
Être capable de travailler en équipe, en mode… 80%
Avoir le sens des responsabilités 78%
Self- Être capable d'écouter et d'intégrer des points de… 73%
evaluation Low Medium High N/A Savoir réagir face à l'imprévu et à l'urgence 73%
average Être curieux, à l'écoute de son environnement et… 73%
6% 27% 56% 11%
profile Être capable d'analyser une situation pour guider… 72%
Savoir recueillir les informations pertinentes et les… 72%
Savoir s'adapter aux situations et au contexte 70%
On 40 evaluated skills, 21 are declared Savoir s'organiser, structurer ses actions dans le… 69%
as Highly developed (>60%) Savoir gérer le stress et différentes actions en… 68%
Faire preuve d'autonomie dans ses décisions et ses… 67%
Faire preuve de pragmatisme et de persévérance 66%
Association experience help to develop Savoir se donner des objectifs et s'attacher à les… 66%
key competencies transferable into Savoir résoudre des problèmes 65%
professional life Savoir adapter son discours à sa cible 63%

Faire preuve d'initiative, de créativité et d'innovation 63%
Être capable de mesurer les impacts de ses… 62%
Savoir convaincre, mobiliser, motiver et fédérer 61%
Savoir prendre du recul et se remettre en question 61%
Faire preuve d'humilité 61%
Savoir accepter la différence et gérer la diversité 61%
50% 55% 60% 65% 70% 75% 80%

4. Feedback from Students Self-Assessment
GS – Generic Skills developed by students during international semester
Skills developed (2015)
during the GS3.4 : Savoir accepter la différence et gérer la diversité 1%
3% 94% 1%
international 0%8%
GS11.2 : Savoir gérer et mener à bien une relation d'échange… 92% 0%
experience GS11.1 : Savoir analyser et comprendre les principaux traits…1% 11% 88% 0%
GS5.1 : Savoir s'adapter aux situations et au contexte 0% 15% 83% 2%
GS3.3 : Être curieux, à l'écoute de son environnement et…1% 18% 80% 1%
GS6.3 : Être capable d'écouter et d'intégrer des points de vue…2% 18% 79% 0%
GS3.1 : Savoir prendre du recul et se remettre en question 9% 11% 76% 3%
GS9.1 : Faire preuve d'autonomie dans ses décisions et ses…2% 23% 73% 2%
GS3.2 : Faire preuve d'humilité 5% 19% 67% 9%
GS4.4 : Savoir réagir face à l'imprévu et à l'urgence 11% 18% 65% 6%
GS5.2 : Montrer une capacité à apprendre 2% 34% 64% 0%
GS1.4 : Savoir résoudre des problèmes 3% 30% 64% 3%
GS4.1 : Savoir s'organiser, structurer ses actions dans le…5% 30% 63% 3%
GS10.1 : Avoir le sens des responsabilités 5% 30% 58% 8%
GS4.2 : Faire preuve de pragmatisme et de persévérance 5% 34% 55% 7%
GS4.3 : Savoir gérer le stress et différentes actions en parallèle 11% 36% 48% 5%
GS2.1 : Comprendre la situation organisationnelle dans sa…5% 38% 43% 15%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Peu Moyennement Fortement Non-Applicable

4. Feedback from Direct Assessment
15
4. Feedback from Direct Assessment
16
5. Self-Assessment vs Direct Assessment
Initial self‐assessment Final self‐assessment BMA ‐ direct
Feedback to (response rate= 86%) (response rate= 86%) assessment
Heads of
concentration Code Competency %B %M %A %B %M %A %B %M %A
tracks To know how to manage and to control the
financial information of a firm (to master
financial and consolidated accounting, to
MGE PS FE 01 29% 54% 17% 29% 46% 25% 21% 64% 14%
be acquainted with the principles of IFRS,
to know how to manage an accounting
closing)
To know how to analyse and to manage a
firm’s economic performance (to master
MGE PS FE 02 management accounting, cost analysis, 21% 54% 25% 20% 44% 36% 0% 64% 36%
budgets, forecasts, performance
indicators)
To know how to prepare, to make and to
implement a financial decision (short-term,
MGE PS FE 03 46% 46% 8% 38% 46% 17% 11% 71% 18%
long-term, international finance related
issues)
To have a capacity to analyse and look at
MGE PS FE 04 things objectively (to know how to 8% 76% 16% 8% 64% 28% 0% 57% 43%
challenge speculation)
To be capable of taking part in the
MGE PS FE 05 definition and the implementation of a 33% 54% 13% 29% 46% 25% 14% 57% 29%
firm’s strategic choices
6. Launch of the Skills Book
 A New Tool for Professional Project: the Skills Book
 Online platform
 Personalized
 Gathering all students self-assessment and assessments
 Allowing all students to proceed to the self-assessment of their competencies all along the
curriculum on competencies and professional project
 Facilitating the exchange with Career Center coaches to build up your professional project
 Process
 Tested in September 2016 on a sample of students
 Implemented for the Grande Ecole Program since 2017 and Bachelor since 2019
 Reminded by professors for self-assessment within course modules
 Used by Career Booster Coaches during interviews demanded by students
6. The Skills Book
 Structure
 The Skills Book presents 4 tabs
 Tab 1 : Profile
 Tab 2 : Competencies
 Tab 3 : Documents
 Tab 4 : Professional Project
6. The Skills Book
 Tab Competencies
 Students can add a personal reflection on the development of their
competencies
6. The Skills Book
 Tab Competencies
 Detailed view of his(her) self-assessment and assessment
Final user:
Students;
AOL vs BSB® Program Directors;
Corporate Relations &
Career Center Dep.;
Personal Development Dep.;
AOL International Relations Dep.;
Quality Assurance and Program Management Communications Dep.,
•Measure the achievement of Learning Goals and Objectives BSB® Team
•Improve learning process
•Definition of Learning BSB®
Goals and Learning •To adapt targeted competencies
Objectives (portfolios of to companies expectations
competencies) •To provide students with
•Development of tools to feedback on developed competencies
measure learning •To support student in their
Final user:
acquisition professional project
Program Directors
Stakeholders Involvement
23
 Students
 Communication
 at the beginning of the academic year
 in small workshops
 in concentration tracks
 Working groups with students
 Self-assessment moments throughout the curriculum
 Program Directors
 Involvement through steering committee
 Joint definition of the various steps of the project
 Collaborative development of portfolio of competencies
24
 Faculty Members
 Involvement in competency/knowledge definition
 Working group for assessment tests
 Involvement in student assessment (course modules, internship, thesis)
 Companies
 Involvement in competency definition
 Assessment of competencies through internships and apprenticeships
 Participation in communication events for students
25
Next Steps - Ongoing
 Comparing Assessment vs Self-Assessment

 Big Data Analysis based on Skills Book Data
 Analysing the Evolution of Skills Self-Assessments
 Student Clubs & Societies experiences, Learning By Carrying Out Community Initiatives
experiences, internships experiences
 Reinforcing Sustainability in Portfolios of Competencies
 Evolution of Master in Management and Bachelor portfolio of competencies
 Inspired by Lausanne Business School (Competency Assessment of Responsible
Leadership) and CGE-CPU Sustainability Competency guide
26
Documenting Student
Learning with the HEIghten®
Outcomes Assessment
Suite
Arum Perwitasari, Ph.D.
Academic Relations Specialist
ETS Global Amsterdam
Selecting/Developing Assessments
Advantages Drawbacks
- Tailored to curriculum - Assessment quality
- Lower financial cost - Higher time/people cost
Developing Assessment - Flexible - Administration/logistics
- Internal buy-in - Validity study
- Reliability, validity, fairness - Not tailored to curriculum

- Assessment, data quality - Standard, inflexible
Selecting Assessment - Established models, theory - Can limit faculty input
- Credible
Local and external assessments should exist simultaneously ̶

together, they build a more compelling story of student learning
28
Copyright © 2019 by Educational Testing Service. All rights reserved. ETS, the ETS logo, MEASURING THE POWER OF LEARNING and HEIGHTEN are registered trademarks of Educational Testing Service
(ETS). All other trademarks are the property of their respective owners. 407152767
Why the HEIghten®
Assessments? Why now?
A New Approach to Assessment for a New
Era in Accountability and Improvement
Mounting Pressure on
Higher Education Institutions
Student success and retention
Need for curricular improvement
New business models and disruptive innovation
Accreditation, accountability for outcomes
Need for valid assessments
30
Introducing
the HEIghten® Outcomes Assessment Suite
The modular general education assessment mix that you tailor to

meet your institutional goals
A Suite of computer‐based assessments measuring essential
general education student learning outcomes.
The assessment data can be used for accreditation and
accountability initiatives, to guide curriculum improvement,
and to measure growth and development.
31
A Research-Driven Approach
to Assessment Design
32
ETS on Collaboration
 Identifying the areas of student learning

and student success that are important to
institutions
 Working with institutions to identify the best
ways to investigate these areas
 Promoting best practices uncovered by
institutions in their use of our assessments
33
The HEIghten® Approach
What the HEIghten Suite Measures and How
The HEIghten ® Outcomes Assessment Suite
Critical Thinking Written Communication Quantitative Literacy
Intercultural Civic Competency &

Competency & Engagement
Diversity
35
A scenario
Zak grew up in a country where senior members of

the community are accustomed to helping with the
financial concerns of younger people who ask for
assistance. When Zak arrives in a new country on
a student visa, he finds it difficult to manage the
many expenses that are not covered by his
scholarship. He meets with his advisor and, after
exchanging greetings and asking about the
advisor’s family, Zak politely asks for a sum of
money that will help him through the next two
months. His advisor is surprised and flustered at
what seems to him to be an inappropriate request.
36
The HEIghten®
Assessement Intercultural
Competency and Diversity
Module
ICD Framework
 Intercultural Competency (ICC) is defined as the ability to

communicate effectively and appropriately in
intercultural situations based on one’s intercultural
knowledge, skills and attitudes
 Propose a framework that builds on a process model of social
interaction by splitting cross-cultural interactions into three stages
and specifying the skills necessary to support successful
performance in each stage
38
ICD Framework
Approach Analyze Act

Tolerance for Ambiguity Self-Awareness
Behavior
Regulation
Social Monitoring
Positive Cultural
Orientation Suspending Judgment/
Perspective Taking Emotion
Self-Efficacy Cultural Knowledge Regulation
Application
39
A scenario
Zak grew up in a country where senior members of

assistance. When Zak arrives in a new country on a
student visa, he finds it difficult to manage the many
expenses that are not covered by his scholarship. He
meets with his advisor and, after exchanging
greetings and asking about the advisor’s family, Zak
politely asks for a sum of money that will help him
through the next two months. His advisor is surprised
and flustered at what seems to him to be an
inappropriate request.
40
A scenario
What response by Zak’s advisor best
reflects the ability to control emotion in the
face of a possible cross-cultural
Zak grew up in a country where senior members of misunderstanding?
assistance. When Zak arrives in a new country on a
student visa, he finds it difficult to manage the many
expenses that are not covered by his scholarship. He
meets with his advisor and, after exchanging
greetings and asking about the advisor’s family, Zak
• “Let’s get this straight, Zak. You’re asking me to give you a
politely asks for a sum of money that will help him personal loan!?”
through the next two months. His advisor is surprised • “Sorry, Zak. I’m not in a position to just hand out large
amounts of cash.”
and flustered at what seems to him to be an • “Well, Zak. That’s a very unusual request for a student to
inappropriate request. make to an advisor here.”
• “No, Zak. I can’t give you money. You should have made
arrangements for this before you came here.”
41
Institutional Score Report
A Process for Developing a Useful and
Comprehensible Score Report
Score Reporting
Reports Available:
 Institutional Report
 Test Taker Report
 Data Download File
43
44
45
The HEIghten® Outcomes Assessment Suite gives
institutions the tools they need to
 document and describe student achievement of general

education competencies
 provide evidence of program and institutional effectiveness to
accrediting bodies
 make quicker and more informed improvements to curriculum to
enhance student learning
 motivate students to do their best on the test with the student
score report
46
How do I use the HEIghten® assessments?
The HEIghten assessments allow institutions to:

 Paint a complete picture of student performance. Can be used as
a standalone assessment or in conjunction with an institution’s
internal assessment
 Use the actionable data to show that your institution prepares
students with the general education skills needed after graduation
 Mix and match the assessments that fit your individual institution
and add questions to tailor them to specific goals
47
Resources available for evaluating the HEIghten®
Outcomes Assessment suite
Resources available for evaluating the HEIghten Outcomes

Assessment suite:
 ETS research reports
 Test-at-a-glance documents
 Confidential test review copies for assessment directors and
faculty
Learn more at www.ets.org/heighten
48
Questions?
Learning Objectives
 To understand how assessment of learning contributes to improving student

learning outcomes
 To know how to make sure students are aware of their competencies
 To be able to implement some of the assessment tools developed for students
50
Please complete the session evaluation to let us
know how we did!
51

(D2) Competency-Based Approach (Gross - Perwitasari)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

(D2) Competency-Based Approach (Gross - Perwitasari)

Uploaded by

Copyright:

Available Formats

Competency-based

Approach: Impact on Student

Arum Perwitasari, Ph.D.

 To understand how assessment of learning contributes to improving student

 Stakeholders of the Project

 Learning Goals Common to All Programs

INTENDED LEARNING OUTCOMES (ILO) / LEARNING GOALS (LG)

 MSc Wine Management as an Illustration

 An assessment for each experience

 Self-Assessment Tools (progressive implementation since 2010)

 Professional skills (PS)

professional life Savoir adapter son discours à sa cible 63%

50% 55% 60% 65% 70% 75% 80%

Peu Moyennement Fortement Non-Applicable

 Comparing Assessment vs Self-Assessment

- Reliability, validity, fairness - Not tailored to curriculum

Local and external assessments should exist simultaneously ̶

Student success and retention

Need for curricular improvement

New business models and disruptive innovation

Accreditation, accountability for outcomes

Need for valid assessments

The modular general education assessment mix that you tailor to

 Identifying the areas of student learning

Critical Thinking Written Communication Quantitative Literacy

Intercultural Civic Competency &

Zak grew up in a country where senior members of

 Intercultural Competency (ICC) is defined as the ability to

Approach Analyze Act

Zak grew up in a country where senior members of

 document and describe student achievement of general

The HEIghten assessments allow institutions to:

Resources available for evaluating the HEIghten Outcomes

Learn more at www.ets.org/heighten

 To understand how assessment of learning contributes to improving student

You might also like