You are on page 1of 18

DIUS: self-assessment methodology

Susanna Karakhanyan, PhD DIUS National Coordinator ANQA INQAAHE Board of Directors World Bank Expert
DIUS FINAL CONFERENCE September, 2012

OUTLINE
How it all started: getting to common understanding Description of processes run by the universities

Mapping a process: identification of gaps


Organization of self-assessment: major steps

Data collection: major sources of data


Production of a self-assessment report.
3/30/2013 2 DIUS FINAL CONFERENCE

How it all started: getting to common understanding


Need to establish a common understanding on such concepts as quality, quality assurance, selfevaluation and the like;
Quality assurance as fitness for purpose;

Interpretation of ESG standards; Needs analysis per ESG standard; Allocation of roles among the partners.

3/30/2013

Two ESGs per institution; ANQA - external quality assurance; EU partners international peers in an expert panel; MoES observers and evaluators.
3 DIUS FINAL CONFERENCE

Description of processes run by university

QUALITY
3/30/2013 4

QUALITY ASSURANCE

PERFORMANCE

Quality management

Improving

Planning

Evaluation and feedback mechanisms

Implementing

3/30/2013

5 Public Accreditation Agency for Higher Education

Mapping a process

3/30/2013

Public Accreditation Agency for Higher Education

Example: ESG 1.4


1.4 Quality assurance of teaching staff: Institutions should have ways of satisfying themselves that staff involved with the teaching of students are qualified and competent to do so. They should be available to those undertaking external reviews, and commented upon in reports.

3/30/2013

DIUS FINAL CONFERENCE

Example: ESG 1.4

3/30/2013

DIUS FINAL CONFERENCE

Example: mapping teacher assessment

3/30/2013

Public Accreditation Agency for Higher Education

Organization of selfassessment: major steps


Purpose identification Why do we need to engage in selfassessment? Outcomes identification What results do we need to achieve with the self-assessment? Measurements and evaluation design What methodology, tools and mechanisms for measurement should we use? Data collection What type of data do we need to obtain? Analysis and evaluation - What are the methods we are going to apply to analyse the data? Decision-making regarding actions to be taken How are we going to use the results of the analysis in the decision-making regarding new strategies and developments?
Public Accreditation Agency for Higher Education

3/30/2013

How to put together a valid selfassessment ?


The questions to be answered are as follows: What indicators do we have already? Which indicators do I need to control my process? Which indicators are out of my influence? Which indicators do I need for doing the work? Which indicators do we need for improvement and enhancement? What is the cost-benefit ratio of collecting indicator data?

3/30/2013

Public Accreditation Agency for Higher Education

Data collection: major sources of data


Data collected from the feedback mechanisms, evaluation groups, which provide for qualitative information;
e.g. questionnaires, which are a powerful element if devised the right way; discussion-oriented team work quality circles, peer-reviewers (external and internal); focus groups, individual interviews and the like;

Statistical data available at the university units and/or statistical office, that is key performance indicators (KPIs) (data like student success rate, drop-out rate, in-coming out-going students, number of teaching staff, budget allocations and the like); Performance outcomes (samples) and other evidences (thesis papers, exam questions and respective answers, samples of publications, observations of classes and the like).

3/30/2013

Public Accreditation Agency for Higher Education

Organising a self-assessment
DATE 8 months before a planned assessment ACTIVITY Appoint the leader of the assessment process Compose the assessment team, including students dividing up the subjects to be dealt with

The following 6 months


4 months after the start About 5 months after the start 6 months after the start 8 months after the start
3/30/2013

Each person responsible for collecting information and data collects that information Writing drafts of the subjects
Discussion on the drafts in the group Second draft Discussion of the 2nd draft with all faculty staff and students during an open hearing Edit the comments of the hearing for the final draft Carry out an (inter-collegial) assessment

13

DIUS FINAL CONFERENCE

Format: introduction
One page history and current profile of the institution; An overview of the management and organisational structure, including an organisation chart (or equivalent); A brief overview of the institutions quality assurance arrangements and systems,
The scope of the audit, The fitness of QA arrangements for the context of the institution's charter, profile/objectives and special characteristics.

3/30/2013

14

Format: the main document


The text should be developed per criterion/standard and include In-depth analysis of quantitative and qualitative data and factual information not available in other materials (such as annual reports), cross- referenced to, and within, the main document. entry qualifications, non-completion rates, degree results, graduate employment data, stakeholder satisfaction data, performance indicators and measures employed that relate to the scope of the audit. Effectiveness of a particular approach; How this information is used in quality control and assurance processes within the institution, giving examples where possible. Examples of the effectiveness in the operation of quality assurance systems and instances of particularly good practice SWOT analysis.

3/30/2013

15

Appendices
Appendices indicate
how this information is used in quality control and assurance processes within the institution, giving examples where possible; examples of the effectiveness in the operation of quality assurance systems and instances of particularly good practice; The referenced documents (e.g. strategic plan, yearly and other plans); Any other attachments required by the agency.
3/30/2013 16

Challenges ahead
Re- development of policies and procedures that really work; Re-development of the tools: questionnaires, focus group discussions and the like;

Wider involvement of stakeholders in the process;


Analytical approach to self-assessment rather than descriptive; Using the major findings in the institutional planning and enhancement; Establishment of a quality culture at institutions.
3/30/2013 17 DIUS FINAL CONFERENCE

THANK YOU and QUESTIONS


s.karakhanyan@anqa.am s.karakhanyan@gmail.com

3/30/2013

18