3.
4 ASSESSMENT OF COMPETENCE
SPECIFICS: (from EMSA Report)
PROBLEM/CONCERNS
Item -A.
The Board of Examiners
(BOE) did not follow the
approved quality
procedure for validation of
questions to be uploaded
into the database.
Item -B.
Lowering the “pass mark”
and not “ varying the level
of knowledge,
understanding and
proficiency” for candidates
applying for COC’s on
ships with limited GT or
propulsion power did not
CORRECTIVE ACTION TAKEN INTO (To submit Appropriate Evidence (AE) on
ACCOUNT
1. The reconstitution of the
composition of BOE
2. The orientation training of the
newly appointed members of
BOE
3. Proposed quality procedure
QPR 07.4.2 rev02 on
“Construction, Review and
Validation and Administration
of Test Items, which has been
reviewed to give a better
assignment of responsibilities
and risk analysis.
4. Verification of the effectiveness
of the revised procedures.
1. The Theoretical examination as e. Statistics on the result of development
a measure of knowledge and
understanding shall be
streamlined regardless of the
limitations.
2. The Practical assessment
standards shall be delineated
(varying the level of
knowledge, understanding and examination subjects and the pass marks
proficiency.
(per “REMARKs” from EMSA Report)
measures taken, or decided to be taken,
shall include but shall not be limited to: )
a. Details on the reconstitution of the
Board of Examiners.
b. Evidence on the orientation training
received by new appointed members of
the BOE
c. The approved version of QPR 07.4.2
rev02.
d. The result of the verification by the
Quality Representative of the
effectiveness of the revised procedure
QPR 07.4.2
j. Evidence on the training received by the
members of the BOE at the
“Development Academy of the
Philippines’
of new questions
a-Details on the reconstitution of the
Board of Examiners
g. Approved versions of circulars related
to the certification under Regulations II/1,
II/2, III/1. III/2, III/3, and III/6 together with
their annexes establishing the
RECOMMENDATIONS ON HOW TO ADDRESS
THE SHORTCOMINGS
(FROM PMMAAAI)
As per review and analysis on the findings as
well as remarks from the EMSA report, the
PMMAAAI came up with two sets of
recommendations grouped as follows:
a) Corrections – proposed immediate
actions to be taken to close out the
noted shortcomings by the EMSA
Auditors; and,
b) Corrective Actions - to prevent the
recurrence of those findings and
improve further the existing system
for future requirements.
a) Corrections:
Hereunder are the various actions and
processes deemed necessary to close out the
EMSA findings and eradicate the present
dilemma being faced by the country
particularly on the Maritime Administration’s
Assessment of Competence:
1. Review the approved quality procedures
on the “Construction, Review, Validation
and Administration of Test Items” to
1
ensure that the standards
of competence prescribed
in Section A-II/2 and A-III/3
of the STCW Code were
fully demonstrated
Item -C.
The number of questions
available in the database
system for each
competence subject to
assessment (25 to 29) did
not fully ensure a
comprehensive evaluation,
taking into account the
number of questions
randomly generated for a
competence in a test (20)
Item -D.
The computer based
assessment system was
not able to ensure a
uniform level of difficulty
for all candidates because
there was no
differentiation between the
very easy questions and
most difficult.
Item-E.
1. The corrective action – the
continual development of new
questions with a rate of 300 new
questions per month in every level
and department until March 2018-
has been taken into consideration.
1. The corrective action –
differentiation between very easy
and most difficult questions – has
been taken into consideration
1. Training of the members of the
BOE.
j. Evidence on the training received by the
members of the BOE at the “Development
Academy of the Philippines’ 2.
e. Statistics on the result of development 3.
of new questions
h. Details on the measures taken tor
decided to improve the “design and 4.
development of test items”
5.
6.
f. Details on the measures taken or
decided to be taken to ensure 7.
“Differentiation between very easy and
most difficult questions” and the
allocation of the questions within the
questionnaires, depending on their level of
difficulty.
8.
e. Statistics on the result of development 9.
of new questions
identify problems why the BOE was not
able to follow it;
Recalibrate the said quality procedure to
ensure that the processes involved are
practically doable;
Conduct re-orientation to all members of
the BOE and all other EAD concerned
personnel for any new procedure
resulting from the review;
Present evidence of Training of all BOE
Members in the construction, verification
and validation of Test Items;
Closely monitor the manner in which the
BOE members are
developing/constructing test questions;
Ensure that all questions being developed
by BOE should be properly referenced
(complete with source such as book title,
author and page number);
Collaborate with all MTI’s to perform
series of Pilot Testing and validation to all
newly developed Test Questions then
perform item analysis in order to
categorize each from very easy to very
difficult;
Conduct system audit on the MARINA
computerized examination system if it
has categorization of questions in its
system;
Reprogram computerized examination
system to include categorization of test
questions in its data bank and set number
2
The relevance of some
questions to the
competence subject being
evaluated, and their
difficulty associated with
the operational or
management level, could
not be fully demonstrated.
Item -F.
Similar questions within
the database could be
generated in the same
questionnaires
Item -G.
Validation of Scenarios
The Board of Officers
could not demonstrated
that they have established
criteria for the validation of
scenarios used on
simulators to ensure a
uniform level of
assessment throughout all
AC’s.
Item-H
2. Improving the system on the
design and development of test
items
1. Establishing a new digitalized
system to ensure that similar
questions cannot be generated in
the same questionnaire has been
taken into consideration.
1. Training of the members of the
BOE in a specialized institution
“Development Academy of the
Philippines” to build and improve
their knowledge on the
development of examinations and
assessments
2. Development of tools for
competence mapping to ensure
alignment of performance
standards and criteria vis-à-vis
assessments conducted by the
AC’s to ensure a uniform level of
assessment.
1. The corrective action provided in
Annex 9 to the VCAP – proposed
f. Details on the measures taken or
decided to be taken to ensure
“Differentiation between very easy and
most difficult questions” and the
allocation of the questions within the
questionnaires, depending on their level of
difficulty
i. Details on the technical specifications of
the “new digitalized system” and
information on the timeframe established
for its development and commissioning.
g. Approved versions of circulars related
to the certification under Regulations II/1,
II/2, III/1. III/2, III/3, and III/6 together with 13. Validate and approve 10 sets of practical
their annexes establishing the
examination subjects and the pass marks
k. Details on the “tools for competence
mapping to ensure alignment of
performance standards and criteria vis-à- 14. Ensure that all of the above processes
vis assessments conducted by the AC’s,
to ensure a uniform level of assessment”
and evidence of their use.
k. Details on the “tools for competence
mapping to ensure alignment of
performance standards and criteria vis-à-
of test questions per category to ensure
proper distribution of questions per
examinee; (ex. 15% easy, 70% medium &
15% difficult)
10. Upload new sets of test questions as per
approved procedures;
11. Reprogram computerized examination
system to increase the number of items
per competence randomly generated
during written/computerized
examinations;
12. In cooperation with existing PAC’s,
conduct intensive training to all members
of BOE for the development, review,
verification and validation of Practical &
Simulator based Assessment Scenarios;
assessment scenarios for each functional
level addressing all required
competencies with the assistance and
cooperation of all PAC’s to serve as the
standard national assessment scenarios;
would generate records to evidence
performance of each.
3
MARINA could not
demonstrate that it had
established criteria to be
used by examiners in the
oral examinations of
candidates for certification
as ETO
b) Corrective Actions
ETO Circular- has been taken into
consideration.
2. The proposed new circular for
certification as ETO does not
involve an oral examination; all
candidates will be required to pass
a practical assessment
vis assessments conducted by the AC’s,
to ensure a uniform level of assessment”
and evidence of their use.
j. Evidence on the training received by the
members of the BOE at the “Development
Academy of the Philippines’

Hereunder are the different actions or processes to be taken to ensure prevention of recurrence of the noted deficiencies and likewise address future
requirements:

1. Review and amend the existing IRR of RA 10635 to streamline and include various POWERS and RESPONSIBILITIES of the Board of Examiners;
2. Likewise, the review and amendment of the IRR should also include adequate procedures in the vetting, selection and appointment of the members of the
Board to consider recommendations from MARINA recognized and relevant Professional Associations;
3. Moreover, the existing EAD of MARINA should be restructured in order to place the Office of the Board of Examiners into its rightful perspective and not a
separate entity with in the STCW Office. The EAD should be headed by either the Chairman of the BOMDO or BOMEO. This will place the BOE at the helm
of the EAD;
4. Part of the amendment should also state that all members of the BOE should be appointed by the President of the Philippines. (Appointment by the
Philippine President will ensure that the BOE are functionally independent from any authority in MARINA thus, eliminating any intervention from within the
power groups at MARINA and it will further ensure flexibility in terms of making the examination and assessment reliable and reputable);
5. MARINA Manpower & Development Office should ensure to complete the required number of members of the Board of Examiners through the amended
vetting, selection and appointment procedures;
6. The EAD should employ services of Test Questions Experts/ Consultants and/or English and Grammar Experts/Consultants;
7. Also part of the amendment to the IRR will be the rationalization of the functions of the METSS to be made totally focused only on the monitoring and
supervision of the implementation of Education and Training programs and should not have any functions as regards to the Competences’ Examination
and Assessment systems. EAD should be the lead department for Competence Examination and Assessment;
8. Part of the BOE Powers should be to enable them to issue Memorandum Circulars specifically on pertinent requirements of the Board they are
representing;
9. MARINA should ensure that the number of PAC’s accredited do not exceed the number of members of the BOE for proper monitoring and supervision;

4
10. Regional PAC’s may present challenges in the monitoring and supervision, hence MARINA should consider pre-scheduled conduct of Practical
Assessment on those PAC’s like what is being done to the Theoretical Examination;
11. MARINA should be firm in dealing with MHEI’s that do not comply with the program requirements for BSMT & BSMARE;
12. For every completion of term of a BOE member, MARINA should ensure that proper handover is observed, including but not limited to:
i. Organizational Orientation;
ii. DAP Training (Test Question Construction);
iii. QMS Relevant Procedures Training;
iv. Computer based Examination system familiarization training;
v. Simulator based Scenario development training;
vi. Others as applicable
13. Review existing BSMT & BSMar-E Curricula to identify gaps to the requirements of STCW. In the event of having identifies the gaps, it should be
immediately incorporated to the existing curricular programs without any options. IMO Model Courses 7.03, 7.04, 7.02 & 7.01 could be made as
benchmarks in establishing the gaps;
14. Likewise, to address current certification requirements on II/4 & III/4, MHEI’s (except PMMA & MAAP) should be required to include in their curricular
programs 2 months of relevant seagoing experience of their students, prior to the actual Shipboard Training Program;
15. The Quality Management System of MARINA should be certified by a 3rd party accreditation body to eradicate the initial findings of EMSA.
16. Closer collaboration with stake holders should be strengthened.

5
 3.5 ON-BOARD TRAINING
Section 5.6 of the EMSA Inspection Report
SPECIFICS: (from EMSA Report)
(per “REMARKs” from EMSA Report)
(To submit Appropriate Evidence (AE)
CORRECTIVE ACTION TAKEN INTO
PROBLEM/CONCERNS on measures taken, or decided to be
ACCOUNT
taken, shall include but shall not be
limited to: )
The shipboard training, The corrective Actions taken or However, Section 5.A of the new CMO
part of the BSMT and decided to be taken in relation to on- 70/2017 establishes that
BSMarE programs, is not board training mainly refers to CMO “for BSMT students the 12 months
structured as necessary to No 70/2017, which revises the structured Shipboard training should be
ensure the achievement of provisions under CMO No. 20/2014, enrolled and could be any one of the
the prescribed provided in Annex 16 to the VCAP. following options:
competence by all
students. 1. Twelve months as cadet….,
2. At least 6 months as cadet and 6
months as ratings….
3. 12 months accumulated as
ratings”
Accepting service on board as rating is
not in line with the provisions in Section
A-II/1 para 6.1 of the STCW Code, which
requires Parties to ensure
“That during the required period of sea
going service, the candidate receives
systematic practical training and
experience in the tasks, duties, and
RECOMMENDATIONS ON HOW TO ADDRESS
THE SHORTCOMINGS
(FROM PMMAAAI)
As per review and analysis on the findings as
well as remarks from the EMSA report, the
PMMAAAI came up with two sets of
recommendations grouped as follows:
a) Corrections – proposed immediate
actions to be taken to close out the
noted shortcomings by the EMSA
Auditors; and,
b) Corrective Actions - to prevent the
recurrence of those findings and
improve further the existing system
for future requirements.
a) Corrections
Hereunder are the various actions and
processes deemed necessary to close out
this EMSA findings Shipboard Training:
1. Review and revise CMO No. 70/2017 to
ensure that it conforms with the required
provisions of the new STCW;
6
responsibilities of an officer in charge of 2. MARINA and CHED should come up with
a navigational watch”….
The evidence related to the measures
taken or decided to be taken shall
include, but shall not be limited to:
- The “ 12 months structured
Shipboard Training program “ for
BSMT will be accepted only if
served as “Deck Cadet”;
- The STCW circular on “Shipboard
Training Program” enters into force.
- Evidence on the measures taken to
address the inconsistencies
identified in CMO no 70/2017 related
to:
Whether the MHEIs should only use
MARINA –approved TRBs as
provided by circular on “Shipboard
Training Program”, or they could
also use approved TRBs (ie ISF or
Global Met TRBs updated to include
competence requirements of the
Manila //Amendments)?
Whether the assessment of the outcome
of the ship board training shall be
focused on “assessment of the tasks
performed and competencies acquired
onboard during the required service” or 6. CHED should ensure that the Shipboard
on “written or computerized examination
as well as the panel interview (oral
a common requirement for an Approved
Training Record Book to be used by
Cadets during their Shipboard Training
period;
3. A separate CMO can be issued in line with
the career paths of ratings where
shipboard experience as ratings could
not be recognized as structured
shipboard training leading to Officership.
There should be a corresponding
structured training in lieu of their
shipboard experience as ratings
evidenced by an Approved Training
Record Book;
4. MHEI’s should be required to develop
practical assessment scenarios and oral
examination guide questions based on
the approved TRB to assess the Cadets
and administer the same as a requirement
for graduation; These should be made as
a component to the requirements of
CHED being monitored annually;
5. CHED should require MHEI’s to group the
various competencies included in the
TRB’s and be translated as Subjects
undertaken during the Shipboard Training
phase to evidence their passing the
various competencies required;
Training Office of each MHEI should have
qualified Assessors as Regulations I/6;
7
 assessment) to ensure and demonstrate 7. Objective evidences to these
that the students being assessed have recommendations should be made
acquired the necessary learning available.
outcomes for the issuance of BSMT or
BSMar-E Degree”
b) Corrective Actions:

Hereunder are the different actions or processes to be taken to ensure prevention of recurrence of the noted deficiencies and likewise address future
requirements:

1. CHED & MARINA should meet with all MHEI’s having problems on the cadetship program of their students to entice them to join together and invest on
the acquisition of a Training Ship;
2. CHED & MARINA should strictly enforce all stated requirements on MHEI’s and close down those who refuse to comply;
3. CHED & MARINA can look into the possibilities of adopting a sandwich type of curricular programs to MHEI’s except for PMMA, MAAP & the likes. This
will allow them to have a structured of twice 6 months on- 6 months off type of cadetship education & training program during the third and fourth year
phase of their studies;
4. MARINA should require all Merchant Marine Officers to undergo IMO Model Courses 6.09 & 3.12 as they may become qualified trainors and assessors of
cadets on board as per STCW Regulation I/6;
5. CHED and MARINA should ensure proper & strict monitoring on the implementation of the Maritime Education and Training programs of all MHEI’s
including those, which were not approved to conduct such programs;