Professional Documents
Culture Documents
MCBTA Report FINAL
MCBTA Report FINAL
Maintenance Competency-Based
Training Assessments (MCBTA)
Recommended practices
Enter
Maintenance Competency-Based Training Assessments (MCBTA) Contents 2
Section 1
Executive Summary
In the modern maintenance environment, Communication is key for the process. An investment in competency assessment
there are a number of roles within an Opening a two-way dialogue between the is an investment in the company, its people,
organisation and each one of them requires assessor and assessed candidate sets the and it’s future planning. Whilst there
certain strengths and abilities that draw on tone for future competency assessment are legislative minimum requirements,
both non-technical and technical skills. periods. This process should form part of the dependent on geographical placement,
induction program and information should it would be prudent for any organisation
The benefits of a comprehensive and also continue to be shared through mediums to exceed these with the aim of enhanced
functioning Competency Management such as Continuation Training. The process is appreciation of training needs, greater data
System (CMS) for a business are many. dependent on both parties feeling confident harvesting, closer interaction and trust
It allows for a company to assess availability that they can convey their thoughts on the building. For the individual this may present
of skillsets at a corporate and individual process and agree on future training, role an opportunity for further enhancement
level, determine whether current training and authorisations. of responsibilities, definition of career path
is sufficient, and help plan for future and growth.
development of the workforce. For the It is important that these skills are assessed
candidate, it is an opportunity to discuss alongside practical experience to determine
their strengths and weaknesses, identify not only the depth and availability of skills
where they feel that more training at a company, but also to determine future
would be beneficial, and to discuss their planning of training and support for the
responsibilities within the organisation. workforce. This support is not only restricted
Confirming the company’s expectations of to manning levels but, by open conversation,
them and their job description may lead to can be used to assess where there might be
increased performance. shortfalls within the organisation regarding
infrastructure such as tooling, hangarage and
safety concerns.
Maintenance
Maintenance Competency-Based
Competency-BasedTraining
Training
Assessments
Assessments
(MCBTA)
(MCBTA) Section
Introduction
heading Contents 5
Section 2 Introduction
Maintenance Competency-Based Training Assessments (MCBTA) Introduction Contents 6
Section 2
Introduction
Competency can be described as are underlying traits that can translated to Nearly 20% of fatal accidents in the offshore of errors (HSE, 2022). CBTA has a role in
“A dimension of human performance data that can be developed to help effective aviation industry are attributed to System identifying areas which may lead to errors
that is used to reliably predict successful strategies to improve performance. Reliability and Resilience (HeliOffshore, as well as providing a tool to encourage
performance on the job. A competency 2020). These incidents include both feedback in the systems put in place. In the
is manifested and observed through Reviewing competence has been technical and the human interface element. wider aviation sector, it was observed that of
behaviors that mobilise the relevant commonplace amongst the safety critical Whilst there has significant research in the maintenance related AAIB safety reports
knowledge, skills and attitudes to carry sections, Aircrew, Air Traffic Control and the field of Human Factors and Human that 60% of incidents were attributed to
out activities or tasks under specified Maintenance, of the operational side of Performance, the human part of the process incorrectly installed components, see
conditions.” (IATA, 2021) aviation for some time with research mainly is the one that can lead to a variation Figure 1, (RAeS, 2022).
concentrated on Aircrew with regulation
The modern-day airworthiness professional regarding this in place amongst all the
regardless of whether they work in regulation authorities. There has, however,
Continuing Airworthiness, Planning or on been significant research carried out by the Component not installed,
the hangar floor as a Certifying Engineer UK CAA with the release of CAP 1715 and 2%
or mechanic, now have to employ a wider ICAO with DOC 10098 recently that has put Inadequate control of maintenance,
4%
range of skills that may not have been part the spotlight on guiding organisations on Incorrect component installed,
of their job description in past generations. the standards required. EASA has stipulated 0%
Damage not properly assessed,
There are now further demands on a that CBTA is a requirement for maintenance 4% 1
maintenance professionals time and personnel as stated in their latest Acceptable 2
Inadequate servicing, 2
mindset that may impinge on their ability Means of Compliance (AMC). 5% 3
to do the task required. It is important
that there is a mechanism in place to Reviewing the competency skills and gaps
allow the organisation and the individual within an organisation allows for the analysis
to assess their performance. By reviewing of performance of the business. It also allows Damage not detected, 14 33 Component installed incorrectly,
these skills, it offers an opportunity for for further analysis of behaviour patterns 25% 60%
personal development as well as business within certain sections and job descriptions,
development. by using this data it could be possible
to plan for future strategies, manpower
Competency assessments have been in use expectations and planning. For the
since the 1950s in wider society with the individual it also represents an opportunity
general formalisation of a framework by to assess their own performance and look
psychologists at Harvard University that forward to where possible future personal
would allow measuring competencies. These development. Figure 1, Type of Maintenance Error identified in the analysis of AAIB Reports
Maintenance Competency-Based Training Assessments (MCBTA) Introduction Contents 7
Attitude Air Traffic Control Bird Strike Airborne Collision High Intensity Alerting
Collision in Air
Management Oversight Prevention Avoidance System Strobe Lights
Ground SAR/Emergency
Weight and Passenger Flight Dangerous Security
Collision/ Response
Balance Briefing Handling Goods Control
Handling
Fuel
(Hot) Refuelling Fuel Flight Fuel Fuel Testing/ Post-Accident
Exhaustion/
Procedures Checks Planning Reserves Inspection
Contamination
Safety
Multi-crew Personnel Modern/Proven Standards &
Enablers Leadership/ Effective SMS Info Sharing Competency Operations Readiness Technology Oversight
Culture
Section 3
Competency-Based Training and Assessment (CBTA)
CBTA is a performance assessment tool that CBTA and EBT assessments should be
uses the training, knowledge, and experience carried out by a trained assessors who are Current regulations on CBTA
of a candidate to assess their capability familiar with the job descriptions roles and CBTA has been on the radar of regulators for years, initially focusing on aircrew
to carry out their tasks. This system can responsibilities ideally having experienced and various organisations are now offering frameworks specifically aimed at the
also be used to define future training and the tasks. This training can be carried out technical sector:
development scenarios. by the in-house training department or by
third-party specialists. • CAP 1715 Competency Assessment Guidance Document: This document, was
The organisation can also utilise the developed in collaboration with industry and was first published in 2018 to provide
data to evaluate their capabilities as a EBT uses a set of competencies that do guidance on competency assessments for all aspects of the technical sector. This
company regarding the strengths of the not necessarily distinguish between the document details a number of “non-technical” behavioural skills that complement the
workforce, future training requirements “non-technical” and “technical” skills. technical skills required to carry out the responsibilities that are part of the candidate’s
and current capacity. CBTA is a system which, ICAO and UK CAA have issued a list of job description.
if installed correctly, can provide a link to competencies that this recommended
improve communication and trust between practices document is based on, a list • EASA AMC1 145.A.30 Personnel Requirements: EASA published the latest update in
all parties. of which can be seen in Carrying out 2022 of the Easy Access Rules for Continuing Airworthiness and this section defines
a Competency Assessment section. objectives and requirements for organisations. This publication also contains suggested
An element of CBTA is Evidence-Based frameworks for knowledge elements required within certain roles (GM2 145.A.30(e)).
Training (EBT). Due to the nature of the
work of the technical staff the use of this EASA states that the organisation is responsible for assessing competency.
program allows for their experience and This document gives instruction for the role the company plays in the process.
training to assess their capabilities and
their future requirements to continue their • ICAO Doc 10098: First edition of the manual released in 2021. This reference guide
development. The advantage of EBT in the provides advice specifically on CBTA for aircraft maintenance personnel.
technical sector is that the candidate can
produce evidence of prior work and training
to establish their skill level and abilities,
to add to this there are ways to replicate
certain scenarios for assessment.
Maintenance Competency-Based Training Assessments (MCBTA) Carrying out a Competency Assessment Contents 10
Section 4
Carrying out a Competency Assessment
Execution of Competency Check Competency Assessment methods Roles and responsibilities of Assessors Competency Factors
It is important that a competency There are many ways to assess the Regardless of the job description that is There are generally three distinct factors
interview has been arranged in good time candidates but essentially four different being assessed there are basic skills that of competence that cover the range
to allow both parties to be organised methods should be made available to both every assessor should possess. of competencies that influence the
regarding preparation for the discussion. the assessor and the candidate: ability of a candidate to undertake their
The interview should be held in a place • The assessor should understand the responsibilities safely and successfully.
where it can be carried out undisturbed • Demonstration: The assessor should philosophy behind the CBTA program. These competencies must be relevant to
at a time convenient for both parties as choose a task that is within the • Comprehensive familiarity and the workplace and the tasks carried out by
well as business operations. It should candidate’s scope of work that the knowledge of the model adopted by the candidate. By observing the behaviours
follow a structure that allows for full and candidate can discuss the process of the company this must include being and actions of the candidate whilst carrying
frank discussion in a framework to keep carrying out the operation. The assessor able to understand the benchmarks that out or reviewing previous activities
a standardised process for all candidates, should use this as an opportunity to will be used to measure competency. that it is possible to determine whether
see Appendix 1. ask questions about the task and the • Knowledge and experience of the job they have reached the accepted level of
method of carrying it out. description that the candidate and the proficiency required.
• Simulation: A task should be carried tasks and responsibilities associated
out within an environment such as with it. • Knowledge – Comprehension of the
a simulator, workshop or where test • Knowledge of the documentation task and relevant information.
equipment can be used to carry out the used by the company and regulator that • Skills – Capability to carry out the task
tasks. The simulation should provide may be used as evidence of competency efficiently, safely and correctly.
a realistic situation where evidence of and training. • Attitude – Approach to the task and
knowledge and skill can be evaluated. • Human Factors/Performance. the team.
• Observation: Where practicable a task
can be carried out that can be witnessed The assessor should be part of the ICAO and the UK CAA have defined their
and apprised by the assessor to check subject’s line management team and be Competency fields, see Appendix 2,
that all processes have been carried out in a supervisory role relating to them. recommended for use in an organisation’s
to a required standard framework. These suggested topics cover
• Questioning: Questions relating both the “non-technical” and “technical”
to the task to allow the candidate skills required to cover all three of the
to demonstrate that they have the Competency Factors either as one or a
knowledge to complete what is required. combination of them.
Maintenance Competency-Based Training Assessments (MCBTA) Carrying out a Competency Assessment Contents 11
Competency Ratings Training Required At the time of interview, the candidate cannot demonstrate adequate knowledge and understanding of the subject. It is therefore
It is important that any ratings system be recommended that appropriate support, mentoring or training package be developed and delivered before reassessment.
simple, clear and unambiguous in its usage.
It should also allow for discussion between Satisfactory The candidate demonstrates sufficient knowledge and understanding of the subject that allows the candidate to perform required tasks
to an acceptable standard.
the assessor and the candidate.
Exceeds Expectations The candidate demonstrates a high level of experience and knowledge of the subject, which allows the candidate to perform required
In the UK CAA CAP 1715 it recommends tasks to a very high standard. The candidate should be considered for additional responsibilities as part of their personal development.
a three-level rating system, see Figure 3.
This allows for simple understanding Figure 3, UK CAA CAP 1715 Competency Ratings (CAA, 2019)
of the assessor’s decision about the
results of their assessment and for future
recommendations on the candidate’s 1. Candidate Name. 8. Date Competency Completed: Date each 12. Competency: To be used in relation
requirements or ability to undertake tasks 2. Date Started: Commencement of individual competency requirement has to the question posed, this allows for
that are within their job description. employment within the company. been completed. multiple competencies to be assessed
3. Competency Interval: How often is this 9. Assessor Signature. within one question.
Competency Framework Pack process carried out, does it change for 10. Candidate Signature. 13. Competency Result: Based on the UK
As part of an assessment there should be different roles. 11. Topic box: Detailing which section is CAA CAP1715 system, allows for easy
a dedicated pack to allow for recording of 4. Job Description: Different roles require to be covered in the question boxes decision making.
the discussion, results, recommendations, different competency questions, underneath, these should be relevant
copies of documentation and feedback. requirements and levels. to the organisation, local regulator and This pack should be kept as a permanent for
As part of the Competency Assessment 5. Date Completed: Completion date of the job requirements. the duration of the candidate’s employment
Pack there should be a form, see Figure 4, entire process. at the company.
to collate and record information either 6. Staff No.: For easy record keeping.
electronically or as a hard copy. This form 7. Aircraft Types and/or authorisations: For
should be clear and easy to follow to recording the aircraft types held and the
allow for unambiguous decision making. authorisation levels associated with the
A suggested form would be; privileges the candidate holds.
Maintenance Competency-Based Training Assessments (MCBTA) Carrying out a Competency Assessment Contents 12
Competency Framework
(1) Candidate Name (2) Date Started (3) Competency
Interval
(7) Aircraft Types and/or Authorisations (8) Date (9) Assessor (10) Candidate
Required
Training
Satisfactory
Expectations
Exceeds
Competency Signature Signature
Completed
Figure 4, Suggested framework form for us in assessments (see Appendix 3 for a blank template).
Maintenance Competency-Based Training Assessments (MCBTA) Carrying out a Competency Assessment Contents 13
Appendices
Maintenance Competency-Based Training Assessments (MCBTA) Appendix 1 Contents 15
Has candidate
supplied suitable
evidence?
Yes
No gaps
Training sufficient
Discuss further
development
Appendix 2
Reference ICAO UK CAA Suggested Benchmarks
Number Competency Competency
2. Work Management • Ability to plan their and their team’s workload effectively
• Effective time management
• Understanding of operational and commercial requirements
• Ability to delegate as required.
• Communicates to other team members when issues may affect the ability to carry out the task on time
• Recognises deviations from normal practice and acts accordingly
3. Situational Awareness • Ability to recognise changes within a situation whether a system or human change and adapt accordingly
• Ability to report changes that may lead to an unsafe environment
• Recognises hazardous or potentially hazardous situations and reports them
5. System Thinking • Recognises the link between departments within a technical organisation
• Recognises the relationship between policies, processes and company procedures
• Recognises the importance of national and international regulations
• Recognises the importance of not only a company’s safety management system but also national and regulatory safety and reporting systems
• Understands how systemic failures within an organisation may affect their or the company’s performance
Appendix 3
This form provides a clear and easy to follow format for CBTA.
Competency Framework
Candidate Name Date Started Competency
Interval
Required
Training
Satisfactory
Expectations
Exceeds
Completed Signature
Bibliography
Maintenance Competency-Based Training Assessments (MCBTA) Bibliography Contents 20
Bibliography
UK CAA, 2019. CAP 1715 Competency Assessment Guidance Document, Gatwick: CAA.
HeliOffshore, 2020. HeliOffshore Safety Performance Report, London: HeliOffshore.
HSE, 2022. Human factors: Managing human failures. [Online]
Available at: https://www.hse.gov.uk/humanfactors/topics/humanfail.htm [Accessed 18 March 2023].
IATA, 2021. Competency-Based Training and Assessment (CBTA) Expansion within the Aviation System, Montreal: IATA.
RAeS, 2022. Development of a strategy to enhance human-centred design for maintenance, London: RAeS.
Maintenance Competency-Based Training Assessments (MCBTA) Contents 21