Professional Documents
Culture Documents
Internal Evaluation Program 11 01 2008
Internal Evaluation Program 11 01 2008
Evaluation
Program
Manual
IEP
Internal Evaluation Program Manual
Preface
Preface
PRF.1 Intent
This manual is intended to stress self audit responsibilities of individual employees as well as the
evaluation responsibilities of top management to ensure that company policies and procedures
provide for safety and regulatory compliance and allow individuals to work safely and efficiently.
PRF.2 Compliance
The procedures and policies in this manual are to be followed by all company and contracted per-
sonnel.
Record of Revisions
Record the revision number and date of insertion on this page. All additions, corrections, or
changes to manuals will be issued as manual revisions and will be numbered in ascending numer-
ical sequence. Revisions shall be incorporated into the manual as soon as possible, but no later
than the published effective date, and shall not be inserted out of numerical sequence unless so
instructed on the Instruction page that accompanies the revision.
Rev. No. Date Inserted Initials Rev. No. Date Inserted Initials
00 Original - 15 JAN 2004 TR 25
01 03/15/04 TR 26
02 05/01/04 TR 27
03 08/15/04 CC 28
04 11/01/04 TR 29
05 11/15/04 TR 30
06 01/01/05 MD 31
07 05/15/05 MD 32
08 07/15/05 MD 33
09 08/15/05 MD 34
10 09/15/05 MD 35
11 01/15/06 CH 36
12 04/01/07 DM 37
13 11/15/07 CH 38
14 03/01/08 KB 39
15 11/01/08 BAC 40
16 41
17 42
18 43
19 44
20 45
21 46
22 47
23 48
24 49
Record of Bulletins
Bulletins are issued on an as needed basis to convey timely information. Bulletin information may
supersede current information found in this manual or introduce new procedures due to changes
in the regulatory or operating environment. Bulletins are inserted in the area so designated in the
“Placement” section, at the top of the page, for each part of the bulletin. Their insertion or removal
is recorded below. CD-ROM versions of this manual will have the bulletin inserted, the bulletin
Record page updated, and a list of all active bulletins will be printed on the CD. Bulletins will be
removed once the material covered is incorporated into a manual revision (usually the next revi-
sion), or is no longer valid. Bulletins will be numbered consecutively unless otherwise noted. It is
the responsibility of the manual holder to insert bulletins by the date specified on the Manual Status
Report..
Revision Highlights
Non-Regulated
Non-Regulated
Table of Contents
Preface.........................................................................................................PRF-1
PRF.1 Intent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PRF-1
PRF.2 Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PRF-1
PRF.3 Proposing Manual Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PRF-1
PRF.4 Manual Revision Status. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PRF-1
PRF.5 Classification of this Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PRF-1
PRF.6 Proprietary Company Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PRF-1
Record of Revisions..................................................................................ROR-1
Record of Bulletins....................................................................................ROB-1
Revision Highlights ...................................................................................HGH-1
List of Controlled Pages ............................................................................LCP-1
Table of Contents ...................................................................................... TOC-1
Chapter 1. Definitions
1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-1
1.1.1 Internal Evaluation Safety Program: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-1
1.1.2 Comprehensive Evaluations: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-1
1.1.3 Ongoing Evaluations: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-1
1.1.4 Special Evaluations: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-1
1.1.5 Control: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-2
1.1.6 Corrective Action: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-2
1.1.7 Satisfactory Fix: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-2
1.1.8 Evidence: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-2
1.1.9 Inspections: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-2
1.1.10 Audits: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-2
1.1.11 Evaluation: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-3
1.1.12 Class A Findings:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-3
1.1.13 Class B Findings:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-3
1.1.14 Concerns: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-3
1.1.15 Repeat Findings: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-3
1.1.16 Follow up Inspections: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-3
1.1.17 Safety Coordinator/Specialist Qualifications:. . . . . . . . . . . . . . . . . . . . . . . .1-4
1.1.18 Safety Coordinator/Specialist Training . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-4
Chapter 1
Definitions
1.1 Terminology
The following pages identify certain terminology, phrases and acronyms that the Safety and Com-
pliance Department use throughout this manual and in its daily activities.
1.1.5 Control:
A control is a key procedure, responsibility or decision-making process within the company
organization. Comprehensive evaluations will focus on verifying and testing the controls within
the maintenance, flight operations, customer service and ground support and administration
departments.
1.1.8 Evidence:
Evidence is a statement of fact prepared by a Coordinator/Specialist that is based on obser-
vations, measurements or tests that can be verified. Evidence should generally be in the form
of a written document or report that supports the Safety and Compliance Department's anal-
ysis and review. This data is necessary to substantiate findings or concerns and to enable
management or evaluators to determine the root cause of any reported finding. Objective evi-
dence generally comes from the following elements:
- Documents or manuals reviewed.
- Equipment examined.
- Activities observed.
- Interview data.
1.1.9 Inspections:
An inspection consists of observing a particular event or action to ensure that correct policy,
procedures and requirements are followed. The primary purpose of an inspection is to verify
that established standards are followed during the observed event or action.
1.1.10 Audits:
An audit is a methodical, planned review of actual business practices compared with estab-
lished policy and procedures. A Coordinator/Specialist may:
- Interview Personnel
- Review Documents
- Observe Operations
- Select Samples
- Inspect activities
- Document results
An audit builds on the principles of inspection. The results of inspection assist in the audit
objective of determining whether business is being conducted in accordance with established
policies and procedures. During an audit, qualified personnel look for the existence of a sys-
temic problem, but do not estimate the size of a problem, and will keep in mind the company’s
“Risk Assessment Program” which is outlined in the Safety Policies and Procedures Manual,
SPPM, when assigning levels of concern. The results (findings and concerns) of an audit
should be documented and presented to Management who then decides how to address the
audit results.
NOTE: The information gathered in the audit will be confidential and is considered privileged under
Chautauqua Airlines Safety Department policy. If you should choose to share this information with
any third party, you do so at your own accord.
1.1.11 Evaluation:
An evaluation is an independent review of company policies, procedures and systems. An
evaluation should be a comprehensive and continual process that considers the results of
audits, overall effectiveness of the management organization in achieving stated objectives,
and the ability of management to respond to new challenges. The evaluation process builds
on the concept of audits and inspections. An evaluation is a predictable process and is
designed to identify and correct potential findings before they occur.
1.1.14 Concerns:
Concerns pertain to conditions that do not meet Department of Defense Commercial Air Car-
rier Quality and Safety Requirements, or to conditions that if left uncorrected, could lead to
increased risk of injury to personnel or damage to equipment. Concerns should be addressed
and monitored for timely resolution.
Chapter 2
Internal Evaluation Program Elements
2.1 Introduction
Chautauqua Airlines’ Internal Evaluation Program (IEP) focuses on the continuing surveillance of
the Maintenance, Operations, and Flight departments. The objectives of this program are as fol-
lows:
• to assess the adequacy of managerial controls in key programs and systems
• to detect the root causes of deficiencies
• to detect needed improvements to company procedures before deficiencies occur
• to develop corrective action plans for identified deficiencies
• to perform followup evaluations when necessary
• Chautauqua Airlines’ IEP is designed to identify potential problem areas before accidents,
incidents, and findings occur, and to ensure upper management is involved with safety and
compliance issues. The IEP will be maintained and managed by the Director of Safety.
The Internal Evaluation Program does not relieve each department of its own responsibility to con-
duct ongoing evaluations of their own procedures to ensure compliance with Chautauqua Airlines
policy, Federal Aviation Regulations and other external requirements.
2.3.4 Drug and Alcohol Testing Program, Publications and System Controls Audits
The Safety and Compliance Department will conduct a Drug and Alcohol Testing Program
Audit annually, tentatively scheduled in the month of April; the Publications Audit annually, ten-
tatively scheduled for the month of August; and the System Control Audit annually, tentatively
scheduled for the month of October.
2.6 Records
All completed audits, including audit forms, reports and the written responses to finding(s) and/or
concerns, along with follow-up reports, will be maintained by the Safety Department for a minimum
of two years. All such documents dated after October 2001 will be stored electronically on the
Safety Department's hard drive, "corporate\corpdocs\Safety" under "Completed Audits."
Chapter 3
Areas Covered During Evaluations
3.1 Introduction
The Safety and Compliance Department will conduct comprehensive evaluations of the functional
areas within the airline on a continuing basis. The basic operating departments that will be audited
are the Maintenance, Flight Operations, Inflight Services and Customer Service departments.
These evaluations will focus on the effectiveness of controls and ongoing evaluations within each
department, communication within and between departments, and the effect that company policy
and developments have on departmental objectives, and shall include areas not reviewed during
other evaluations.
Functional areas within the airline, such as Maintenance, Flight Operations, Customer Service,
Inflight Services, Security and Ground Support will be audited on a continuing basis. Individual
managers within the departments are responsible for periodically conducting evaluations of their
functional areas.
To effectively identify potential problem areas and correct discrepancies before actual findings
occur, the Safety and Compliance Department will enforce a continual evaluation program. A con-
tinual evaluation program is needed to verify whether findings are isolated instances or actual
symptoms of policy, procedural or managerial problems. The continual evaluation program will be
conducted on annual review cycle. Special evaluations will be conducted when trends are identi-
fied.
3.2.2 Operations
1. Training Programs and Record keeping
a) Pilot
b) Flight Attendant
c) Dispatcher
d) Customer Service
e) Maintenance
f) Security
2. Crew qualification
a) Pilot, Flight Attendant and Dispatcher qualification records
b) Crew and Dispatcher scheduling and assignments
c) Crew and Dispatcher duty records
3. Manuals, Policies and Procedures
a) General Operations Manual
b) General Policies Manual
c) Aircraft Minimum Equipment Lists
d) Pilot Operating Handbooks
e) Flight Attendant Manual
f) Read and Initial file system and control
g) Dispatcher and Crew scheduling manual
4. System Operational Control
a) Communication systems
b) Crew briefing, alert system
5. Station Operations
a) Station flight records retention
b) Radio logs
c) Customer Service
d) Security
a. Ground Support
1) Facility and Housekeeping.
2) Motorized Equipment Program.
3) Non-motorized Equipment Program.
4) Ramp Service Vehicle Lockout Program.
5) Ramp Housekeeping and vehicle condition.
3.2.3 Safety
1. Internal Evaluation Program
2. Safety Reporting System
3. Emergency Response Plan
3.3 Checklists
Checklists developed to aid in performing audits and evaluations are located with the Internal Eval-
uation Manual, under the ‘Tech Pubs’ tab, ‘Electronic Manuals Library’, on the company Intranet.
These checklists may also be obtained from the Safety Department upon request. The checklists
posted on the Intranet include the following:
• 01_ Safety Internal Audit Checklist
• 02_Pilot Training
• 03_Crew Scheduling
• 04_Enroute Operations
• 05_System Control
• 06_Flight Operations Audit Checklist
• 07_Drug & Alcohol Testing Program
• 08_Inflight Services
• 20_Quality Control and Engineering Internal Audit Checklist
• 21_Materials Internal Audit Checklist
• 22_Maintenance Internal Audit Checklist
• 22_Maintenance Internal Audit (G & H)
• 23_Production Control
• 24_Station Internal Audit Checklist
• 25_Publications Internal Audit Checklist
• 26_Hazmat and Dangerous Goods
• 27_Occupational Safety and Health
Appendix A
References
References and Guidance Material
Aviation Safety Programs: A Management Handbook, Chaps. 4, 11, 18, Wood, Richard H., Jeppe-
sen Sanderson, 1997.
Flight Safety Digest, Consensus on Need for FAA Guidance Helps Propel Era of Director of Safety,
Flight Safety Foundation, January, 2000.
Joint Flight Standards Handbook Bulletin for Air Transportation: HBAT 99-19: 14 CFR Part 121 and
135 Air Carrier Safety Departments, Programs, and the Director of Safety, FAA, 11/30/99.
The Practice of Aviation Safety: Pg. 8-11, Flight Safety Foundation. June 1990.
Incident Analysis Programs, in Air Line Pilot, APLA, Oct. 1999.
Operator's Flight Safety Handbook, Secs. 2, 3, Global Aviation Information Network (GAIN), Work-
ing Group A, June 2000, Issue 1.
The BASIS for Safety Management, Holton, Mike, FOCUS (British Airways Safety Services), Nov.
1991.