cityof CINCINNATI Interdepartment Corespondence Sheet Date: January 6, 2017 FOR YOUR INFORMATION MEMO To: Mayor and Members of City Council From: Harry Black, City Manage} Subject: Emergency Communications Outage Please see the attached summary of an outage experienced at the 911 call center yesterday determined to be the result of a software malfunction by one of Cincinnati Bell's subcontractors. My office and the Cincinnati Police Department are engaging with Cincinnati Bell on a number of fronts, to ensure solutions are reached to prevent this from occurring again. Emergency communications represents a critical public service and maintaining service levels is non-negotiable. Additional information about what occurred and how it will be Prevented from occurring in the future is being gathered by Cincinnati Bell. Ce. Eliot Isaac, Police ChiefCINCINNATI Interdepartmental Correspondance Sheet Date: January 6, 2017 FOR YOUR INFORMATION To: Harry Black, City Manager From: Colonel Ellot K. Isaac, Police Chief Subject: Emergency Communications Center Outage BACKGROUND On January 5, 2017 at 10:56 a.m., the Emergency Communication Center (ECC) experienced a critical failure. ‘The Center discovered it was not receiving incoming calls for service on the 911 system. As CPD made preparations to make notifications to City Administration and the community via traditional and social media, the system came back online and ECC began receiving calls at 11:18 a.m, ‘SYNOPSIS As soon as it was leamed ECC was not receiving incoming calls, supervisors quickly began investigating the situation, possible causes, and immediately made notifications. Within minutes It was determined that alls were not being routed through our vendor, Cincinnati Bell. Typically, there are several contingencies automatically built into to system when these types of failures occur: * Calls that are not able to be routed to ECC should be automatically routed to the Hamilton County Communications Center for processing. * When the system comes back up, it should provide a log of missed calls and incoming calls at the Cincinnati Bell server location and also on the ECC telephone system. At this juncture, we are not clear if any of these contingencies occurred. Preliminary information from Cincinnati Bell indicates their system was not processing calls and point to a software malfunction provided by their subcontractor, COMTECHTEL. The maifunction prevented 911 calls from being processed from all Public Safety Answering Points within the Cincinnati Bell service area which includes areas within Kentucky, Indiana, and Ohio. Cincinnati Bell is currently working with thelr vendor, COMTECHTEL to determine the exact cause(s) of the failure and will soon provide the City a report outlining those details and possible solutions. We have been working directly with Cincinnati Bell on this matter, including submitting a series of questions about this incident, contingencies and solutions. We will continue to engage with them until their vendor issues are resolved.