1
A Healthcare Facilities Qualitative and Multivariate Quantitative Assessment
Methodology
Abstract— A qualitative analysis, multivariate analysis and
a scoring system is proposed as the methodology to evaluate a
healthcare infrastructure and technology city network. The
methodology requires visiting each facility; interviewing the
staff; completing an infrastructure checklist; assessing each
room including area measurements; making pictures of each
room, all the installations and all the major infrastructure
elements, making a full equipment inventory including the
working conditions of the equipment; identifying the exact
location of all the inpatient beds of each facility and writing an
evaluation report. The collected information is then cleaned to
do a multivariate analysis with “R”. The assessment of each
facility is focused on 4 areas (dimensions): infrastructure,
installations, support services and equipment. A compound
score is calculated using weights for each component that
builds each dimension. The weights are defined by a group of
experts. The resulting scores for each dimension of each facili-
ty are then evaluated and crosschecked with the pictures and
the evaluation (qualitative) report. The level of confidence of
the collected information is also rated. The proposed method-
ology was used to assess 33 healthcare facilities of Ulaanbaatar
City in Mongolia: 9 specialized hospitals, 5 District Health
Alliances, 6 district hospitals, 6 secondary general hospitals, 3
maternity hospitals and 4 Village Hospitals. The obtained
results were used to prioritize new investments for the health
sector over the next years. Further work is required in order
to adjust the proposed methodology to better identify or classi-
fy a facility’s conditions, adding more answer options and
including more experts in the weight definition stage.
Keywords— Healthcare Facilities, Assessment, Scoring Sys-
tem, Healthcare Technology.
I. INTRODUCTION
The Health Sector Strategic Master Plan 2006-2015
(HSSMP) [1] under the Pharmaceutical and Support Ser-
vices chapter defined for the first time the need for system-
atic approach to plan, procure, maintain and evaluate
Healthcare Facilities and Healthcare Technology (HT).
The Strategic Analysis of the PPTA for the Fourth Health
Sector Development Project (FHSDP) (written in May
2010) reported that “A sector wide survey was conducted in
2005 to evaluate the standards on medical equipment and it
found that approximately 60 percent of major medical
equipment in Mongolia had been provided before 1990s.”
[4].
The budget between 2010 and 2012 was said to be
46,800 million tugrik (28 million EUR) for medical equip-
ment. In 2012 the budget for major public hospital invest-
ment was already 41,456 million tugrik (23 million EUR)
claib2014_HealthcareFacilitiesAssessment_NoAuthors.docx
and in 2013 the budget just for medical equipment was set
in 9,300 million tugrik (4,65 million EUR)[4].
The boom in healthcare investments is clearly related to
the mining sector boom the country is experiencing since
2007, which affects all areas of investment. As a result a
Capital Planning System becomes mandatory to rationalize,
assess, plan, implement, audit and monitor all healthcare
investments.
A first step for any capital (assets) planning is to clearly
identify the current status of the current assets.
There have been only two previous experiences on
equipment inventories in 2005 and then 2009 but none of
them included any information on facilities, any of them
followed a comprehensive analytical methodology and the
collected data was not available for any further planning or
monitoring purposes.
Ulaanbaatar has a total of 38 public hospitals: 3 Village
Hospitals, 8 district hospitals, 7 general hospitals, 8 District
Health Alliances and 12 specialized hospitals. There was no
information to allow any future investment planning until
now.
The evaluation of the infrastructure and equipment of the
healthcare facilities network aims to assist the ministry of
health in the capital planning process.
A rigorous methodology should be followed to allow a
future replication and monitoring of the assessment.
No detailed methodology was found in the bibliography.
A final report for the development of the National Hospital
Master Plan for the Republic of Moldova provided an idea
for the proposed approach [5].
II. METHODOLOGY
The methodology can be divided in two blocks, the sur-
vey methodology and the analysis methodology.
A. Survey Methodology
The Evaluation of each facility requires a complete walk
around the facility, interviewing responsible staff, making
pictures of all relevant rooms, infrastructure issues, equip-
ment, prepare an inventory with all the equipment, measure
all room, copy available drawings, put all the collected
information in preformatted excel sheets and prepare a final
report with the main observations from the site visit.
A number of forms were already preformatted and pre-
pared for the surveyors to do the assessment following a
predefined protocol.
The forms to be filled in where the following ones:
2

• Assessment checklist The checklist was divided in 7 sections and each section
• Rooms per department (with sqm) had subsections. Each one of these sections is considered to
• Beds per department have a greater or lower impact in each level of facilities. For
• Main medical equipment inventory instance, the existence of a “Hospital Information System”
Additionally to complete the file of each facility the fol- (HIS) may be considered to be crucial in a third level
lowing documents were also necessary: hospital while it is considered as not important in a village
• Pictures of the visit hospital. In this way the weights across different levels may
• Architecture drawings when available have different values.
• A report of the visit covering the main points included
in the Assessment Checklist adding any additional personal Descriptive Analysis: A descriptive analysis was done on
comments or views out of the visit. the first section of the checklist.
This paper is focused just on the assessment checklist
and the crosscheck done between this checklist and others Applied Weights: At the same time different components
documents obtained during the assessment. of the same sections may be more important than others.
For instance in the case of the construction section, the
Assessment Checklist: The assessment checklist is a doc- foundation is going to be much more important than the
ument that covers all the possible areas of interest in terms façade condition: bad condition of the foundation may put
of infrastructure and healthcare technology. It includes the in danger the live of people inside the building while the
following chapters: façade may just have an aesthetic impact.
1. General Information The list of concepts to consider were rated by a couple of
2. Clinical Services experts with experience in hospital design and construction.
3. Clinical Support Services An average weight for each component was then calculated
4. Non Clinical Support Services and applied to the results of the surveys.
5. Administration
6. Building, Infrastructure, Engineering Conditions Table 1: Example of used weights
7. Equipment Inventory
8. Schedule of Accommodations Sec- 3rd 2nd 1st
Code Description
tion level level level
B. Quantitative and Qualitative Analysis Methodology 2.1 General Conditions 3 3 3
2.2 Roof 4 5 5
The analysis methodology used for the assessment 2.3 Facade (outside surface) 3 2 2
checklists of the study is based on statistical techniques, 2.4 Outside walls (structure) 5 5 5
Infrastructure

from descriptive analysis of the variables of interest using a 2.5 Windows / doors 4 4 4
2.6 Internal walls and partitions 4 3 3
“Scoring system model”. 2.7 Slabs 5 5 5
2.8 Floor finishing 2 3 3
2.9 Wall finishing 2 3 3
2.10 Ceiling construction 3 3 3
2.11 Foundation 5 5 5
3.2.8 Stairs (qty., types) 2 1 1

Multivariate Analysis: The survey can be considered as a

Figure 1: Scoring System Model model of a complex system [2] that in this particular case
has 82 variables. All these variables evolve at the same time
so the multivariate analysis can be used to reduce the num-
The modeling process of the scoring system took into
ber of variables to a reduced number of dimensions that
consideration not only the installation conditions but also
actually would still represent the system. In the study case
the importance of each variable according to the level of the
the model was reduced to 5 vectors that were able to repro-
facility and the importance of having such conditions. At
duce the results of 31 of the 33 cases under study. A model
the end, there would be a score for each part (dimension) of
with 8 vectors would have been able to reproduce all 33
the survey and a final rating for every facility which can be
cases.
used to determine the need for future capital investments
(renovations, new equipment and new buildings).

claib2014_HealthcareFacilitiesAssessment_NoAuthors.docx
 3

The multivariate analysis was also used to obtain the

high correlation between different variables and to detect
the level of mistake in the surveys.

Scoring Analysis: The scoring was applied to 6 of the 7

sections of the checklist with the proposed weights. The
scores were calculated in four major dimensions of interest:
Infrastructure, Installations, Support Services and Equip-
ment.

Figure 4: UB City healthcare facilities scores for equipment

III. RESULTS

A. The survey
The implementation of the designed survey required over
6 months for the collection, pre processing of the data and
reporting each case. It was completed by a working group
of 15 Mongolian biomedical engineers [3].

B. Scoring
Scores were calculated for four dimensions: infrastruc- Figure 5: UB city healthcare facilities scores for support services
ture, installations, equipment and services. A final score
was then calculated with the combination of the previous 4
scores. The next figures show the results obtained for each
hospital for each dimension and the final score as a combi-
nation for all of them:

Figure 6: Final scores

The contribution of each dimension to the final score us-

ing the proposed weight is represented in Figure 7:
Figure 2: UB city healthcare facilities scores for infrastructure

Figure 3: UB city healthcare facilities scores for installations

claib2014_HealthcareFacilitiesAssessment_NoAuthors.docx
4

V. CONCLUSIONS

The methodology proved to be useful for the objectives

of the assignment and may become the main tool evaluation
of other facilities in the country.
The results show that all the facilities require some kind
of works that may go from the full reconstruction to chang-
es in just a few services.
A revision of the survey checklists should consider the
inclusion of additional verification questions, a broader
range of answers and a larger group of experts to determine
the scoring system weights.

Figure 7: Contribution weight of each area to the final score

ACKNOWLEDGMENT
A special thank to the people that helped us test the pro-
A line was draw at 0,3 (or 30%) of the scores given the
posed methodology, starting by the working group from the
fact the survey had 3 options for most of the questions (bad,
Mongolian University of Science and Technology that did
average and good) and given that out of those three possible
the initial data collection; Yuchen Li, for helping us prepar-
statuses a score below 30% would mean either the need for
ing the data for the statistical analysis, to Anna Worm for
new construction, new installation, or new equipment or a
helping us putting together the all the collected equipment
new. At the same time any value between 30% and 65% is
inventory and to the rest of the Meirovich Consulting team
considered not to provide conclusive results and would
and FHSDP for their continuous support.
require further evaluation to determine of any kind of reha-
bilitation is required. Any value over 65% would be consid-
ered as acceptable but allowing space for improvement. CONFLICT OF INTEREST
The authors declare that they have no conflict of interest.
IV. DISCUSSION

The results show very low scores (between 10% and in
very few cases just above 50%). The reason for such low
scores is clearly the fact there has been almost no new con-
struction, renovation or equipment for over 20 years and
that today the facilities are in alarming conditions. Most of
the results could be confirmed by direct qualitative evalua-
tion done by healthcare facilities experts. The proposed
scoring system however, would have the advantage of
providing also a quantitative measure to prioritize the works
to be implemented at a national or network (regional) level.
REFERENCES
1. ADB. Project preparation technical assistance fourth health sec-
tor development project improving sector governance - Strate-
gic Analysis - TA 7309-MON. Asian Bank of Development,
Ulaanbaatar, Mongolia, 2010.
2. Johnson, R. A., Wichern, D. W. (2007). Applied Multivariate
Statistical Analysis (Sixth ed.)
3. Meirovich C, Bold A. et al. (2014) First Ulaanbaatar City Hospi-
tal Network Infrastructure and Healthcare Technology Assess-
ment, PAHCE 2014 Pan American Health Care Exchanges,
Brasilia, Brazil, 2004, in press.
4. Ministry of Health. The implementation framework of the health
sector strategic master plan - 2006-2015. Ulaanbaatar, Mongo-
lia: Supported by MHLW & JICWELS, Japan., 2007
5. Top Consult Köln GmbH. (2009) National Hospital Master Plan
2009-2018 for the Republic of Moldova. Final Report.

claib2014_HealthcareFacilitiesAssessment_NoAuthors.docx