LEUKOS

The Journal of the Illuminating Engineering Society

ISSN: 1550-2724 (Print) 1550-2716 (Online) Journal homepage: https://www.tandfonline.com/loi/ulks20

Subjective Assessments of Lighting Quality: A

Measurement Review

Alicia C. Allan, Veronica Garcia-Hansen, Gillian Isoardi & Simon S. Smith

To cite this article: Alicia C. Allan, Veronica Garcia-Hansen, Gillian Isoardi & Simon S. Smith
(2019) Subjective Assessments of Lighting Quality: A Measurement Review, LEUKOS, 15:2-3,
115-126, DOI: 10.1080/15502724.2018.1531017

To link to this article: https://doi.org/10.1080/15502724.2018.1531017

© 2019 The Author(s). Published with

license by Taylor & Francis Group, LLC.

Published online: 04 Feb 2019.

Submit your article to this journal

Article views: 5291

View related articles

View Crossmark data

Citing articles: 16 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=ulks20
LEUKOS
2019, VOL. 15, NOS. 2–3, 115–126
https://doi.org/10.1080/15502724.2018.1531017

Subjective Assessments of Lighting Quality: A Measurement Review

Alicia C. Allana, Veronica Garcia-Hansena, Gillian Isoardib, and Simon S. Smithc
a
School of Design, Queensland University of Technology, Brisbane, Australia; bLight Naturally, Brisbane, Australia; cInstitute for Social Science
Research, University of Queensland, Brisbane, Australia

ABSTRACT ARTICLE HISTORY

Lighting is an important component of indoor environmental quality that can affect occupant Received 31 January 2018
satisfaction, well-being and productivity. Lighting quality is a broad abstract concept and this has Revised 11 September 2018
implications for its assessment. Subjective evaluations of lighting are an important complement to Accepted 28 September 2018
objective photometric information; however, there is limited existing guidance for the selection of KEYWORDS
such measures. We review and highlight the advantages and limitations associated with measures Glare; lighting quality;
of general lighting quality and discomfort glare. Existing measures of lighting quality have broad lighting satisfaction;
coverage of individual lighting features but do not always clearly form cohesive scales measuring postoccupancy evaluation;
an underlying construct. Questions used in experimental glare research focus narrowly on glare visual discomfort
severity, with ambiguous response rating scales. There is a need for the development of reliable
and valid tools to assess lighting quality and its components, with clearly defined definitions and
constructs, and explicit reporting of psychometric scale properties. The development of rigorous
self-report tools will improve the understanding and design of quality lighting environments.

1. Introduction individual completing the tasks and activities), sug-

gesting that lighting quality depends on an interac-
Indoor environmental quality has implications for
tion between lighting, place, and person. Evaluation
the satisfaction, well-being, and productivity of
of lighting typically involves detailed photometric
building occupants (Humphreys and Nicol 2007;
measures of the environment; however, photometric
Newsham et al. 2009; Singh et al. 2010). Together
measures alone do not fully capture the “person”
with thermal, acoustic, and air quality components,
component of lighting quality. Established protocols
lighting is a key aspect of indoor environmental
for describing lighting (CIE 213: 2014; International
quality. Lighting can affect occupant well-being and
Energy Agency [IEA] 2016) acknowledge the impor-
productivity, and dissatisfaction with lighting can
tance of capturing subjective evaluations in addition
influence operational costs if it results in redesign
to objective photometric information; however, they
or unanticipated occupant intervention into the
do not provide any summary, comparison, or gui-
environment. Although a desirable outcome, light-
dance for selecting measures. Therefore, it is not
ing quality remains an abstract concept without
clear how existing subjective evaluation measures
a widely agreed definition. Boyce (2014) suggests
overlap and what gaps remain. This review aims to
that lighting quality is best considered in terms of
summarize existing measures of subjective lighting
its impact on outcomes such as visual performance
quality, identify advantages and limitations of exist-
and behavior, and Veitch and Newsham (1996)
ing surveys to inform measure selection, and provide
define lighting quality more broadly, as lighting
recommendations for future measure development.
that supports visual performance, task and beha-
vioral performance, social interactions, mood, health
and safety, and aesthetic judgments. Integral to both 2. Review of measures
definitions are acknowledgement of the purpose of
To identify measures, we conducted database
the space (the tasks or activities that take place in
searches, examined established protocol and
a particular setting) and the user of a space (the
guideline documents from international lighting

CONTACT Alicia C. Allan alicia.allan@qut.edu.au School of Design, Queensland University of Technology, GPO Box 2434, Brisbane 4000, Australia.
Open Access for this manuscript was supported by the Illuminating Engineering Society and International Commission on Illumination.
© 2019 The Author(s). Published with license by Taylor & Francis Group, LLC.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-
nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built
upon in any way.
116 A. C. ALLAN ET AL.
bodies, and examined methods and reference lists
of articles reporting subjective evaluation. We
identified four types of subjective measures: gen-
eral evaluations of lighting quality, questions
examining discomfort glare (one specific feature
of the lighting environment), indoor environmen-
tal quality scales, and affective response scales. Of
these, general lighting quality measures and mea-
sures of glare were the most prevalent and are
reviewed in detail.
2.1. Measures of general lighting quality
Subjective assessments form an important compo-
nent of postoccupancy evaluations (POEs) of the
built environment. There are a number of occupant
self-report measures that include general lighting
quality as either a substantial component or the sole
focus. Tools that have not been published in a peer-
reviewed source or have an associated fee were not
included, because these restrictions limit utility to
researchers, widespread adoption, and comparability
of results. The CIE guide to protocols for describing
lighting (CIE 2014) provides examples of existing
measures, two of which are reviewed here (the
Thorn Lighting Work Space Appraisal was excluded
because it is a proprietary unpublished tool, and the
Ergonomic Lighting indicator [Tralau et al. 2009] is
a lighting design tool that does not capture occupant
ratings). In addition to the CIE protocol examples,
three other measures were included.
Table 1 summarizes the key details of the five
reviewed measures, including the number of ques-
tions, whether non-lighting questions are included,
the type of response format used, the time frame
participants are asked to consider when responding,
and whether psychometric scale properties (e.g.,
Cronbach’s alpha for evaluation of internal consis-
tency of scale items or results of structural validity
via factor analysis) were reported. The remaining
columns indicate what specific features of the light-
ing environment are addressed by the tool.
2.1.1. Office lighting survey
The office lighting survey includes evaluations of
broad (overall satisfaction and comfort; for example,
“Overall, the lighting is comfortable”) and specific
characteristics of lighting (illuminance, brightness
ratio, veiling reflections, discomfort glare, color, and
flicker) over the long term. It includes statements that
assess quality (e.g., “The lights flicker throughout
the day”), comfort (e.g., “Overall, the lighting is com-
fortable”), and performance (e.g., “Reflections from
the light fixtures hinder my work”). It has been used
in both field and experimental settings (Borisuit et al.
2014; Boyce et al. 2006; Eklund and Boyce 1996).
2.1.1.1. Advantages. The survey allows diagnosis
of issues resulting from individual lighting fea-
tures, as well as an overall assessment of lighting
quality. Convergent validity is thoroughly demon-
strated via associations with previously developed
tools, as is test–retest reliability (the stability of
a test over time) over one week. The authors
clearly identify the aim of their measure and ratio-
nale for its development. Normative data for this
survey are reported, providing a practical compar-
ison point for those using the tool in the field.
2.1.1.2. Limitations. The survey does not evaluate
daylighting; however, an additional daylighting
question is described by Veitch (2010). The item
evaluating glare (“The light fixtures are too bright”)
does not explicitly address discomfort and excludes
daylighting. One question requires a comparative
quality judgment (i.e., “How does the lighting com-
pare to similar workplaces in other buildings?”),
which may produce unreliable responses. Some
scale properties such as internal reliability are not
reported, although many aspects of reliability and
validity are examined. Finally, the norms provided
are based on surveys of offices in the mid-1990s.
Given the changes in building design and lighting
technology in the intervening period, these norms
may no longer be a useful reference point.
2.1.2. Lighting conditions survey
Developed as part of the Daylight in Buildings task
for the IEA Solar Heating and Cooling Programme
(IEA 1999), the Lighting Conditions Survey con-
sists of 37 items that assess long-term lighting
satisfaction, quality, and other features of the
workplace (i.e., office layout, orientation) in
a field setting. It includes broad questions relating
to overall satisfaction, as well as questions about
individual aspects of lighting (e.g., “Does the light-
ing cause reflections in your work material?”).
 LEUKOS 117

Table 1. Characteristics of existing lighting quality surveys.a

Lighting characteristics

Artificial light

Task lighting
Natural light
Performance

Distribution
Reflections
Comfort

Brightness
Windows

Shading
Control
Flicker

Color
Glare

View
Number Scale
of Other Response Time properties Overall
Survey questions areas? type frame reported? satisfaction
Office Lighting 10 No Agree/disagree LT Selected x P x x x x x x
Survey (Eklund three-point
and Boyce 1996) rating scale
Lighting Conditions 37 Yes Mixed (rating LT No F x x x x x x x x x x
Survey (IEA 1999) scales, multiple
choice, free-
text)
National Research 10 No Five-point CC Yes x P, x x
Council (NRC) rating scale L, S
Canada Lighting
Quality Scale
(Veitch and
Newsham 2000)
IEA retrofit 36 Yes Mixed, (rating LT & No F x x x x x x x x x
monitoring user scale and free- CC
assessment text)
survey (IEA 2016)
Satisfaction with 18 Yes Seven-point CC Yes S x x x
Environmental rating scale
Features Veitch
et al. (2007)
a
P = presence, L = location, S = severity, LT = long-term, CC = current conditions.

2.1.2.1. Advantages. Selected items from this sur- and the presence of reflections). The phrasing of the
vey have been used or adapted in subsequent questions relates to lighting during the current day.
research (Aries et al. 2010; Velds 2002), providing
the potential for comparison across different set- 2.1.3.1. Advantages. The authors performed fac-
tings. The survey covers many different individual tor analysis to identify two subcomponents of their
lighting characteristics, including a particular scale, lighting quality and glare, each of which
focus on daylighting (see Table 1 for a summary consists of multiple questions that can be averaged
of individual lighting characteristics covered). to give an overall subscale score and has high
internal reliability (as demonstrated via
Cronbach’s alpha). Multiple characteristics of
2.1.2.2. Limitations. A small number of specific
glare are recorded in this survey.
lighting features are not addressed (e.g., flicker and
control), nor is an overall rating of visual comfort.
2.1.3.2. Limitations. This survey was designed for
Questions focus on identification of individual
use in a mock-up experimental setting. Therefore,
problematic lighting features, and questions
it measures only selected lighting features, uses
describing the environment and environmental
a day-long reference period, and does not include
aspects (e.g., “Do you ever notice cold draughts
any daylighting questions.
near the windows?”) are interspersed with judg-
ments of satisfaction and quality, without an
attempt to clearly distinguish how questions 2.1.4. IEA retrofit monitoring user assessment survey
group together or to measure defined constructs. The lighting monitoring protocol for retrofits
described by the Solar Heating and Cooling
Programme of the International Energy Agency
2.1.3. NRC Canada Lighting Quality Scale includes a “user assessment” portion where occupant
Veitch and Newsham (2000) developed this scale as ratings are captured (IEA 2016). A case report using
part of an office simulation experiment. It includes this tool has also been described (Gentile et al. 2016).
broad overall satisfaction questions, as well more It assesses individual features but does not include
specific assessment of individual features (i.e., visual a broad question assessing overall satisfaction with
performance, presence, location and severity of glare, lighting. Assessments are made about both current
118 A. C. ALLAN ET AL.
and long-term conditions, and it includes judgments
of satisfaction (e.g., “General satisfaction with the
following aspects of this room”) and quality (e.g.,
“Do you ever experience flickering from the electric
light sources in this room?”).
2.1.4.1. Advantages. This tool captures occupant
satisfaction and evaluations of a range of specific
lighting features (see Table 1 for a summary of
coverage), including daylighting. The inclusion of
questions relating both to current conditions and
to long-term experience provides multiple per-
spectives to the administrator.
2.1.4.2. Limitations. Some lighting features are
assessed only as a point-in-time rating. Although
the authors report using scale design methodology,
environmental feature questions are interspersed
with lighting questions, and it is not clear how
multiple items could be combined. As such, the
constructs measured are not clearly identified, nor
are psychometric properties reported.
2.2. Indoor environmental quality surveys
Broader surveys of indoor environmental quality gen-
erally include one or two questions addressing lighting
in addition to items regarding acoustics, noise, and
other characteristics of the indoor environment.
Indoor environmental quality scales are generally
reported as an overall environmental satisfaction or
utility score without an individually identifiable light-
ing section and therefore most are not included in this
review. However, the Satisfaction with Environmental
Features survey includes a substantial lighting assess-
ment component reported as a distinct construct and
is therefore included in Table 1.
2.2.1. Satisfaction with environmental features
This survey assesses occupants’ satisfaction with
three separate environmental features: privacy/
noise, ventilation/temperature, and lighting (Veitch
et al. 2007). Five of the 181 items relate to lighting,
specifically satisfaction with brightness, reflections,
glare, and view. Participants are asked to respond
based on conditions at the time of evaluation.
2.2.1.1. Advantages. The authors report on field
validation results and use an appropriate
psychometric development approach and describe
the rationale, constructs assessed, scale factor ana-
lysis, and relevant scale properties.
2.2.1.2. Limitations. Because it is part of a broader
questionnaire on environmental features, this survey
is brief and measures few aspects of lighting quality.
2.3. Subjective ratings of discomfort glare
All of the general lighting quality scales reviewed
included questions relating to glare, indicating its
importance for any occupant assessment of lighting.
Glare is one specific component of subjective light-
ing quality that has received considerable experi-
mental attention. Glare occurs when the range of
simultaneous luminances within the field of view
causes discomfort or loss of visual performance
(Boyce 2014). Discomfort glare causes annoyance
or discomfort to the viewer, and a large body of
experimental work has attempted to describe the
physiological mechanisms involved. Although it is
a subjective process, there has been substantial work
in the development of glare indices in an effort to
quantify the likelihood of glare occurring in a given
luminous scene. Glare indices such as the daylight
glare probability index (Wienold and Christoffersen
2006) assess the luminances in a visual field to esti-
mate the likelihood of glare occurring. However,
glare prediction indices do not entirely explain the
perception of glare, and their performance degrades
the further they are applied from highly controlled
settings (Jakubiec and Reinhart 2015; Van Den
Wymelenberg and Inanici 2016).
As Boyce (2014) comments, even with the best
characterization of the physiological processes asso-
ciated with glare “… there would still be the funda-
mental fact that discomfort glare involves
a psychological element in the form of the different
criteria used by individuals” (179). Glare indices
represent only a measure of glare potential rather
than being a direct measure of the experience of
glare itself, which requires assessment from
a viewer. Further, subjective observer ratings are
used to validate glare prediction indices and there-
fore directly inform their efficacy. As a result, it is
important to consider how subjective assessments
are obtained. The majority of studies assessing glare
are completed in highly controlled experimental

Table 2. Items used to assess glare severity in experimental settings.
Source Question details
De Boer (1967) Nine-point rating scale, cues at odd numbers: 1 =
unbearable, 3 = disturbing, 5 = just acceptable, 7 =
satisfactory, 9 = just noticeable
Osterhaus and Exact question wording not always reported
Bailey (1992) 4-point rating scale: imperceptible, noticeable, disturbing,
intolerable
Definitions quantifying the amount of time individuals
could work with glare present often provided
Iwata et al. (1992) Exact question wording not always reported
Four-point response scale: just perceptible, just acceptable,
just uncomfortable, just intolerable
Van Den Visual comfort measured using the question: “This is
Wymelenberg a visually comfortable environment for the intended work”
and Iancini (2014) Seven-point rating scale (strongly disagree to strongly agree)
settings, where specific lighting conditions or indivi-
dual fixtures are manipulated carefully in order to
establish photometric thresholds for glare response.
Table 2 describes individual items that have been
developed for this purpose. Although some are admi-
nistered alongside other questions relating to lighting
and glare, single items are used as the outcome vari-
able capturing glare severity. All of the items in Table
2 were developed for use in experimental settings to
gather point-in-time ratings of current conditions.
2.3.1. De Boer scale
One of the oldest and most widely used glare scales is
that developed by De Boer (1967). This question asks
participants to rate currently experienced glare on
a nine-point scale, with anchors at 1 = unbearable, 3
= disturbing, 5 = just acceptable, 7 = satisfactory, and
9 = just noticeable. The scale was originally designed
for the evaluation of visual perception in road traffic
in motorists.
2.3.1.1. Advantages. This scale has been used in
research assessing glare from indoor and outdoor
luminaire sources.
2.3.1.2. Limitations. The limitations associated
with the De Boer scale (and other glare scales)
are summarized thoroughly by Fotios (2015). Of
most concern is its failure to provide an option for
reporting the absence of glare—the response scale
presumes some level of glare. This is a major
drawback—asking participants to rate the extent
of the glare they are currently experiencing is
inappropriate if there is none present. In addition,
the scale has very ambiguous descriptive anchors,
LEUKOS 119
Examples of subsequent use
Bullough et al. (2008); Balk and Tyrrell (2011)
Christoffersen and Wienold (2004); Wienold and Christoffersen
(2006); Jakubiec and Reinhart (2015); Karlsen et al. (2015);
Rodriguez et al. (2015)
Kent, Altomonte et al. (2017); Tuaycharoen and Tregenza
(2007; with additional scale points); Velds (2002; with
behavioural definitions and a “no glare” response)
Van Den Wymelenberg and Inanici (2016); Mahić et al. (2017)
which may not be consistently interpreted by par-
ticipants. It was designed for the assessment of the
glare following direct exposure to luminaires and
therefore may not transfer well to capturing the
overall experience of glare in a field setting, parti-
cularly where daylight components are present.
2.3.2. Imperceptible–intolerable four-point scale
This four-point rating scale, as originally described by
Osterhaus and Bailey (1992), is frequently used to
capture subjective assessments of glare in experimen-
tal settings. The exact wording of the question varies
across studies; however, the verbal cues for the four-
point rating scale are generally imperceptible, notice-
able, disturbing, and intolerable. Many studies, but not
all, provide participants with an extended descriptive
definition for each scale point describing the length of
time that glare could be tolerated. Ratings on this four-
point scale are often dichotomized at the point
between imperceptible and noticeable or between
noticeable and disturbing (Rodriguez et al. 2017; Van
Den Wymelenberg 2014). An adaptation of this scale
has been used by Jakubiec and Reinhart (2015) to
assess the experience of glare over the long term (an
academic semester), asking participants to rate the
typical amount of discomfort glare they experienced
in the morning, midday, and afternoon, with addi-
tional questions to assess glare source and intervention
strategies. Some subsequent reports use this in con-
junction with questions assessing other aspects of glare
and lighting (Wienold and Christoffersen 2006).
2.3.2.1. Advantages. This rating scale has been
widely used, which facilitates the comparability and
statistical comparison and combination of studies
120 A. C. ALLAN ET AL.
conducted in different settings. Additionally, glare
indices used to estimate the glare potential of the
luminous environment (such as the daylight glare
probability index) have been validated against these
scale anchors in experimental settings. Therefore, if
a main aim is to compare and combine results with
other studies, this scale is desirable. The adaptation
of this question to a long-term rating of the environ-
ment (Jakubiec and Reinhart 2015) provides a useful
bridge between the measures used in experimental
glare research and field settings.
2.3.2.2. Limitations. The question used to elicit
participant responses is not always the same, which
may impede comparability between studies. The
phrasing often implies the presence of glare, and the
response scale points are ambiguous. The lowest rat-
ing (imperceptible glare) is not plain English and is an
oxymoron—by definition, glare is a perceptual process
that requires a sensation of annoyance or discomfort
to exist. This could be rectified by substituting imper-
ceptible with another scale anchor that clearly com-
municates that no glare is present. The use of
unnoticeable by Rodriguez et al. (2015) as the lowest
response scale point improves but does not entirely
remove this issue. The use of extended behaviorally
anchored definitions is likely to provide more reliable
judgments; however, including definitions may not be
feasible or desirable in a field setting.
2.3.3. Glare sensation vote
A glare sensation vote subjective rating (Hopkinson
1972; further adapted by Iwata et al. 1992) is similar to
the above imperceptible–intolerable scale (Osterhaus
and Bailey 1992), with slightly different scale presenta-
tion and labels. It has also been used with extended
behavioral definitions (e.g., Kent, Fotios et al. 2017).
2.3.3.1. Advantages. Versions of this scale have
been used in numerous studies, sometimes with
the addition of a no glare response and behavioral
definitions for each of the scale points, which
improves clarity for respondents.
2.3.3.2. Limitations. As with the previous scale,
there is not always an option to indicate that
there is no glare present. The question is not
always clearly described, and the semantic
descriptors of the response scale do not by them-
selves communicate a clear order of increasing
severity.
2.3.4. Visual comfort rating
Van Den Wymelenberg (2014) proposes that glare
may be better captured by phrasing questions not in
terms of discomfort but in terms of comfort. A series
of studies (Mahić et al. 2017; Van Den Wymelenberg
and Inanici 2014, 2016) have used a positively
worded question with a seven-point response scale
to assess glare. This question has been rated along-
side more general questions of lighting quality that
address visual comfort, visual appearance, bright-
ness, reflections, and distribution.
2.3.4.1. Advantages. This question is easy to
understand and does not presume the existence
of glare in the phrasing of the question. The verbal
anchors are clearly ordered, unambiguous, and do
not require extended behavioral definitions.
2.3.4.2. Limitations. The construct of visual com-
fort can be interpreted more broadly than the
absence of glare. This question is suitable for asses-
sing glare in experimental settings where the lighting
conditions are carefully managed and manipulated
(the setting this item was originally applied in).
However, visual comfort can be influenced by other
aspects of lighting (e.g., flicker), and in a field setting
this question may not be specific to glare.
2.4. Other subjective measures of lighting
2.4.1. Descriptive scales
Flynn et al. (1979) provide a series of semantic
differential scales to evaluate user impressions
resulting from a series of lighting scenarios.
Descriptive prompts such as “clarity,” “order,” “spa-
ciousness,” “relaxation,” and “privacy” are rated by
occupants to characterize their affective response to
the environment. These scales allow a more impres-
sionistic description of the lighting in a space in
a way that the previously discussed surveys do not.
They have been used to help understand how light-
ing design affects occupants (Durak et al. 2007;
Manav and Yener 1999). However, this descriptive
approach does not provide any indication of
whether these characteristics match the observer’s

aesthetic preferences, making them difficult to trans-
late into a measure of lighting quality or satisfaction.
2.4.2. Lighting preferences, beliefs, and behavioral
consequences
Finally, there are also studies of general lighting
preferences (Haans 2014; Veitch et al. 2010) or
beliefs (Veitch and Gifford 1996); however, we
believe that these are better conceptualized as indi-
vidual differences that predict subjective assessments
of lighting quality, rather than forming part of the
lighting quality assessment itself. Additionally, beha-
vioral consequences of poor lighting quality (e.g., use
of blinds) are a related yet conceptually distinct set of
behaviors. As a result, none of these have been
included in this review.
2.5. Gaps and recommendations for future
measure development
2.5.1. General lighting quality scales
The reviewed general lighting quality surveys dif-
fered considerably in terms of type and breadth of
content (i.e., lighting characteristics), the time period
they asked respondents to consider (i.e., current
conditions vs. long-term assessments), and the type
of judgment they asked people to make. Readers can
use Table 1 in combination with the in-text descrip-
tions to select or adapt a measure that may be useful
for their particular setting. We identified a number
of overall limitations with existing subjective assess-
ments of lighting and provide some suggestions for
the future development of measures.
2.5.2. Consensus on lighting characteristics
There is not currently agreement on which indivi-
dual lighting characteristics should be included in
a subjective assessment of lighting quality. Though
all addressed glare, questions regarding daylighting/
natural lighting, flicker, lighting control, and task
lighting were inconsistently included in measures
(see Table 1). None of the measures included items
relating to aesthetic judgments or satisfaction, nor
did any address variability of light across the day,
which may be relevant in heavily daylit spaces or as
a result of variable or smart lighting schemes.
The field would benefit from some consensus
on what specific lighting characteristics should be
included in subjective assessments of lighting
LEUKOS 121
environments. Future protocol documents could
recommend not only that subjective evaluations
should be gathered but also specify what lighting
characteristics should be assessed by occupants.
The lighting characteristics listed in Table 1
could serve as a starting point for this discussion.
We suggest that a broad coverage of characteristics
would be of most value and that questions relating
to daylighting are critical inclusions for field sur-
veys, given the heavy reliance on daylighting stra-
tegies in modern buildings.
2.5.3. Evaluation type
Judgments of lighting quality (rating from bad to
good), judgments of satisfaction (unsatisfactory to
satisfactory), or endorsement of a problem or issue
(e.g., “The lighting causes deep shadows,” as
included in the Office Lighting Survey) were all
used within surveys. Authors should think criti-
cally about the relevance of the type of judgments
they are asking occupants to make about the light-
ing and whether or not it is useful to combine
different types of evaluations. Depending on con-
text, satisfaction ratings may be more meaningful
than quality ratings, which rely on knowledge and
may not always be suitable for nonexpert occupant
respondents (Veitch et al. 2007).
2.5.4. Articulation of constructs and use of
psychometric scale development methods
There is a focus on asking respondents to rate or
identify problems with individual lighting charac-
teristics at the expense of broader evaluations (e.g.,
of satisfaction or comfort). It is not always clear
how items group together or how existing ques-
tionnaires could map onto broader components
and existing definitions of lighting quality (e.g.,
including visual comfort, performance, and aes-
thetics). Two of the measures reviewed reported
no psychometric properties of their scale (such as
internal reliability, factor structure, or correlations
with other related measures) to assess validity and
reliability, despite previous calls for authors to use
psychometric methods for measure development
(CIE 2014). In these measures, lighting appraisal
questions are presented intermixed with questions
assessing other aspects of the physical environ-
ment (e.g., type of office or proximity of windows).
Although this broader environmental information
122 A. C. ALLAN ET AL.
is critical to collect in a POE evaluation, these
questions do not themselves directly address light-
ing quality and should be clearly differentiated.
There are conventional statistical approaches for
the creation of self-report measures that are
described extensively in psychological literature
and have been widely adopted within other disci-
plines (see Cronbach and Meehl 1955; DeVellis
2012). Because subjective experiences are intangi-
ble and not directly observable, we rely on self-
report instruments to measure underlying con-
structs—in this case, lighting quality—and their
relationships. At a minimum, scale development
requires clear articulation of the theoretical con-
structs to be measured and a sound argument for
why items relate to that construct. Following this,
a comprehensive pool of items (larger than the
eventual intended length of the measure) is gen-
erally administered to an initial group and reduced
based on initial validation. Statistical psychometric
methods are used to refine the item pool. Factor
analysis can be used to identify and confirm the
underlying structure of a scale (and possible sub-
scales) that estimates levels of a more abstract
theoretical construct (e.g., satisfaction with light-
ing). Future measures should document the ratio-
nale and conceptual basis for their development
and overtly label the variable being measured by
each question or group of questions. Methods and
rationale for measure development should be pub-
lished with the results of psychometric analyses
such as internal consistency of the items in
a scale, factor analysis, or item response analyses.
Rather than being a perfunctory exercise, enga-
ging in formal statistical analysis of scale proper-
ties helps authors determine what construct is
being measured, reveal the structure of any sub-
components, explore whether questions group
together effectively, assess similarity to and differ-
ence from other related constructs and measures,
and identify whether there are particular items
that are not performing as intended. These ana-
lyses help the authors to demonstrate the quality
of their measures and identify relationships and
hierarchies between the individual items and
broader constructs and theoretical frameworks.
They can also be used to identify which questions
are most valuable to retain in shorter forms or in
broader surveys of indoor environmental quality.
2.6. Glare scales
Glare is a more specific construct representing just
one component of lighting quality. The single-item
questions shown in Table 2 have generally been
applied in more controlled research settings. As
such, there is more clarity regarding what is
being measured and how it is being measured.
Despite this, there remains a lack of agreement
on a universal rating scale (Fotios 2015; Van Den
Wymelenberg), which limits comparability
between experimental studies assessing glare.
Fotios (2015) notes that despite the creation of
glare indices based on the results of subjective
ratings, there is still considerable ambiguity in
the methods used to elicit those ratings.
It may be that measurement of behavioral or
physiological responses is a more useful indicator
of glare in some settings than stated preferences
(Fotios 2018). However, behaviors are not them-
selves preferences and can be influenced by other
factors, particularly in a field setting. For example,
the use of blinds may serve as a practical indicator
of glare presence in a field setting but is also
dictated by external factors such as view desirabil-
ity and social dynamics. Therefore, rating scales
will likely remain an important component of
glare evaluation.
2.6.1. Dichotomization of multipoint scales
Multipoint ratings of glare are often dichotomized
to create categories of comfort and discomfort;
however, there is disagreement on the placement
of this border, creating inconsistencies and lack of
comparability between studies (Van Den
Wymelenberg). The dichotomization of continu-
ous variables can be problematic for both concep-
tual and statistical reasons. Dichotomization
considerably reduces statistical power and
increases the likelihood of false-positive results
(i.e., detecting discomfort where none exists).
Dichotomized results do not accurately represent
the differences between groups, because they treat
respondents straddling the borderline as being
very different, when in fact they are very similar
(Altman 2006). However, the ongoing difficulty in
identifying a clean threshold that differentiates
luminous scenes that are comfortable from those
that are problematic suggests that imposing a cut-

point does not solve these issues and instead runs
the risk of creating spurious results. Analyzing
interval or multipoint data requires different sta-
tistical approaches but is very feasible, and we
recommend increased analysis of these scales in
their gathered format.
2.6.2. Question construction
Most existing glare questions do not provide
respondents with an option to indicate absence
of discomfort or glare, and some response scale
anchors (i.e., the descriptive words associated with
each scale point) are ambiguous and not clearly
ordered. Further, the exact wording of questions
used to elicit ratings is not always reported in full,
leaving room for procedural discrepancies and
reduced comparability between studies. A sizeable
body of literature has shown that the wording and
order of questions has a demonstrable impact on
participant responses, and question wording has
implications for readability, translations, and study
replication.
The exact wording of questions posed to parti-
cipants and the associated response scale anchors
should always be reported in full. Any changes to
the wording of previously published measures
should be explicitly described, and questions
should be phrased to avoid inferring the presence
of discomfort glare. Response options must
include a clear option for participants to indicate
that they are not experiencing discomfort or glare,
and response scale descriptors should have an
unambiguous order that indicates increasing or
decreasing severity.
2.6.3. Additional glare characteristics
Glare research has focused heavily on luminous
conditions that lead to the acute experience of
glare. When glare is assessed at a point in time
(i.e., observers are asked to rate current conditions)
in a highly controlled experimental setting, the
primary interest is the severity of glare, and the
questions in Table 2 address glare severity (or
simply glare presence when they are dichoto-
mized). However, in POE or field settings,
researchers, practitioners, and building managers
may be more interested in occupants’ experiences
of glare over the long term.
LEUKOS 123
Across this broader time frame, other glare char-
acteristics may be of interest, such as glare source,
which is often controlled in an experimental setting
(Christoffersen and Wienold 2004; Hirning et al.
2013), glare frequency (IEA 2016), glare duration,
and glare timing (Jakubiec and Reinhart 2015),
both across the day and across the year. The general
lighting quality surveys all include glare questions.
Some ask occupants to rate the frequency rather than
severity of glare; for example, “Does the artificial light
ever cause glare strong enough to bother you?” (IEA
1999) or, “In general, how often do you experience
glare from direct sunlight?” (IEA 2016). These addi-
tional glare characteristics have not been systemati-
cally described and are therefore not reviewed
separately. However, describing and defining these
additional glare characteristics and measuring them
in field settings will allow us to better determine their
importance and better predict the experience of
glare. We recommend that authors consider which
glare characteristic they are assessing and explicitly
state this when describing their measures.
2.7. Broad recommendations for lighting
research
2.7.1. Consistency across research areas
The field would benefit from greater consistency
in measures used to assess glare and lighting qual-
ity in general. Questions designed for administra-
tion in experimental contexts for moment-in-time
ratings may differ from those designed for long-
term assessments in field settings. However, there
could be greater consistency in the structure or
phrasing of questions to allow both types of mea-
surements to be meaningfully integrated and com-
pared. Additionally, the discomfort glare literature
often uses the terms visual discomfort and glare
interchangeably. Visual comfort can be influenced
by a range of lighting issues, including insufficient
light and flicker (Boyce 2014), and may represent
a slightly broader construct. Consideration of the-
oretical definitions and careful construction of
lighting quality scales could help to resolve this
hierarchy of constructs.
2.7.2. General advice for scale construction
We strongly recommend that scale development
texts are consulted in the development of future
124 A. C. ALLAN ET AL.
measures, because there is a wealth information
available regarding the construction of questions
for self-report scales (see DeVellis 2012; Kline
1986). In general, questions should avoid using
technical jargon and ambiguity, remain relatively
short, and ensure that they address only one aspect
or issue. Questions should avoid leading the respon-
dent to a particular conclusion (e.g., inferring the
presence of glare), and negatively worded questions
should be written with care, because they can be
confusing to readers. Response rating scales should
be labeled in a way that implies approximately equal
weighting across the scale, is unambiguous, and has
a clear increasing or decreasing order. There are
mixed views regarding the optimal number of scale
points in Likert-type responses; however, five- to
seven-point scales are generally desirable, because
higher numbers can become impractical and lower
numbers can provide inadequate variation.
Although single-item measures are desirable for
their brevity, asking multiple questions measuring
the same underlying variable reduces the likelihood
that the specific phrasing of the question results in
misinterpretation by the respondent. Further, multi-
item measures allow researchers to assess the inter-
nal consistency of responses using Cronbach’s alpha.
Single items that effectively measure a construct can
emerge from and be validated against longer mea-
sures, although even carefully designed single-item
responses are typically less stable than those
obtained from multi-item measures (DeVellis 2012).
In experimental settings where it is practical to
obtain only one or a handful of ratings per condi-
tion, identical conditions can be repeated and
responses averaged to give a more stable subjective
estimate of the measured variable. However, even
well-designed subjective scales cannot rectify over-
sights in experimental design that allow for the
influence of confounding variables, order effects,
and anchoring biases (see Kent, Fotios et al. 2017;
Kent et al. 2018). These effects can be avoided
through careful experimental design (e.g., via ran-
domization or counterbalancing of conditions)
and are not specific to rating scales—they can
also affect behavioral indicators or physiological
responses.
Finally, though questions that name a specific
task (e.g., computer work) or setting (e.g., offices)
increase the tangibility of questions to participants,
they have narrow applicability. Increased transfer-
ability to a wider variety of settings could be
achieved by writing questions that are task and
setting agnostic (i.e., use of language such as “this
space” or “usual tasks or activities”) or by phrasing
questions such that the named task or setting can
be easily substituted without changing question
structure. More broadly applicable questions may
encourage increased examination of lighting qual-
ity outside of an office building context.
3. Conclusions
Subjective evaluations of lighting are an important
complement to photometric measures when deter-
mining lighting quality. Existing protocols acknowl-
edge the role of subjective measures of lighting, and
this review reveals that although a number of such
measures exist, there is room for further develop-
ment of reliable and valid tools to assess lighting
quality and its components. Therefore, development
of future measures should be based on clear defini-
tions of lighting quality and its component parts,
follow established methods for scale development,
and explicitly report the psychometric scale.
The lack of consensus on scales assessing lighting
quality greatly limits the comparability of results and
the pooling of data from multiple studies to compare
and contrast lighting quality in different settings.
Greater consensus and consistency in measures
would enable broader examination of how different
environmental, architectural, and lighting features
influence occupants’ judgments of lighting quality
and satisfaction. Further, consistent implementation
of standard measures would allow establishment of
norms as originally undertaken with the Office
Lighting Survey (Eklund and Boyce 1996). This
would dramatically improve the utility of such scales
for practitioners, who could compare results from
individual postoccupancy evaluations to existing pub-
lished norms and report benchmarks to clients.
Gathering subjective assessments is inherently
less precise than capturing physical measurements
but is no less important. The careful design of
subjective measures can greatly reduce measure-
ment uncertainty. By developing reliable and valid
tools, we can improve our understanding of light-
ing quality and establish design solutions that lead
to more satisfied building occupants.

Disclosure statement
Gillian Isoardi is a consultant for Light Naturally, who have
provided support to an associated project. Light Naturally has
no business or financial benefits directly relating to the contents
of this review. The other authors have no interests to declare.
Funding
This work was supported by Australian Research Council
Linkage grant “Designing Healthy and Efficient Lighting
Environments in Green Buildings”(LP150100179) in partner-
ship with AECOM and Light Naturally.
References
[CIE] Commission Internationale de l’Eclairage. 2014. Guide
to protocols for describing lighting. Vienna (Austria): CIE.
Publication No: CIE 213:2014.
Altman DG. 2006. The cost of dichotomising continuous
variables. BMJ. 332:1080–1080. doi:10.1136/bmj.332.7549.
1080
Aries MBC, Veitch JA, Newsham GR. 2010. Windows, view,
and office characteristics predict physical and psychologi-
cal discomfort. J Environ Psychol. 30:533–41. doi:10.1016/
j.jenvp.2009.12.004
Balk S, Tyrrell R. 2011. The Accuracy of Drivers’ Judgments
of the Effects of Headlight Glare: Are We Really Blinded
by the Light?. In: Proceedings of the Sixth International
Driving Symposium on Human Factors in Driver
Assessment, Training and Vehicle Design, June 27-30,
Olympic Valley — Lake Tahoe, California. Iowa City, IA:
Public Policy Center, University of Iowa, 2011:510–517.
Borisuit A, Linhart F, Scartezzini J-L, Munch M. 2014. Effects
of realistic office daylighting and electric lighting condi-
tions on visual comfort, alertness and mood. Light Res.
1–18. doi:10.1177/1477153514531518
Boyce P, Veitch JA, Newsham GR, Jones CC, Heerwagen J,
Myer M, Hunter CM. 2006. Lighting quality and office work:
two field simulation experiments. Light Res Technol.
38:191–223. doi:10.1191/1365782806lrt161oa
Boyce PR. 2014. Human factors in lighting. 2nd ed. Boca
Raton (FL): CRC Press.
Bullough JD, Brons JA, Qi R, Rea MS. 2008. Predicting
discomfort glare from outdoor lighting installations.
Light Res Technol. 40:225–42.
Christoffersen J, Wienold J 2004. ECCO-build Report ECCO-
DBUR-0303-01 Monitoring Procedure for assessment of
user reaction to glare. October:1–70.
Cronbach LJ, Meehl PE. 1955. Construct validity in psycho-
logical tests. Psychol Bull. 52:281–302.
De Boer JB. 1967. Visual perception in road traffic and the
field of vision of the motorist. In Public Lighting. Edited
by JB de Boer. Eindhoven (The Netherlands): Philips
Technical Library.
LEUKOS 125
DeVellis R. 2012. Scale development: theory and applications.
3rd ed. Thousand Oaks (CA): SAGE.
Durak A, Camgöz Olguntürk N, Yener C, Güvenç D, Gürçinar Y.
2007. Impact of lighting arrangements and illuminances on
different impressions of a room. Build Environ. 42:3476–-
82. doi:10.1016/j.buildenv.2006.10.048
Eklund NH, Boyce PR 1996. The development of a reliable, valid,
and simple office lighting survey. J. Illum. Eng. Soc. 25:25
Flynn J, Hendrick C, Spencer TOM. 1979. A guide to the
methodology procedures for measuring subjective impres-
sions in lighting. Journal of the Illuminating Engineering
Society 1979;8:95-110.). J Illum Eng Soc. 8:95–110.
Fotios S. 2015. Research Note : uncertainty in subjective eva-
luation of discomfort glare. Light Res Technol. 47:379–83.
doi:10.1177/1477153515574985
Fotios S. 2018. Correspondence: New methods for the eva-
luation of discomfort glare. Light Res Technol. 50:489–491.
doi:10.1177/1477153518773577
Gentile N, Dubois M, Osterhaus W, Stoffer S, Naves C,
Amorim D, Geisler-Moroder D, Jakobiak R. 2016.
A toolbox to evaluate non-residential lighting and day-
lighting retrofit in practice. Energy Build. 123:151–61.
doi:10.1016/j.enbuild.2016.04.026
Haans A. 2014. The natural preference in people’s appraisal
of light. J Environ Psychol. 39:51–61. doi:10.1016/j.
jenvp.2014.04.001
Hirning MB, Isoardi GL, Coyne S, Garcia Hansen VR,
Cowling I. 2013. Post occupancy evaluations relating to dis-
comfort glare: A study of green buildings in Brisbane. Build
Environ. 59:349–57. doi:10.1016/j.buildenv.2012.08.032
Hopkinson RG. 1972. Glare from daylighting in buildings.
Appl Ergon. 3:206–15.
Humphreys MA, Nicol JF. 2007. Self-Assessed Productivity
and the Office Environment. ASHRAE Trans. 113:606–16.
International Energy Agency. 1999. Post occupancy evalua-
tion of daylight in buildings. Galve (Sweden): Centre for
Built Environment.
International Energy Agency. 2016. Monitoring protocol for
lighting and daylighting retrofits. Stuttgart (Germany):
Fraunhofer-Institut für Bauphysik.
Iwata T, Shukuya M, Somekawa N, Kimura K. 1992.
Experimental study on discomfort glare caused by
windows. J Archit Plan Environ Eng. 432:21–33.
Jakubiec JA, Reinhart CF. 2015. A concept for predicting
occupants’ long-term visual comfort within daylit spaces.
Leukos. 2724:1–18. doi:10.1080/15502724.2015.1090880
Karlsen L, Heiselberg P, Bryn I, Johra H. 2015. Verification of
simple illuminance based measures for indication of dis-
comfort glare from windows. Build Environ. 92:615–26.
doi:10.1016/j.buildenv.2015.05.040
Kent MG, Altomonte S, Wilson R, Tregenza PR. 2017.
Temporal effects on glare response from daylight. Build
Environ. 113:49–64. doi:10.1016/j.buildenv.2016.09.002
Kent MG, Fotios S, Altomonte S. 2017. Discomfort glare
evaluation: the influence of anchor bias in luminance
adjustments. Light Res Technol. 1–16. doi:10.1177/
1477153517734280
126 A. C. ALLAN ET AL.
Kent MG, Fotios S, Altomonte S. 2018. Order effects when
using Hopkinson’s multiple criterion scale of discomfort
due to glare. Build Environ. 136:54–61. doi:10.1016/j.
buildenv.2018.03.022
Kline P. 1986. A handbook of test construction:
Introduction to psychometric design. New York (NY):
Methuen & Co.
Mahić A, Galicinao K, Van Den Wymelenberg K. 2017. A pilot
daylighting field study: testing the usefulness of
laboratory-derived luminance-based metrics for building
design and control. Build Environ. 113:78–91. doi:10.1016/
j.buildenv.2016.11.024
Manav B, Yener C. 1999. Effects of different lighting arrange-
ments on space perception. Archit Sci Rev. 42:43–47.
doi:10.1080/00038628.1999.9696847
Newsham G, Brand J, Donelly C, Veitch JA, Aries M,
Charles K. 2009. Linking indoor environment conditions
to job satisfaction: a field study. Build Res Inf.
37:129–47. doi:10.1080/09613210802710298
Osterhaus W, Bailey IL 1992. Large area glare sources and their
effect on discomfort and visual performance at computer work
stations. In: Proceedings of the 1992 IEEE Industry
Applications Society Annual Meeting, 4-9 October. Houston,
Texas.
Rodriguez RG, Garretón JAY, Pattini AE. 2015. Glare and cogni-
tive performance in screen work in the presence of sunlight.
Light Res Technol. 1477153515577851. doi:10.1177/
1477153515577851
Rodriguez RG, Yamín Garretón JA, Pattini AE. 2017. An
epidemiological approach to daylight discomfort glare.
Build Environ. 113:39–48. doi:10.1016/j.buildenv.
2016.09.028
Singh A, Syal M, Grady SC, Korkmaz S. 2010. Effects of green
buildings on employee health and productivity. Am
J Public Health. 100:1665–68 [accessed 2018 Jan 19].
http://www.ncbi.nlm.nih.gov/pubmed/20634460.
Tralau B, Dehoff P, Schierz C. 2009. Introduction of the ergo-
nomic Lighting Indicator as an objective measure for lighting
quality. In: Experiencing Light 2009 - International Conference
on the Effects of Light on Wellbeing. Eindhoven, The
Netherlands.
Tuaycharoen N, Tregenza PR. 2007. View and discomfort
glare from windows. Light Res Technol. 39:185–98.
doi:10.1177/1365782807077193
Van Den Wymelenberg K. 2014. Visual comfort, discomfort
glare, and occupant fenestration control: Developing a
research agenda. Leukos. 10:207–21. doi:10.1080/
15502724.2014.939004
Van Den Wymelenberg K, Inanici M. 2014. A critical investiga-
tion of common lighting design metrics for predicting human
visual comfort in offices with daylight. Leukos. 10:145–64.
doi:10.1080/15502724.2014.881720
Van Den Wymelenberg K, Inanici M. 2016. Evaluating a new
suite of luminance-based design metrics for predicting
human visual comfort in offices with daylight. Leukos.
12:113–38. doi:10.1080/15502724.2015.1062392
Veitch JA, Charles KE, Farley KMJ, Newsham GR. 2007.
A model of satisfaction with open-plan office conditions:
COPE field findings. J Environ Psychol. 27:177–89.
doi:10.1016/j.jenvp.2007.04.002
Veitch JA, Gifford R. 1996. Assessing beliefs about lighting
effects on health, performance, mood and social behaviour.
Environ Behav. 28:446–70.
Veitch JA, Newsham GR 1996. Determinants of lighting
quality II: research and Recommendations. In: Annual
Meeting of the American Psychological Association.
Toronto, Canada. p. 58. doi:10.1006/jevp.1999.0169
Veitch JA, Newsham GR. 2000. Exercised control, lighting
choices, and energy use: an office simulation experiment.
J Environ Psychol. 20:219–37.
Veitch JA, Newsham GR, Mancini S, Arsenault CD. 2010.
Lighting and office renovation effects on employee and orga-
nizational well-
being. Ottowa (Canada): National Research Council of
Canada. doi:10.4224/2037/4532
Velds M. 2002. User acceptance studies to evaluate discom-
fort glare in daylit rooms. Sol Energy. 73:95–-
103. doi:10.1016/S0038-092X(02)00037-3
Wienold J, Christoffersen J. 2006. Evaluation methods and
development of a new glare prediction model for daylight
environments with the use of CCD cameras. Energy Build.
38:743–57. doi:10.1016/j.enbuild.2006.03.017