Professional Documents
Culture Documents
• Medical
Medical
Noti«
Medic.me i:s a n e:ver-chang:i ng science. As new -research and clinical exper'1e nce broaden o ur k nowledge, changes in treatmen t a nd
drug th erapy are requ ited. 'lhe ed ito rs an d th e pub lish er o f this \vork h ave ch ecked with sources be lieved to be re.liable in the ir
efforts to provid e: info rmatio n tha t is complete an d generally in accord w ith the: s tan dards accepted at the time of pub1'1catio n.
or
l lowever, ·m ,t,ew o f the posl':1bili1y huma n error o r changes in med ical sciences, n e:ith er1he editors, no r the publish er, no r
a ny othe r pa rty w ho has been involved in the prepa ra tio n o r publicatio n o f this worl<, warra nt th at the in format io n contained
h e.re.in is in every respect accu rnte: o r comp le te:. Readers are en couraged to confirm the info rmatio n con ta·med h ere:i n w ith o the r
sources. For example .1nd in pa rticular, read ers are .1dvised. to ch e:cl< the p ro duct ·info rma tion sh eet 'mduded ·m th e package o r
each drug they p la n to adm'mister to be certa in th at th e: ·mfu rm.if1o n contained in th is book is accurate a nd tha t ch anges h ave
no t bee n made in th e: recommended d ose o r in the: con trai r.id icatio ns fo r adm in·1s 1ra(1o n. Th is recomme nda(1o n is o f particu lar
·m,po rta nce in connecf1o n with new o r in frequently used drugs.
IWe'fy e ffort has been made to trace an d acknowledge: copy r·1g h ted materia l. ·11,e autho rs an d publish er len d er their apolo~f1es
.sh o u ld a ny in frlngemenl h ave occurred.
En quiries s ho u ld be made to th e publish er via ,,..· ww.mcgraw-hill.com.a u o r ma rked for the attentio n o fthe per missio ns edito r
at th e address below.
Preface XI
Introduction Xlll
Formative evaluation 38
3.1 Formative evaluation: testing methods and materials befo.re
starting a project or program of work 38
3.2 Formative evaluation for different types of health promotion
intervention 47
3.3 Summary 48
References 48
Process evaluation 51
4.1 Assessing the.implementation of health promotion
projects and programs 51
4.2 Methods for conducting process evaluation 54
4.3 Summary 61
References 61
..
Evaluation methods for program replication,
dissemination and institutionalisation 104
7.1 Stages in assessing the sjgnificance of programs 105
7.2 Replication (intervention demonstration) 107
7.3 Dissemination 111
7 .4 Institutionalisation 114
7.5 Summa1y 116
References 11 7
Appendices 130
Glossaiy 141
Index 149
Chapter 1
Table 1.1 Hypothetical logic model: planning a health
promotion program to improve healthy food
in school canteens 15
Chapter 2
Table 2.1 The differences and similarities between
practitioner and scientific perspectives of health
promotion programs 21
Table 2.2 Case studies of research illustrating the stages in
the model using different health promotion
interventions 30
Chapter 3
Table 3.1 Examples of the uses of formative evaluation in health
promotion 43
Chapter4
Table 4.1 Practical tasks in carrying out process evaluation tasks 55
Table4.2 Examples of process evaluation (PE) 57
Chapter 5
Table 5.1 Examples of research design 72
Table 5.2 Sampling methods used in the evaluation of health
promotion interventions 76
Table 5.3 Measurement at ruffe1·ent stages of program evaluation 85
LI ST OF TABLE S A ND FIGURE S Vii
Chapter 6
Table 6.1 Published e."<amples of CPEs showing 'complexity' in
formative, process and in1pact evaluation 100
Chapter?
Table 7.1 Replication and d issemination examples 110
Chapters
Table 8.1 Examples of systematic reviews of health
promotion evidence 126
Appendices
Table A2.1 Examples of statistical techniques and analysis in a
hypothetical (and effective) weight-loss intervention 131
Table A3.1 Some examples of techniques and methods used to
assess measure:me:nt reliability and validity 133
Chapter 1
Figure 1.1 The planning and evaluation cycle 2
Figure 1.2 Health promotion actions and outcomes 8
Figure 1.3 Theoretical distrib ution over time of outcomes
from health promotion inte1ventioos 12
Chapter 2
Figure 2.1 Building evidence: for public health programs:
stages of research and evaluation 24
Chapter 3
Figure 3.1 Stages showing formative, process and impact/
outcome evaluations 39
Viii LI ST OF TABLE S AND FIGURES
Chapter4
Figure 4.1 Advanced statistical modelling in process evaluation:
understanding bow interventions work through
mediator and moderator analyses 60
Chapter 5
Figure 5.1 The evaluation process 66
Figure 5.2 Quantitative evaluati on designs for individual
progran1s (stage 3 of Figure 2.1), ranked from
'most scientific' experimental designs to less scientific
'pre-experimental' designs 74
Figure 5.3 Reliability (reproducibility) and responsiveness of
measurement 83
Chapter 6
Figure 6.1 Complex interventions: planning and
evaluation stages 92
Chapter 7
Figure 7.1 Stages in scaling up interventions to achieve
population health 106
Figure 7.2 Conceptual framework for research replication and
dissemination 107
Chapter B
Figure 8.1 Variation in the use of evidence in health promotion:
planned, responsive and reactive practice 120
Adria n Bauman is Sesquicentenary Professor of Public Health and Health
Promotion at the University of Sydney, Australia. He directs the WHO
Collaborating Centre on Physical Activity, Nutrition and Obesity, which
has a major interest in chronic disease prevention. He has taught health
promotion research methods and program evaluation to public health
students for 30 years, and has applied these principles in his population
health research program. His recen t research interests are in research
translation and in evaluating scaled-up health promotion inteIVentions at
the population level.
Acknowledgments
Several people have contributed to the development of Evaluation in a
Nutshell. We would like to acknowledge the contribution of Bill Bellew and
Lesley King for their constructive comments and advice on early drafts of this
edition of the book.
his book is for students of health promotion and for health promotion
T practitioners. Evaluation in a Nutshell is intended to equip the reader
with the ability to understand, interpret and assess the quality of published
research, and to excite interest in evaluation and promote further study
that will lead to the development of core skills in evaluation. It provides
foundation knowledge and recommended further reading.
In writing this book, we have drawn on many years of expe1ience in
conducting evaluations of health promotion programs, working with health
promotion practitioners and teaching public health students. From this
experience we recognise the need for students and practitioners to understand
the basic principles of evaluation and the application of these principles in
evaluation design, whether for the: purpose of conducting an evaluation or for
assessing the work of others.
Any review of published research will reveal that not all health promotion
programs are equally successful in achieving their goals and objectives.
Experience tells us that programs are most likely to be successful when the
determinants of a health problem are well understood; when the needs and
motivations of the target population are addressed; and when the context in
which the program is being implemented is taken into account. That is, the
intervention 'fits' the problem.
Similarly, in developing an evaluation design for a health promotion
program, the evaluation design needs to fit the circumstances of the program.
Programs can be evaluated in a ran ge of ways, may demand differing levels of
resources and may use various evaluation designs. This book illustrates how
evaluation questions change as a program evolves, and shows how programs that
are truly innovative need dose scrutiny and highly structured and comprehensive
evaluation. On the other hand, programs that have previously been shown to
work in a variety of circumstances, and are Low-cost and Low-risk, v.rill require
more modest monitoring for the purposes of accountability and quality
control. The evaluation of projects ""ith tightly defined objectives in a controlled
environment will be different to that of multi-component, long-term programs.
Evaluations must be tailored to suit the activity and circumstances of
individual programs: no single method or design will be 'right' for all
programs.
Xii PREFACE
This chapter introduces the purposes and functions of evaluation and describes the
stages in planning and implementing a health promotion program or intervention.
These range from defining the problem and developing solutions to planning and
implementing programs. Each stage is linked to the evaluation methods described in
this book.
~ 2
Solution
generation
6
Intermediate
3 outcome
Resource
5
Impact
4
Theory indicates
how to achieve
organisation change
measurements
and raise community
for use in
awareness Theory provides a benchmark
eva luation
against which actual can be
com pa red with ideal program
others, and knowledge of the local context in which the program will be
implemented.
The development of program strategies that incorporate the right
combination of actions in the right sequence completes the planning stage of
the program and should link clearly and logically to the project or program
outcomes that were identified in the first stage of planning. Some modification
of the objectives may be required at this stage, together with refinements
to the measures used to evaluate the short-term impact of a program. The
measurement of impact and outcome is addressed in more detail in Chapter 5.
It is essential to define clearly the planned optimal structure and
sequencing of the intervention in the second step of the planning process.
This definition provides a point of reference for evaluating the process of
implementation that has led to progress (or lack of progress) in achieving
program goals and objectives. This is desuibed in more detail in Chapter 3
and is a compulsory element of any program evaluation. This process
provides key information that will be very useful to practitioners in refining
and reproducing future interventions.
rn
<
Health promotion Health promotion lnte rmed iate Social health )>
r
actions outcomes health outcomes outcomes c
)>
(outcomes of the process (program impact, or (long·term outcomes) ....
of intervention) short-term outcomes) 0
z
z
Education Health literacy Healthy lifestyles )>
Health outcomes
Healthy environments
Ad..acacy Healthy public policy and Measures include
Measures include
Examples include organisational practices reduced morbidity,
safe physical environment,
lobbying, poli1ical Measures include policy reduced disability and
supponive economic and
orga ni sat ion and statements, legislation, avoidable mortali1y
soda I conditions, good
ac1ivism, and regulation and re source
allo cation organisational food supply and res1ri cted
overcoming
access to 1obacco/ alcohol
bureaucratic inertia practices
At the previous stage, health promoticm outcomes (shown in Figure 1.2) refer
to modifiable personal, social and environmental factors that lead to the
detem1inants of health (intermediate health outcomes). These may represent
the immediate results of planned health promotion activities. Thus, they
include measures of the cognitive and social skills that determine the ability
of individuals to gain access to, understand and use health information
(health literacy in the model). Examples of health promotion outcomes
would include improved health knowledge and an understanding of where
to go and what to do to gain access to health seivices and other support.
Social action and influence includes organised effo1ts to promote or enhance
the actions and control of social groups over the detemunants of health.
This includes the mobilisation of human and material resources in social
action to overcome suuctural barriers to health, enhance social support and
reinforce social norms conducive to health. Examples of outcomes range from
improved social 'connectedness' a nd social support to improved community
empowem1ent.
Healthy environments are largely detem1ined by healthy public policy and
organisational practices. Policy-determined legislation, funding, regulations
and incentives significantly influence organisational practices. Thus, e,'(amples
of health promotion outcomes here might be changes to health and social
policies that lead to improvements in services, social benefits, the built
environment and housing. These would influence organisational practices
and redirect resource allocation to suppo1t environments conducive to health.
Using the HIV-prevention project as an e,'(ample, health promotion
outcomes could include:
• in1proving health literacy by raising levels of accurate knowledge
about HIV prevention and improving skills and confidence
to put into practice recommended behaviours in challenging
circumstances among the target population;
• facilitating a more supportive environment through social action,
to improve commitment to safe sex practices in commercial sex
establishments, and to support access to condoms and other
preventive behaviours;
• improving health policy and services by providing more facilities
for, and better access to, HIV testing and regular health checks
among the target population.
Figure 1.2 also indicates three health promotion actions- what to do, as
distinct from what outcomes are achieved. Education consists primarily of
the creation of opportunities for learning that are intended to raise levels of
personal health literacy and thereby increase the capacity of individuals and
communities to act to improve and protect their health. Social mobilisation
10 EVA L UA TIO N I N A N U TS H E LL
1.6 Evaluation
Different types of evaluation tasks occur at each stage of the cycle shown in
Figure 1.1. These start with formative evaluation, a set of activities designed to
develop and pre-test program materials and methods (see Chapter 3); and
process evaluation, a set of activities directed towards assessing progress in
program implementation (see Chapter 4).
Health promotion interventions have different types of impact and
different outcomes over time. Change in the different types of outcome will
occur according to different timescales, depending on the nature of the
intervention and the type of social or health problen1 being addressed. As a
consequence, different evaluation methods are used to measure impact and
outcome at different stages in the life of a program.
Impa ct evaluation measures t he short-term effects that are predicted and
defined during the planning stage of the program. These impact measures
are termed health promotion outcomes in Figure 1.2. These health promotion
outcomes are intended to lead to subsequent change in intermediate health
outcomes, for example, in behaviours and environments, in ways that will
ultimately improve health and social outcomes in the model. Outcome
assessment involves the measurement of individual behaviour or changes
to the social, economic and en vironmental conditions that determine
health. Figure 1.2 provides e,'Camples of these intermediate outcomes. The
assessment of social and health outcomes is the end-point assessment of
health promotion.
Figure 1.3 illustrates how a comprehensive set of health promotion
interventions might produce different outcomes over time. This model was
originally developed to illustrate likely progress over a five-year period in a
comprehensive, community-base<l heart disease- prevention program. Thus
the 'units of time' are years and the model shows how, after one year, the
program impact could be measured mainly in terms of increased community
awareness and participation in intervention components. By the third year,
good progress in achieving short-term program impacts (health promotion
outcomes) should be measurable, alongside peak-level community awareness
and participation. Early progress in achieving intem1ediate outcomes (such as
behavioural risks) should also be measurable. After five years, major progress
in these intem1ediate outcomes should be obseivable and early impact
Another random document with
no related content on Scribd:
THE COUNSELLOR’S CUP.
Rub a quarter of a pound of sugar upon the rinds of two fine China
oranges, put it into an enamelled stewpan, and pour on it a pint of
water; let these boil gently for two or three minutes, then pour in half
a pint of China orange-juice mixed with that of one lemon, and
previously strained through muslin; the moment this begins to boil,
pour it into a hot jug, and stir to it half a pint of the best Cognac
brandy. Serve it immediately. When preferred cold, prepare the syrup
with the juice of the fruit, cover it down in the jug, set it into ice, or
into a very cool place, and add the spirit only just before the cup is
wanted for table. Should the fruit be very acid, increase the
proportion of sugar. A few slight strips of the rind of a Seville orange
cut very thin, would to many tastes be an agreeable addition to the
beverage; which should be made always with fresh sound fruit.
Sugar, 4 oz. (6 if needed); rasped rinds of China oranges, 2; water,
1 pint: 3 minutes. Strained juice of China oranges mixed with that of
1 large lemon, 1/2 pint; best Cognac brandy, 1/2 pint.
Obs.—For a large cup these proportions must be doubled. Sherry
or Madeira substituted for the brandy, will make a pleasant cool cup
of this kind; and equal parts of well made lemonade, and of any good
light white wine, thoroughly cooled down, will give another agreeable
beverage for warm weather; but a much smaller proportion of wine
would better adapt it to many tastes.
CHAPTER XXX.
Persons who drink coffee habitually, and who are very particular
about its flavour and quality, should purchase the best kind in a raw
state, keep it for two or three years if they are not certain that it has
been so long harvested—as when new it is greatly inferior to that
which has been kept—and have it roasted at home. This can be
cheaply done in small quantities by means of the inexpensive
apparatus shown above; the cost of it not exceeding seven or eight
shillings, and the supply of charcoal needed for it being very trifling
indeed; or, with that inserted below, which is larger and about double
the price. The cylinder which contains the coffee should be only half
filled, and it should be turned rather slowly over the fire, which
should never be fierce, until a strong aromatic smell is emitted; the
movement should then be quickened, as the grain is in that case
quite heated, and it will become too highly coloured before it is
roasted through, if slowly finished. When it is of a fine, light, equal
brown, which must be ascertained, until some little experience has
been acquired, by sliding back the door of the cylinder, and looking
at it occasionally towards the end of the process, spread it quickly
upon a large dish, and throw a thickly folded cloth over it. Let it
remain thus until it is quite cold; then put it immediately into canisters
or bottles, and exclude the air carefully from it.
To boil coffee and refine it, put the necessary quantity of water into
a pot which it will not fill by some inches; when it boils stir in the
coffee; for unless this be at once moistened, it will remain on the top
and be liable to fly over. Give it one or two strong boils, then raise it
from the fire, and simmer it for ten minutes only; pour out a large
cupful twice, hold it high over the coffee pot and pour it in again, then
set it on the stove where it will keep hot without simmering or moving
in the least, for ten minutes longer. It will be perfectly clear, unless
mismanaged, without any other fining. Should more, however, be
deemed necessary, a very small pinch of isinglass, or a clean egg-
shell, with a little of the white adhering to it, is the best that can be
used. Never use mustard to fine coffee with. It is a barbarous custom
of which we have heard foreigners who have been in England
vehemently complain.
Coffee, 2 oz.; water, 1 pint to 1 quart, according to the strength
required. Boiled 10 minutes; left to clear 10 minutes.
Remark.—Filtering is, we should say, a far more economical, and
in every way a superior mode of making coffee to boiling it; but as
some persons still prefer the old method, we insert the receipt for it.
CAFÉ NOIR.
An ounce of chocolate, if
good, will be sufficient for one
person. Rasp, and then boil it
from five to ten minutes with
about four tablespoonsful of
water; when it is extremely
smooth add nearly a pint of new
milk, give it another boil, stir it
well, or mill it, and serve it
directly. For water-chocolate use
three-quarters of a pint of water
instead of the milk, and send
rich hot cream to table with it.
The taste must decide whether it
shall be made thicker or thinner.
Chocolate, 2 oz.; water,
quarter-pint, or rather more; milk, 1-3/4: 1/2 minute.
Obs.—The general reader will understand the use of the
chocolate-mill shown in the engraving with the pot; but to the
uninitiated it may be as well to observe, that it is worked quickly
round between both hands to give a fine froth to the chocolate. It
also serves in lieu of a whisk for working creams, or jellies, to a froth
or whip.
A SPANISH RECIPE FOR MAKING AND SERVING CHOCOLATE.
Take of the best chocolate an ounce for each person, and half a
pint of cold water; rasp or break it small in a mortar, set it over a slow
fire, and stir or mill it gently until it has become quite smooth like
custard; pour it immediately into deep cups, and serve it with a glass
of sugar and water, or with iced water only[185] to each cup; and
with plates of very delicate dried toast cut in narrow strips, or with the
cakes called “ladies’ fingers.” Should the chocolate appear too thick,
a little water must be added. Milk is sometimes substituted for it
altogether.
185. Sometimes with a water ice, which should be of an appropriate character.
TO MAKE COCOA.
Directions for making it are usually sold with the prepared, or best
quality of cocoa, which is merely mixed with boiling water in the
proportions indicated on the packets. That which is prepared from
the nibs requires several hours’ boiling, and should be left until it is
quite cold, that the oil which will be found on the surface may be
cleared from it before it is again heated for table: this is particularly
needful when it is to be served to persons in delicate health.
CHAPTER XXXI.
Bread.
REMARKS ON HOME-MADE BREAD.
It is surely a singular fact that the one article of our daily food on
which health depends more than on any other, is precisely that which
is obtained in England with the greatest difficulty—good, light, and
pure bread—yet nothing can be more simple and easy than the
process of making it, either in large quantities or in small. From
constant failure, it is nevertheless considered so difficult in many
families, that recourse is had to the nearest baker, both in town and
country, as a means of escape from the heavy, or bitter, or ill-baked
masses of dough which appear at table under the name of
household or home-made bread; and which are well calculated to
create the distaste which they often excite for everything which bears
its name. Without wishing in the slightest degree to disparage the
skill and labour of bread-makers by trade, truth compels us to assert
our conviction of the superior wholesomeness of bread made in our
own homes. When a miller can be depended on to supply flour of
good quality, and the other ingredients used in preparing it are also
fresh and good, and mingled with it in due proportions, and the
kneading, fermentation, and baking, are conducted with care and
intelligence, the result will uniformly be excellent bread. Every cook,
therefore,—and we might almost say every female servant—ought to
be perfectly acquainted with the proper mode of making it; and skill
in preparing a variety of dishes, is poor compensation for ignorance
on this one essential point.[186] Moreover, it presents no more real
difficulty than boiling a dish of potatoes, or making a rice pudding;
and the neglect with which it is treated is therefore the less to be
comprehended or excused.
186. Only those persons who live habitually on good home-made bread, can form
an idea of the extent to which health is affected by their being deprived of it.
We have been appealed to on several occasions for household loaves—
which we have sent to a considerable distance—by friends who complained
of being rendered really ill by the bread which they were compelled to eat in
the sea-side towns and in other places of fashionable resort; and in London
we have heard incessant complaints both from foreigners and habitual
residents, of the impossibility of obtaining really wholesome bread.