You are on page 1of 150

1

HSS 4101: Development and


Evaluation of Health Programs

Faculté des sciences de la santé | Faculty of Health Sciences


uOttawa.ca
2

Welcome to HSS 4101!

Lara Pilutti, PhD


Interdisciplinary School of Health Science

Office hours: Tuesdays 2:00-3:00pm in-person (Lees


E250G) or via Zoom
lpilutti@uottawa.ca

Emails: Please include HSS 4101A in subject line and allow


2 business days response time

uOttawa.ca
3

Outline

1. Syllabus review & course organization


• Course format
• Required readings & resources
• Objectives & content
• Evaluation methods
• Other information

2. Introduction to health promotion programs and


planning
• Reading: At A Glance: The six steps for planning
a health promotion program

uOttawa.ca
4

Course Format

• Meeting dates/times (Section A):


– Tuesday 11:30-12:50pm
– Friday 1:00-2:20pm

• Delivery mode: Hybrid


– Mix of in-person and online (Zoom) classes
– Follow topic outline & calendar for delivery method

uOttawa.ca
5

Course Format - Brightspace

• Main form of communication and dissemination of


course materials
• All course content will be posted to Brightspace

• Submission and grading of assignments will be


managed through Brightspace
• Links to live Zoom lectures and office hours via
Brightspace
• Please check Brightspace regularly

uOttawa.ca
6

Course Format – Office Hours

• Available before and after class


• Tuesdays 2:00-3:00pm in-person (Lees E250G) or via
Zoom
• Zoom office hours – drop-in, one-on-one meetings
• Please let me know if you would like to meet as a
group

uOttawa.ca
7

Readings & Resources

Links to readings within syllabus and on Brightspace

uOttawa.ca
8

Readings & Resources

Planning workbook
• Ontario Agency for Health
Protection and Promotion (Public
Health Ontario). Planning Health
Promotion Programs:
Introductory Workbook. 5th ed.
Toronto, ON: Queen's Printer for
Ontario; 2018.

uOttawa.ca
9

Readings & Resources

Evaluation workbook
• Ontario Agency for Health Protection
and Promotion (Public Health
Ontario), Snelling S, Meserve A.
Evaluating health promotion
programs: introductory workbook.
Toronto, ON: Queen's Printer for
Ontario; 2016.

uOttawa.ca
10

Planning Workbook & Worksheets

uOttawa.ca
https://www.publichealthontario.ca/en/health-topics/public-health-practice/program-planning-evaluation/planning-programs
11

uOttawa.ca
https://www.publichealthontario.ca/en/health-topics/public-health-practice/program-planning-evaluation/planning-programs
12

Readings & Resources

uOttawa.ca
13

uOttawa.ca
14

uOttawa.ca
15

uOttawa.ca
16

Evaluation

uOttawa.ca
17

Evaluation: Midterm & Final Exam

• Mix of multiple choice and short answer questions

• Topics covered on exams will be drawn from lecture


materials, course readings, guest lectures, and class
discussions

• The midterm exam will be held in class and will last the
duration of the class period

• Midterm exam content will cover Planning Steps 1-5 and


guest lectures

uOttawa.ca
18

Evaluation: Midterm & Final Exam

• The final exam will be held during the final examination


period (April 14-27)

• The final exam will primarily cover Planning Step 6,


Evaluation Steps 1-10 & guest lectures; however, students
should be familiar with key concepts that are built upon
throughout the entire semester

• Please review all exam procedures within the syllabus

uOttawa.ca
19

Evaluation: Midterm & Final Exam

• Deferred exams:
– Only be offered in exceptional circumstances
– Students can request a maximum of one (1) deferred
evaluation per course
– Approval to write a make-up exam must be confirmed in
advanced with the professor
– DFR request forms are only required for final exams
• Midterm make-up: February 21st, 10:00am & 1:00pm
• School-hosted exam make-up sessions: April 28th,
9:00am & 1:00pm

uOttawa.ca
20

Evaluation: Group Project

• Goal is to gain experience applying the steps involved in


planning, developing, and evaluating a health program
• Groups of 5-6 students
• The project will be completed in three components:
Proposal, presentation & final paper
• Project overview, instructions, templates and grading
rubrics for each component of the project are posted in
Brightspace
• Class time will be dedicated to project application and
review
• New this year – CSL/CityStudio opportunity

uOttawa.ca
21

Syllabus - Other Information

• Please review syllabus details carefully


– Late submissions and missed evaluation policies
– Grade review and letter grade assignment
– Academic fraud and plagiarism
– Class code of conduct
• Email policies – please include HSS4101A in subject
line; please allow 2 business days for response
– Student resources

uOttawa.ca
22

QUESTIONS?

uOttawa.ca
23

INTRODUCTION TO HEALTH
PROMOTION PROGRAMS
uOttawa.ca
24

Health Promotion

https://www.youtube.com/watch?v=G2quVLcJVBk
uOttawa.ca
World Health Organization, 1986; https://www.phac-aspc.gc.ca/ph-sp/docs/charter-chartre/pdf/charter.pdf
25

Health Promotion

“The process of enabling


people to increase control over,
and to improve, their health.”
Ottawa Charter for Health Promotion, 1986

• 3 Strategies
• 5 Action Areas

uOttawa.ca
World Health Organization, 1986; https://www.phac-aspc.gc.ca/ph-sp/docs/charter-chartre/pdf/charter.pdf
26

Health Equity

• Health inequalities: differences in the health status of


individuals and groups

• Health inequity: health inequalities that are unfair or unjust


and modifiable

• Health equity: absence of unfair systems and policies that


cause health inequalities. Health equity seeks to reduce
inequalities and to increase access to opportunities and
conditions conducive to health for all

uOttawa.ca
Social Determinants of Health and Health Inequalities Government of Canada; 2022.
27

uOttawa.ca
Raphael et al., 2020. Social Determinants of Health: The Canadian Facts. Oshawa: Ontario Tech University Faculty of Health Sciences and
Toronto: York University School of Health Policy and Management.
28

Socio-ecological Model

• Complex interactions
between people, groups and
their environments
• Helpful for understanding
interactions between and
across different levels
• Helps to develop, deliver &
evaluate health programs

uOttawa.ca
McLeroy et al., Health Education Quarterly 1988;15:351-377
https://blogs.uw.edu/somehm/2017/08/12/social-ecological-model/
29

Socio-ecological
Model Applied

uOttawa.ca
Orpana et al., Health Promotion and Chronic Disease Prevention in Canada, Research
Policy & Practice 2016;36:1-10
30

What is a Health Promotion Program?

• Focus on improving “population health outcomes


by reducing preventable disease, injury and death
and taking action on health inequities.”

• Planned strategies and activities designed to


improve health behaviours and outcomes within a
target population

• Examples? So, how do you create


a health promotion
program?
uOttawa.ca
Ontario Ministry of Health and Long-Term Care, 2018
31

Creating Health Programs

Evaluate
Identify
the
program a need

Plan and
implement
the program

uOttawa.ca

https://www.participaction.com/
32

ParticipACTION

https://www.youtube.com/watch?v=7VAEESeLaQo&t=35s
uOttawa.ca
33

Creating Health Programs

Evaluate
Identify
the
program a need

Plan and
implement
the program

uOttawa.ca

https://www.participaction.com/
34

What is Planning All About?

• Collecting and analyzing information and evidence


• Creation of activities to achieve desired goal
• System of organization for where to go and how to
get there
• Provides direction for resource allocation
• Helps to align expectations
• Adapting and modifying plan as circumstances
change

uOttawa.ca

Public Health Ontario, 2018


35

Models for Program Planning

• A framework can be used to organize the planning


process
• Helps to provide structure and direction
• Several different approaches exist; most share
common elements
• Approach will depend on circumstances and
organization
• Our approach: Ontario Six-Step Planning Model

uOttawa.ca
36

Step 1: How will


Six-Step Planning Model key partners work
together to make
decisions
Step 2: “Should
we proceed, and
if so, how?”

Steps 3-5: Planning


decisions relative to
program goals,
objectives, strategies,
and indicators

Step 6: Review
of the plan using
uOttawa.ca
a logic model
Public Health Ontario, 2018
37

Other Planning Model Examples

• MAPP: Mobilization for


Action Through
Planning and
Partnership

uOttawa.ca
Achieving Healthier Communities through MAPP: A User’s Handbook by the National
Association of County and City Health Officials
38
PRECEDE – "predisposing, reinforcing, and
enabling constructs in educational/ecological
PRECEDE-PROCEED diagnosis & evaluation"

PROCEED – "policy,
regulatory, and
organizational constructs
in educational &
environmental
development"

uOttawa.ca
Green & Kreuter, 2005. Health promotion planning: An educational and ecological approach.
McGraw-Hill, New York; Porter et al., Health Education Journal 2016;75;753-764.
39

uOttawa.ca

http://bchealthycommunities.ca/wp-content/uploads/2020/05/The_Healthy_Communities_Approach_A_Framework_For_Action_2011.pdf
40

Next Class…

• Step 1: Manage the planning process


– Reading: Step 1 – Planning Workbook
– At A Glance: The six steps for planning a health
promotion program

• Group Project Introduction

uOttawa.ca
41

QUESTIONS?

uOttawa.ca
42

Group Project Introduction

Faculté des sciences de la santé | Faculty of Health Sciences


uOttawa.ca
43

Supporting Documents – Brightspace

• Project overview
• Proposal – instructions, grading rubric, templates
• Presentation – instructions, grading rubric
• Final paper – instructions, grading rubric, templates

uOttawa.ca
44

Overview

• Gain experience with key steps in planning, designing,


implementing, and evaluating health programs

• Apply course concepts to real-world health issues and contexts

• Students will select a health topic/behaviour of interest

• Assume the role of a health planner/promoter/evaluator

• Develop a complete health program to address the health


problem following the steps outlined in class
• The project will not include all steps covered in class

uOttawa.ca
45

Overview

• The project will be completed in the following three


components:
– Proposal (5% of final grade) – due February 3rd
– Presentation of the health program (10% of final grade) –
slides due March 23*; presentation dates March 24 – April 4
– Final paper (25% of final grade) – due March 31st

• Class time dedicated to project application and review

• Examples will be provided throughout the course & via guest


lectures

uOttawa.ca
46

Format

• Project must be completed in groups of 5-6 students maximum

• All students must self-enroll to a group within the ‘groups’ tab


on Brightspace

• All students must self-enroll to a group by Friday January 20th

• Students who do not enroll to a group by this date will


automatically be enrolled to a group

• No individual assignments will be accepted


• Discussion topic within ‘Team project’ section on Brightspace

uOttawa.ca
47

Format

• Course submissions may be made in either official language


• Academic Regulation 8.5: “Except in programs and courses
for which language is a requirement, all students have the
right to produce their written work and to answer
examination questions in the official language of their
choice, regardless of the course’s language of instruction.”

uOttawa.ca
48

Format – Presentation

• 12-minute presentation maximum


• Power Point slides or equivalent format
• Presentations will be delivered via Brightspace Zoom
• Groups will have the opportunity to select their presentation
date
• All presentations slides must be uploaded to Brightspace by
the due date regardless of the presentation date
• All group members must be present on the selected
presentation date

uOttawa.ca
49

Assignment Submissions

• All assignments will be submitted electronically on


Brightspace

• Submissions are due by 11:59 pm on the dates specified

• It is your responsibility to ensure that all assignments are


fully submitted in Brightspace by the due date/time indicated

• The submission date/time as recorded in Brightspace will be


considered the official timestamp of submission

• NO extensions will be granted

uOttawa.ca
50

Proposal

1. Program overview (~1 page – template provided)

2. Situational assessment plan (~1 page – template provided)

3. Team contract (~1 page – template provided)

*See instructions, grading rubric, and templates for additional


details*

uOttawa.ca
51

Templates

uOttawa.ca
52

Grading Rubrics

uOttawa.ca
53

Presentation

1. Background

2. Program description

3. Significance and relevance

*See instructions and grading rubric for additional details*

uOttawa.ca
54

Final Paper

A. Program logic model (~1 page – figure format)

B. Situational assessment (~3-4 pages “executive summary”)

C. Intervention plan (~2 pages)

D. Program evaluation plan (~3-4 pages – template provided)

E. Ethical considerations (~1 page – template provided)

*See instructions, grading rubric, and templates for additional


details*

uOttawa.ca
55

Formatting Guidelines

• Please adhere to page length and formatting requirements:


– Page limits exclude title page and references
– Page limits include all tables/figures (must use templates
where indicated)
• Word or PDF documents:
– Text: double-spaced, 12-point font, 1-inch margins
– Tables: single-spaced, 12-point font, 1-inch margins
– Portrait or landscape orientation
• All pages must be numbered and appear in a consecutive order
• Clearly identify each section (1-3) and subsection (a-c)

uOttawa.ca
56

Formatting Guidelines

• References:
– Not required for proposal; ~15-20 for final paper
– Consistent referencing format (e.g., APA, Vancouver,
Nature, etc.) – may use any standardized, peer-reviewed
journal reference formatting
– Peer-reviewed and grey literature are acceptable sources
– Ensure references are current and reliable sources

• Upload one single file with the submission; one submission


per group

uOttawa.ca
57

QUESTIONS

uOttawa.ca
58

Project Topic Selection

• Can focus on priority topic identified through the


Community Service Learning (CSL)/CityStudio initiative

• CSL/CityStudio collaboration: City of Ottawa


– Topic: Mental Well-being (Mental health and addiction
services access and literacy)
– City of Ottawa project overview posted on Brightspace
*please review for next week*
– Introduction: in class January 20th

• OR may select a topic of your choice

uOttawa.ca
https://www.csottawa.ca/
59

Where to Begin?
?

? Health
Setting/
issue/
context
outcomes

Target
audience/
population

uOttawa.ca
60

CPHO Report 2017 & 2018

uOttawa.ca
Public Health Agency of Canada, 2017; 2018
61

CPHO Report 2019

https://www.youtube.com/wat
ch?v=omA3yaruV14

uOttawa.ca
Public Health Agency of Canada, 2019
62

CPHO Report 2020

https://www.youtube.com/watch?
v=_wt4WcZM_0A

uOttawa.ca
Public Health Agency of Canada, 2020
63

CPHO Report 2021

uOttawa.ca
Public Health Agency of Canada, 2021
64

CPHO Report 2022

uOttawa.ca
Public Health Agency of Canada, 2022
65

Readings & Resources

uOttawa.ca
66

Next Class: Situational Assessment

• Reading:
– Step 2 - Planning Health
Promotion Programs:
Introductory Workbook
– Focus On: Six strategic
steps for situational
assessment

• Sign up for a group on


Brightspace
• Review CityStudio project
overview

uOttawa.ca
67

Wellness Week 2023

Wellness Week | Campus life (uottawa.ca)


uOttawa.ca
68

Situational Assessment

Faculté des sciences de la santé | Faculty of Health Sciences


uOttawa.ca
69

Situational Assessment

• Readings:
– Step 2 – Planning workbook (p. 12-27)
– Focus On: Six strategic steps for situational
assessment

uOttawa.ca
70

uOttawa.ca
71

Objectives
1. Summarize the concept of situational assessment

2. Understand and apply the 6 key steps in a situational


assessment

3. Describe the types, sources and methods of data collection


that can be used in situational assessment

4. Evaluate and synthesize different sources of data to


determine how to proceed in program development

5. Understand how to communicate situational assessment


results using different formats and for different audiences

uOttawa.ca
72

What is a Situational Assessment?

• “…influences planning in significant ways by examining the


legal and political environment, project partners, the health
needs of the population, the literature and previous
evaluations, as well as the overall vision for the project.”

• Different than ‘Needs Assessment’


– Focuses on problems, deficits, & difficulties
– Ignores strengths, opportunities, & assets

• Foundation to guide future planning decisions – very


important!

Population Health Assessment and Surveillance Protocol (2018); Ontario


Public Health Standards (2018); Public Health Ontario, 2018
uOttawa.ca
73
How much
data do
6 Steps in Situational Assessment you need?

1. Identify key questions to answer through the assessment


2. Develop a data-gathering plan
3. Gather the data
4. Organize, synthesize and summarize the data
5. Communicate the information to key partners and
decision-makers
6. Consider how to proceed with planning

Consider previous discussions with


project partners, timeline & budget
uOttawa.ca

Public Health Ontario, 2018


74

Approaches to Situational Assessment

Chronic disease, cancer research

uOttawa.ca

Public Health Ontario, 2018


75

Evidence-Informed Decision Making (EIDM)

uOttawa.ca
https://www.nccmt.ca/tools/eiph
76

uOttawa.ca
https://www.nccmt.ca/tools/eiph
77

6 Steps in Situational Assessment

1. Identify key questions to answer through the assessment


2. Develop a data-gathering plan
3. Gather the data
4. Organize, synthesize and summarize the data
5. Communicate the information to key partners and
decision-makers
6. Consider how to proceed with planning

uOttawa.ca

Public Health Ontario, 2018


78

Step 1: Identify key questions to be


answered Key (Questions to Answer)

1. What is the situation?

2. What is making the situation better or worse?

3. What possible solutions, interventions and


actions can be taken to deal with the situation?

Worksheet 2.1

uOttawa.ca

Public Health Ontario, 2018


79

Step 1: Identify key questions to be answered


Key (Questions and how to Answer)
1. What is the situation? Worksheet 2.1
• What impact does the current situation have on health outcomes,
quality of life and other societal costs, such as noise, air pollution
or increased healthcare spending?
• Which groups of people are at higher risk of health problems and
poorer quality of life?
• Which settings or situations are high risk, or pose a unique
opportunity for intervention?
• How do local stakeholders perceive the situation? What is their
capacity to act? What are their interests, mandates, current
activities?
• What are the needs, perceptions and supported directions of key
influential community members, and the community-at-large?

uOttawa.ca

Public Health Ontario, 2018


80

Step 1: Identify key questions to be answered


Key (Questions and how to Answer)
2. What is making the situation better or worse?
• What political, economic, environmental, social and technological trends are
influencing the situation?
• What internal strengths and weaknesses are present in your organization that
may affect the course of action?
• What high-risk or negative health behaviours by various groups of people are affecting
the situation?
• Which underlying causes or conditions are driving these behaviours (e.g. individual,
community, organizational or system-level causes)? Are there protective factors that
can help avoid or alleviate the situation (such as ensuring walkable communities or
encouraging strong parent-child relationships)?
• Which strengths and weaknesses present in your organization may affect your course
of action? Which opportunities and threats in your environment may affect your course
of action?

Worksheet 2.1

uOttawa.ca

Public Health Ontario, 2018


81

Step 1: Identify key questions to be answered


Key (Questions and how to Answer)
3. What possible solutions, interventions and actions can be taken to deal with the
situation?
• What are other similar organizations doing to address this situation?
• What has your organization done in the past?
• What evidence exists to support various courses of action?
• What are other organizations doing, or what have they done in the past, to address
this situation? Specifically, what local policies, programs and environmental supports
are being developed or implemented within the community? What evaluation data are
available for these activities?
• What is the best available evidence that exists to support various courses of action?

Worksheet 2.1

uOttawa.ca

Public Health Ontario, 2018


82

Frameworks
for Developing
Questions

• PICO/PECO

Want specifics

Health Evidence™ (2009, November 25). Developing an Efficient Search Strategy. Retrieved
[insert date you downloaded this document, e.g. January 13, 2018],
uOttawa.ca https://healthevidence.org/practice-tools.aspx#PT2
83

Frameworks for Developing Questions

uOttawa.ca
PublicHealthOntario.ca
84

uOttawa.ca
PublicHealthOntario.ca
85

Frameworks for Developing Questions

• Qualitative approaches

Health Evidence™ (2009, November 25). Developing an Efficient Search Strategy. Retrieved
[insert date you downloaded this document, e.g. January 13, 2018],
uOttawa.ca https://healthevidence.org/practice-tools.aspx#PT2
86

1. Key Questions to Answer


1. What is the situation?
- What impact is the current situation having on health and quality of life for various groups of people?
- Which groups of people are at higher risk of health problems and poorer quality of life?
- How do local stakeholders and the public perceive the situation?
- How do local stakeholders and community members describe their needs related to the situation?
- What is currently known about the determinants (causes) of any identified health inequities, particularly as they relate to the situation?
- What solutions do local stakeholders and community members currently favour and why?
- What are the benefits of acting now versus later
1. What is making the situation better or worse?
- What risky or negative health behaviours of various groups are affecting the situation?
- What political, economic, environmental, social and technological trends are influencing the situation?
- What conditions in the social or organizational environment, or at the broader public policy level, are causing or helping to
alleviate the situation?
- What internal strengths and weaknesses are present in your organization that may affect the course of action?
- What opportunities and threats in your environment may affect the course of action?
- What unintended impacts, both positive and negative, might your program have on program participants and the broader public?
1. What possible solutions, interventions and actions can be taken to deal with the situation?
- What are other similar organizations doing to address this situation?
- What has your organization done in the past?
- What strategies and/or program modifications could you feasibly implement to maximize positive impacts and minimize negative
ones?
- What evidence exists to support various courses of action?
uOttawa.ca
87

2. Develop Data Gathering Plan

• Clear and specific plan for guiding data collection process


• Questions will guide approach for data gathering
• Factors to consider:
– Planning context
– Types and sources of data
– Methods for data collection
– Quality and rigor of data
– Importance and feasibility of data collection
• Plan for time, expertise, roles, and resources necessary to
carry-out this process
Worksheet 2.2

uOttawa.ca

Public Health Ontario, 2018


88

2. Develop Data Gathering Plan

• Plan should detail where and how collected data will be


organized
– Evidence table: source of information, date, research
design, quality, type of evidence, research question it
addresses …
• Plan for how references and resources will be managed
• Very important when multiple team members are collecting
data
• Review plan with project partners

Worksheet 2.2

uOttawa.ca

Public Health Ontario, 2018


89

3. Gather the Data

• Primary vs. secondary data:


– Primary – information that is collected first hand
• Source can be target audience or others
– Secondary audiences, key informants,
community leaders, etc.

– Secondary – information that has already been


collected by someone else

uOttawa.ca

Public Health Ontario, 2018


90

Primary Data Collection

• Methods of primary data collection:


– Surveys/questionnaires (face-to-face, written,
telephone, electronic)
– Interviews & focus groups
– Community forum or meetings
– Media scan/media listening – (already existing)
– Self-assessment/organizational assessment tools
– Existing internal data

uOttawa.ca
McKenzie et al, 2016. Planning, Implementing & Evaluating Health Promotion Programs: A Primer. Pearson Education Inc.
Mazza et al 2022 Frontiers in Oncology 2022;12:863641
91

uOttawa.ca
www.york.ca/workplacewellness
92

Primary Data Collection

• Methods of primary data:


– Observation
• Specific approaches:
– Windshield tours
– Walk-throughs
– Photovoice

uOttawa.ca
McKenzie et al, 2016. Planning, Implementing & Evaluating Health Promotion Programs: A Primer. Pearson Education Inc.
BMC Public Health. 2015; 15: 605.
93

Secondary Data Collection

• Methods: reviews and searches


• Sources of secondary data:
– Existing peer-reviewed literature
• Primary research articles
• Reviews & meta-analyses
– Government
– Organizations
– Existing records and databases
– Existing organizational plans and reports
– Guidelines/best practice documents

uOttawa.ca
94

Data Sources Relative to Key Questions

uOttawa.ca

Public Health Ontario, 2018


95

Data Sources Relative to Key Questions

uOttawa.ca

Public Health Ontario, 2018


96

Data Sources Relative to Key Questions

uOttawa.ca

Public Health Ontario, 2018


97

Types, Methods & Sources of Data

1. What is the situation?


– E.g., ?
2. What is making the situation better or worse?
– E.g., ?
• How do local stakeholders and community members describe their
needs related to the situation?

1. What possible solutions, interventions and


actions can be taken to deal with the situation?
Worksheet 2.1
– E.g., ?
uOttawa.ca
98

Secondary Data Collection

uOttawa.ca
https://cbpp-pcpe.phac-aspc.gc.ca/public-health-topics/mental-health-and-wellness/
99

Secondary Data Collection

https://www.healthevidence.org/default.aspx
uOttawa.ca
100

Secondary Data Collection

uOttawa.ca
https://health-infobase.canada.ca/
https://health-infobase.canada.ca/data-tools/cypc/
101

Secondary Data
Collection

Ontario Agency for Health Protection and Promotion (Public Health Ontario). Medeiros A, Richmond SA. Effectiveness of
awareness campaigns, education and training on road safety. Toronto, ON: Queen's Printer for Ontario; 2020

uOttawa.ca
102

uOttawa.ca
https://csepguidelines.ca/downloads/
https://ccsa.ca/sites/default/files/2022-08/CCSA-LRDG-Update-of-Canada%27s-LRDG-Final-report-for-public-consultation-en.pdf
103

Evidence-Informed Decision Making (EIDM)

uOttawa.ca
https://www.nccmt.ca/tools/eiph
104

Evidence Quality Assessment

Quality Assessment of Community Evidence (QACE) Tools

National Collaborating Centre for Methods and Tools (NCCMT) 2020


uOttawa.ca User’s Guide: Quality Assessment of Community Evidence (QACE)
Tools; https://www.nccmt.ca/qace
105

National Collaborating Centre for Methods and Tools (NCCMT) 2020


uOttawa.ca User’s Guide: Quality Assessment of Community Evidence (QACE)
Tools; https://www.nccmt.ca/qace
106

National Collaborating Centre for Methods and Tools (NCCMT) 2020


uOttawa.ca User’s Guide: Quality Assessment of Community Evidence (QACE)
Tools; https://www.nccmt.ca/qace
107

Health Sciences Library

https://uottawa.libguides.com/c.php?g=265115

uOttawa.ca Peter Farrell (office365.com)


108

6 Steps in Situational Assessment

1. Identify key questions to answer through the assessment


2. Develop a data-gathering plan
3. Gather the data
4. Organize, synthesize and summarize the data
5. Communicate the information to key partners and
decision-makers
6. Consider how to proceed with planning

uOttawa.ca

Public Health Ontario, 2018


109

Next Class…

• CSL/City Studio Introduction


– CSL/CityStudio overview
– City of Ottawa overview

• Situational Assessment – Part 2 (Tuesday)

*By Friday - Sign up for a Group on Brightspace*

uOttawa.ca
110

Purpose of Health Program

Faculté des sciences de la santé | Faculty of Health Sciences


uOttawa.ca
111

uOttawa.ca
112

Setting Goals, Audiences & Objectives

• Reading:
– Step 3 – Planning workbook (p. 28-45)

uOttawa.ca
113

Objectives
1. Define common planning terminology surrounding
goals, audiences, objectives, strategies, and
activities
2. Describe how to select target audiences for a
health intervention
3. Create appropriate goals and objectives for a
health intervention

uOttawa.ca
114

Planning Goal
Terminology
Audiences

Logic Model Objectives

Strategies
Terms are
specific Activities

Resources
uOttawa.ca
115

Planning Terminology: Goals

• Broad statement that describes results to be achieved


• Provides overall direction and intent of program
• Typically contain verbs:
– Improve, increase/decrease, understand, promote,
prevent …

• E.g., To improve the availability of local, fresh fruits


and vegetables in neighbourhoods throughout the
region
Worksheet 3.1

uOttawa.ca

Public Health Ontario, 2018


116

Planning Terminology: Audiences

• The intended beneficiary of the health program


• Consider which individuals/settings are in the
greatest need and are willing/able to make a change
– Primary:
• Direct recipients
• Who you hope to engage/change health
• Egs children
– Secondary:
• Indirect recipients
• People who influence the primary audience
uOttawa.ca • Eg: parents
Public Health Ontario, 2018
117

Planning Terminology: Objectives

• Specify the desired impact or effect of health


program over a defined period of time
• Represent smaller, short-term steps that lead to
achievement of overall goal
• Specify who, what, how much, and by when
• Include outcome and process objectives
• Include short-, medium- and long-term objectives

uOttawa.ca

Public Health Ontario, 2018


118

Planning Terminology: Goals vs. Objectives

uOttawa.ca

Public Health Ontario, 2018


119

Developing Objectives: Key Elements


Every
1. Who? objective must
– The target audience include all 4!
– Health outcome your trying to change
2. What?
– The outcome to be achieved, or what will change
3. How much?
– The criterion for deciding whether the outcome has
been achieved
4. By when?
– When the change will occur

uOttawa.ca
McKenzie et al, 2016. Planning, Implementing & Evaluating Health Promotion Programs: A Primer. Pearson Education Inc.
120

Developing Objectives: Key Elements

• Who (audience/target)? • How much (criterion)?


– Participants – 10% improvement
– Employees – A majority
– Students – At least 25
• What (outcome)? – What has been done
– Awareness of risk factors and how much
for skin cancer
• By when (timeline)?
– Stigma surrounding
– A specific date
substance use
– After the program
– Access to mental health
services – 6-months after the
course
uOttawa.ca
McKenzie et al, 2016. Planning, Implementing & Evaluating Health Promotion Programs: A Primer. Pearson Education Inc.
121

Planning Terminology: Objectives

• Outcome objectives:
– Specify the intended effect of the program
– What your target audience will know/be able to do
as a result of your program/activity
– Can be short-, medium- and long-term

E.g., To increase by 20%, the percentage of seniors


in Ottawa who believe that they are personally at
risk of falling, by May 31, 2024.

uOttawa.ca

Public Health Ontario, 2018


122

Example
Niagara Road Safety Campaign

• Goal: To reduce the number of motor vehicle


collisions and related injuries in the region

• Audience:
– Primary: Male drivers within the region, aged
16-40
– Secondary: All drivers who work and live in the
region
Worksheet 3.1

uOttawa.ca
Public Health Ontario, OHPP Sample Plan Master - Think and Drive
123

Example
HOW MUCH
• Outcome objectives: WHO
1. To increase by 25%, the number of drivers and road
users in the region (with a focus on male drivers) who
are aware of the region's crash problem and of road
WHAT safety skills related to the crash problem, by December
2008. BY WHEN

2. To increase by 20%, the percentage of drivers in the


region (with a focus on male drivers) who report positive
attitudes in terms of taking personal responsibility for
road safety, by December 2008.
Worksheet 3.1

uOttawa.ca
Public Health Ontario, OHPP Sample Plan Master - Think and Drive
124

Example
• Outcome objectives:
3. To increase by 10%, the percentage of drivers and road
users in the region (with a focus on male drivers) who
report positive changes in driving behaviour/safer
driving habits, by December 2008.

4. To decrease by 10%, the number of motor vehicle


collisions where factors related to driver action are a
primary consideration, by December 2011.

Worksheet 3.1

uOttawa.ca
Public Health Ontario, OHPP Sample Plan Master - Think and Drive
125

Developing Objectives

• Short-term: ~several months to 1 year


– Early changes expected
– E.g., Knowledge, attitude, awareness, skills

• Medium-term: ~1-5 years


– Result from short-term changes
– E.g., Behaviours, norms, policies Change
takes time!
• Long-term: ~5 years+
– Ultimate anticipated health impact
– E.g., Morbidity, mortality, quality of life
uOttawa.ca

Public Health Ontario, 2018


126

Developing Objectives: Levels of Change

Results from situational


uOttawa.ca
assessment should guide process
McLeroy et al., Health Education Quarterly 1988;15:351-377
Public Health Ontario, 2018
127

Developing Objectives: Levels of Change

• Individual: target awareness, attitudes,


knowledge, intentions, behaviours

• Interpersonal: target social networks, supportive


activity, connections

• Organizational: target policy change, program


adoption, agenda building within organizations

Tables 6 & 7

uOttawa.ca

Public Health Ontario, 2018


128

Developing Objectives: Levels of Change

• Community: target collaborations, social norms


and support within community settings

• Public policy: target societal norms/values, public


policy support, implementation and change

Tables 6 & 7

uOttawa.ca

Public Health Ontario, 2018


129

Levels of Change (Table 8)

uOttawa.ca

Public Health Ontario, 2018


130

Levels of Change (Table 8)

uOttawa.ca

Public Health Ontario, 2018


131

Planning Goal
Terminology
Audiences

Objectives

Strategies
Intervention
Activities

Resources
uOttawa.ca
132

Planning Terminology: Strategies, Activities &


Resources
• Strategies: Broad approach or plan that helps achieve the
identified change
– E.g., health education, create supportive environments,
build healthy public policy

• Activities: Specific product or service


– E.g., brochures, fact sheets, workshops, clinics

• Resources: Quantifiable investments that support


program implementation
– E.g., personnel, financial, material
uOttawa.ca

Public Health Ontario, 2018


133

Planning Terminology: Process Objectives

• Process objectives:
– Describe program activity and implementation
elements
– Typically short-term

E.g., To develop six, 90-minute workshops on


effective parenting strategies for first-time parents,
by September 30, 2023.

uOttawa.ca

Public Health Ontario, 2018


134

Example
• Process objectives: HOW MUCH
1. To distribute 5,000 brochures for pilot and 75,000 for
WHAT full campaign on the topic of safety items on passenger
vehicles, for homes with CAA membership, by Sept.
2006 and May 2007, respectively.
WHO
BY WHEN
2. To develop one Regional Road Safety Annual Report,
for the general public/target audience, by June
annually, 2006-2008.

uOttawa.ca

Public Health Ontario, 2018


135

Planning health program


GROUP PROJECT

uOttawa.ca
136

Project Proposal

1. Program overview (~1 page – template provided)

- Creative and organized

2. Situational assessment plan (~1 page – template provided)

3. Team contract (~1 page – template provided)

uOttawa.ca
137

1. Program Overview (~1 page)

a. Program name/title (e.g., ‘Think and Drive’)


b. Identify the organization where the health program will be
developed and delivered (e.g., Ottawa Public Health)
c. List the proposed target audience(s) and setting (e.g., school,
workplace, community)
d. Identify the key project partners (minimum of 5 different
parties)

uOttawa.ca
138

Program Overview

uOttawa.ca
139

Project Partners role

• Core: participate on the


planning team

• Involved: frequently consulted


as part of the planning process

• Supportive: provide some form


of assistance

• Peripheral: need to be kept


informed at key junctures
Worksheet 1.2

uOttawa.ca
140

2. Situational Assessment Plan (~1 page)

a. List the research questions that will be used to guide your


situational assessment (minimum of 3 questions - 1 per key
question area in situational assessment)
– What is the situation?
– Factors making it better/worse?
– Possible solutions?

b. Describe the planned methods for data collection including


types of data, methods for collection, and specific sources
that will be used

uOttawa.ca
141

Situational Assessment Plan

uOttawa.ca
142

Team Contract (1 page maximum)


• List the role/contribution of each team member to the project
• Each team member must sign and date the contract

Reach out/connect with


team members

uOttawa.ca
143

CSL/CityStudio – Project Topic

• The CSWB plan aims to improve system navigation for people


to access services and supports for mental well-being,
problematic substance use, and substance use disorders
• The plan also wants to promote mental well-being in a stigma-
free environment
• Knowledge gaps in the areas of system navigation and stigma
reduction have been identified

uOttawa.ca

City of Ottawa_CityStudio_MWB Project Description


144

CSL/CityStudio – Project Topic

• The overarching question is, “How can we improve system


navigation for residents to access mental health and addiction
services and information to improve their mental health
literacy?“

• Example research questions:


– What does it take to navigate Ottawa’s mental health and
addiction system for a client?
– How can we leverage social media to improve mental health
literacy and reduce stigma?

uOttawa.ca

City of Ottawa_CityStudio_MWB Project Description


145

Resources

• WHO Report
• Mental Health Commission of Canada
• Canadian Mental Health Association

uOttawa.ca https://mentalhealthcommission.ca/mhcc-strategic-plan-2021-2031/
World mental health report: transforming mental health for all.
Geneva: World Health Organization; 2022.
146

Socio-ecological
Model Applied

uOttawa.ca
Orpana et al., Health Promotion and Chronic Disease Prevention in Canada, Research
Policy & Practice 2016;36:1-10
147

Health Literacy & Mental Health Literacy

• Health literacy: “The ability to access, comprehend, evaluate


and communicate information as a way to promote, maintain
and improve health in a variety of settings across the life-
course.” Rootman & Gordon-El-Bihbety 2008, CPHA

• Mental health literacy:


– Understanding how to foster and maintain positive
mental health
– Understanding mental disorders and their treatments
– Decreasing stigma surrounding mental health
– Enhancing help seeking behaviours

uOttawa.ca
Watson & McDonald (2016). Mental Health Promotion: Let’s start speaking the same language. Waterloo, ON: Region of Waterloo Public Health
& Emergency Services; Kutcher, Wei & Coniglio (2016). Can J Psychiatry 61(3): 154–158. Rootman & Gordon-El-Bihbety (2008). A Vision for
a Health Literate Canada: Report of the Expert Panel on Health Literacy. Canadian Public Health Association.
148

CPHO Report 2019

https://www.youtube.com/watch?v=omA3yaruV14

https://www.canada.ca/en/public-health/corporate/publications/chief-
public-health-officer-reports-state-public-health-canada/addressing-
uOttawa.ca stigma-toward-more-inclusive-health-system.html
Public Health Agency of Canada, 2019
149

Where to Begin?
?

? Health
Setting/
issue/
context
outcomes

Target
audience/
population

uOttawa.ca
150

Next Week…

• uOttawa Health Promotion – guest lecture via Zoom

• Step 4: Strategies, Activities & Resources


– Planning workbook: Step 4

• Project proposal & CSL registration: Friday


February 3rd

uOttawa.ca

You might also like