Professional Documents
Culture Documents
Q CAF
Table of Contents
Introduction.........................................................................4
PROGRAM OVERVIEW............................................................4
History of LGBTQIA at Loyola University Chicago.................................................4
Program Context............................................................................................ 4
QCAF
Departmental Overview.................................................................................... 5
Selection Criteria of Q Cafe...............................................................................5
Program Description........................................................................................ 5
Program Purpose............................................................................................ 6
Conceptual Framework.................................................................................... 7
Current Research and Findings...........................................................................8
Stakeholders.................................................................................................. 8
Direct Stakeholders......................................................................................... 9
Indirect Stakeholders....................................................................................... 9
Marketing..................................................................................................... 9
Past Assessments.......................................................................................... 10
Logic Model........................................................................10
Inputs........................................................................................................ 11
Outputs...................................................................................................... 12
Outcomes................................................................................................... 13
Assumptions................................................................................................ 14
External Factors........................................................................................... 15
Evaluation Approach...........................................................15
Evaluation Questions..................................................................................... 16
Evaluation Standards..................................................................................... 17
QUANTITATIVE APPROACH....................................................17
Survey Population......................................................................................... 17
Sampling Method......................................................................................... 18
Survey Overview.......................................................................................... 18
Evaluation Design......................................................................................... 18
Survey Timeline........................................................................................... 19
Response Rate............................................................................................. 19
Incentives................................................................................................... 20
Survey Instrument......................................................................................... 20
Pilot Testing................................................................................................ 21
Statistical Analysis........................................................................................ 22
Final Report................................................................................................ 23
QUALITATIVE APPROACH......................................................23
Qualitative Approach Rationale........................................................................23
Focus Group Participation............................................................................... 25
Focus Group Protocol.................................................................................... 26
Focus Group Implementation...........................................................................27
Focus Group Analysis.................................................................................... 28
Limitations................................................................................................. 29
Results Presentation...................................................................................... 29
Budget...............................................................................30
Timeline.............................................................................30
QCAF
Next Steps.......................................................................... 31
References.........................................................................32
APPENDICES.......................................................................33
Appendix A: Q Cafe Logic Model.....................................................................34
Appendix B: Q Caf Google Forms Survey.........................................................35
Appendix C: Survey Construct Map...................................................................37
Appendix D: Focus Group Interview Protocol......................................................39
Appendix E: Q Caf Consent Forms..................................................................43
Appendix F: Email Communication Templates.....................................................45
Appendix G: Priori Construct Map....................................................................47
Appendix H: Budget...................................................................................... 48
Appendix I: Timeline..................................................................................... 49
Appendix J: PowerPoint Presentation.................................................................50
Introduction
Withamultitudeofprogrammingopportunitiesthebothofushaveexperienced
andcontinuetocoordinatethroughoutourhighereducationexperience,webothfound
theuniquecombinationoffusingtheconceptofcommunity,art,andidentity,inanopen
micnightintriguing.Whileprogramsareoftenverycutanddryfocusingonagendas,
detailedinformation,significantpreplanning,andincrediblyspecificanddetailed
learningobjectives,thefluidnatureofQCafwassomethingthatwasincredibly
QCAF
interestingtobothofusfromanevaluationandassessmentpointofview.Byexploring
theseconceptsthroughthegroundedconceptsofevaluationandassessment,wehopethat
thisdocumentcreatesasustainableplantoimprovetheprogramofQCafandengageon
anevendeeperandmoremeaningfullevel.
PROGRAM OVERVIEW
History of LGBTQIA at Loyola University Chicago
Loyola University Chicago started creating specific programming for LGBTQI
(Lesbian, Gay, Bisexual, Transgender, Queer/Questioning, Intersex) (Loyola University
Chicago, 2014a). Approximately three years ago, starting off with what were then called
Ally Trainings, and a year later, adding what has become known as Q-Caf.
Program Context
LGBTQIA+ Programming at Loyola University Chicago currently is and has
been housed within the Department of Student Diversity and Multicultural Affairs where
a graduate assistant, who is supervised by a program coordinator (who also coordinates
two other initiatives), executes most of the programming. As a Catholic, Jesuit
institution, the programs are explored not only on a level that focused on LGBTQIA
identity development, but also a level that is connected to Jesuit pedagogy and social
justice mission of the university.
Departmental Overview
Q-Caf is a two-hour monthly program that is coordinated and implemented by
the Department of Student Diversity & Multicultural Affairs. It is housed within the
QCAF
QCAF
of the school may oftentimes support LGBTQIA+ student identity development, there
have also been instances where significant controversy has arisen regarding Loyolas
recognition of LGBTQIA+ community folks (Loyola University Chicago, 2013).
Program Purpose
Q Caf is an open-mic night that was created three years ago to engage students
as well as Rogers Park and Edgewater community members through various forms of art.
At the time the program was initially created, there was a perceived lack of community
for LGBTQI individuals on campus, as stated in an informal conversation with the
program coordinator. Also, Q Caf provides a space for its participants to have a space to
touch on issues of identity through art. Participants are encouraged to write poetry, play
music, show art and engage in a creative, fun and interactive way.
The program is held at Metropolis Coffee Company (1039 W. Granville Ave.).
Metropolis Coffee Company is not only a local business that has rotating artists that
display their work, but also offers single-stall all gender inclusive restrooms, a point of
significance for many members of the LGBTQIA+ community. At Loyola, all restrooms
are current gendered, including the ones that are single stall, single occupancy they are
presented as unisex/family restrooms. As many may not identify with the sole identity of
man or woman, having a restroom that does not force someone into a binary system of
living is an asset to the program.
Conceptual Framework
Our conceptual framework for this evaluative study will be to try and determine
whether Q Caf is creating a space for LGBTQIA+ individuals to find community;
QCAF
whether Q Cafe is a space that embraces identities through various forms of art and
expression; and that whether it is able to generate conversations about LGBTQIA+ topics
(i.e. identity, politics, art, etc.) among its participants.
Major components and activities of the program are all centered on these
overarching goals. Tying the goals to the program itself, a typical session includes the
typical components: First, the program hopes to engage in one-on-one check-ins to see
how these students are doing on a personal level. This is done by interacting with staff to
make sure that they feel that staff is invested in them as well as to determine what else
can be done to support its students.
Second, the program provides teambuilding activities with its students to get to
interact with one another, but also learn more about themselves by participating in social
justice teambuilding activities. Most of the time some of these activities are reflective
exercises where they can speak about their intersecting identities and how it plays a part
into their development.
Third, the hope is that throughout these various activities they promote a good
sense of community. Community can mean various things, but the program hopes they
are able to connect with other students and hopefully gain meaningful relationships with
other students outside of the classroom.
The folks running the program sessions are all LGBTQIA+ identified staff,
graduate assistants, or student workers from the Department of Student Diversity &
Multicultural Affairs.
Current Research and Findings
The current research, specifically from the 2010 State of Higher Education for
Lesbian, Gay, Bisexual & Transgender People, indicates that spaces like this can be
QCAF
considered as part of campus potential best practices (Rankin et. al, 2010). The list of
potential best practices includes: Develop inclusive policies, demonstrate institution
commitment, integrate LGBTQQ issues and concerns in curricular and co-curricular
education, respond appropriately to anti-lgbtqq incidents/bias, create brave spaces for
student dialogues in on-campus housing, offer comprehensive counseling and healthcare,
and improve recruitment and retention efforts (Rankin et. al, 2010 p. 16 - 17). By
creating a space that has inclusive policies, is supported by the institution, and creates a
co-curricular space for LGBTQIA+ dialogues to occur, Q Caf as a singular program is
including multiple best practices to create a needed space.
Stakeholders
Stakeholders for this program include the students, faculty, staff, community
members, presenters/artists, Metropolis Coffee, and Loyola University Chicago as an
institution. While this sounds like an incredibly broad range of individuals and
organizations, utilizing the definition presented by Wholey et. al., (2010), stakeholders
are individuals, groups, or organizations that can affect or are affected by an evaluation
process or its findings (pg. 31).
Direct Stakeholders
The participants who directly are associated with the program namely, students,
faculty, staff, Rogers Park community members, Edgewater community members, and
presenters/artists, are all engaged in the space and are a part of the overall experience.
These different folks, by participating in an experience together, hopefully co-construct
an environment that meets the intended goals and outcomes mentioned earlier. These
QCAF
direct stakeholders are the individuals whose experiences are directly linked to the
program and its overall utility.
Indirect Stakeholders
Loyola University Chicago, as well as the Department of Student Diversity and
Multicultural Affairs, as supporters and funding sources of this program, have their name
and values directly linked and embedded within the execution of the program. The
Department of Student Diversity and Multicultural Affairs, as well as Loyola as a whole
institution, benefit from an affirming space that creates a lasting impact on students and
community members.
Marketing
Currently, staff members who work with the program have expressed some
frustration with not having a centralized marketing location. Staff have also expressed a
desire to see more engagement and solidarity with the LGBTQI community of Chicago.
Even with these concerns, though, the program has grown to a point where a venue
change to accommodate more attendees may be necessary. Metropolis Coffee is
comfortable in the staffs eyes for under twenty people, but if there is an influx for open
mic performances, a larger space would need to be acquired. The location change could
possibly affect the budget which currently includes only purchasing coffee, tea, and
pastries for attendees and $50 performance fees for guest artists that would effect how
the finite resources of the LGBTQI Initiatives within the department would be
distributed.
QCAF
10
Past Assessments
The program does have an assessment component and the data is collected each
time it convenes and is then stored for yearly reports that the Department of Student
Diversity & Multicultural Affairs creates. Currently, the assessment only asks a few
questions regarding community and support, but it could be changed to gather more data
that could create compelling suggestions to improve the program in the future.
Logic Model
In order to start and delve deeper into the evaluation process it is pivotal to begin
mapping out all that goes into program planning and reflect on the eventual results that
the program produces. It was important to create a logic model because it does not seem
as though the program has been evaluated from all facets specifically, the programs
past evaluation has only focused on basic satisfaction and perception of community. The
hope is that with the logic model we get a more well-rounded view as to how the program
is put together from start to finish and see what factors inhibit the program from reaching
its full potential. The logic model consists of various aspects such as: inputs, outputs,
outcomes, assumptions and external factors these will all be analyzed throughout this
piece.
Specifically with regards to stakeholders (both direct and indirect), logic models
prove to be a concise telling of a programs utility. As Wholey et. al,. (2010) stated, logic
models can be the basis for a convincing story of the programs expected performance
(pg. 56). By using a logic model to generate a map of sorts to explain the expected
QCAF
11
QCAF
12
express. Often times these participants are not given the opportunity to do so and that is
why this program was implemented as part as the Q initiatives to make sure that
participants feel valued at the university. Also, Q Caf provides a space folks to explore
identity through art. Participants are encouraged to write poetry, play music, show art
and engage in a creative, fun and interactive way.
Outputs
There were a few things that were considered when coming up with the actual
outputs of the program. We looked into the actual services that are provided to the
students that engage in the program. The participants are vital to making sure that the
activities that are put into place are successful. We found that students, staff, faculty and
of course Metropolis coffee were important to the success of Q Caf. Basic outputs
include students taking time to greet one another, gather coffee and pastries (provided by
the Department of Student Diversity & Multicultural Affairs), introducing an artistic
exercise or guest performer that identifies as LGBTQIA+, creating an open mic time for
participants to share what they choose to, and finally, ending the program with a largegroup processing conversation that focuses on the impact of art and sharing art with one
another. These outputs are referenced in Appendix A (the Logic Model) and are designed
from a programmatic view to create a space that is impactful and embraces the
experiences of its participants.
Outcomes
The third section of our logic model evaluates the short, medium, and long term
outcomes and goals of the program.
QCAF
13
There were two short term outcomes that Q Caf hopes to provide. Q Caf makes
sure to have something for these students each month and the continually look forward to
attending. Having a reoccurring program from the university that creates a space for
LGBTQI identified individuals (not just from Loyola) shows the institutional support and
buy-in that can be perceived as sometimes lacking. The second outcome in this section is
to provide coffee and pastries for attendees while not immediately something that
suggests importance and significance, the act of creating a shared experience in
community can lead to stronger bonds amongst participants.
There were a couple medium impacts that Q Caf provides. This was mentioned
earlier in that this program provides a space, but more importantly it provides a space that
can create cognitive dissonance in beginning stages of identity development. This is
done through the dialogues that are had each month, but it is also the intentional
conversations that are had with Metropolis Coffee to make sure that they are comfortable
with being the hosts for this program while serving as an ally for the Loyola community.
Most of the long-term goals are being achieved in giving students an artistic voice
to express their own identities. By creating intentional spaces that affirm and create an
area to explore identity, efficacy of the participants own lived truth and identity is a
hoped for by product for LGBTQIA+ individuals that attend Q Caf sessions. There is
also a significant hope that the Rogers Park and Edgewater Communities will engage and
connect with the program as well.
Assumptions
The assumptions section of the logic model really drives the reasons as to why the
program is needed or made. We have made various assumptions about the program itself
QCAF
14
as to whether the assumptions are valid is something that we plan on taking a look at as
we move farther along in the evaluation process. An assumption that is made is that the
program is effective based on prior information from assessments that the office gave to
its participants. The assessment was comprised of four questions, each centering on the
undefined notion of community. By utilizing this information and integrating it into our
evaluation approach, we are automatically making the assumption that community is a
desired outcome, and an attainable one, when there have not been set parameters for it in
past evaluations. Every month Q caf continues to gain more student attendance and
because of this we assume that the students enjoy it and therefore value it. By also not
questioning the growth of Q Caf and its implications, we are assuming that the growth
of the program is inherently positive for the longevity and nature of the space. We are
therefore not necessarily taking into consideration how artistic spaces may need more
intimacy and smaller numbers to create an atmosphere that is welcoming and safe to take
risks regarding personal identities.
We are also assuming that the communities of Rogers Park and Edgewater value
this program, and also want to be a part of it, based on information from informal
interviews with staff members. This assumption also includes the additional assumption
that the inclusion of community members in the space is beneficial to stakeholders,
without the evaluation or questioning of what a Loyola-only focused space could add or
detract from an experience.
QCAF
15
External Factors
When looking at the external factors there were not very many based on the
surface level information that were shared with us from the program coordinator. This is
something that we have revisited as we delved deeper into the inner constructs of the
program. Some external factors that we did find were based whether or not the Rogers
Park community was invested in this program and if they saw any real value in it or know
that it exists. We also saw some external factors relating to attendance, specifically in
winter months, as well as during peak times for student engagement.
Evaluation Approach
For our evaluation plan, we both feel strongly to utilize the formative approach,
defined by Wholey et. al., (2010) as being a way to improve a programs implementation.
The program is supported by the department and university and has existing assessment
data from participants that show it is valued and desired. Rather than evaluating if the
program should or should not exist, by focusing on formative assessment, we can
evaluate the best practices and ways to improve the program given all of its factors
(internal and external) the program can move to a place of perhaps even flourishing,
rather than just succeeding. While the program creates spaces and conversations
currently, its impact is focused and oftentimes limited to the occurrence and space that it
is presently in by creating a flourishing program, these conversations and spaces could
be recreated by participants as smaller, more intimate Q Caf sessions amongst their
peers, friends, etc., rather than only occurring at a set space and time.
QCAF
16
This approach will allow us to assess how to improve the experience for all
participants those affiliated, as well as those not affiliated, with Loyola University
Chicago. This evaluation process seems appropriate because the staff members of the
Department of Student Diversity and multicultural Affairs are interested in improving the
program, as well as monitoring student learning through ongoing feedback and
assessment data. Specifically, we want to ensure with the formative approach that we
gather information that can demonstrate if participants find value in the space, and what
that may or may not look like for them.
We will be able to gather information and data from the participants and staff
members, two primary sets of stakeholders that have direct interaction with the program.
While the program is small by some standards, having approximately 45 different
participants over the course of an academic year, we will be able to gather the narratives
of the participants and ensure a quantitative survey to construct overarching themes and
messages, which can then help inform best practices for the program.
By engaging directly with the stakeholders, we are able to gather personal
feedback and testimony from virtually all of the direct participants. Since this program is
so multifaceted and has such a variety of different stakeholders, utilizing multiple
formative assessment approaches allows for the most voices to be heard and the clearest
picture to emerge.
Evaluation Questions
Revisiting our conceptual framework the evaluative questions that will guide this
project are whether Q Caf is creating a space for LGBTQIA+ individuals to find
community; whether Q Cafe is a space that embraces identities through various forms of
QCAF
17
art and expression; and that whether it is able to generate conversations about
LGBTQIA+ topics (i.e. identity, politics, art, etc.) among its participants.
Evaluation Standards
We also want to ensure that when we are able to discern data that is significant to
our study and findings that it only shows valid results, and does not suggest causation for
unrelated variables. With a smaller sample size (again, approximately 45 participants),
ensuring our findings are statistically significant and not fluke coincidences will be
crucial to ensure that we only provide information rather than claiming causation.
QUANTITATIVE APPROACH
Survey Population
The survey will be administered to all those that participate and attend Q Caf
sessions throughout the academic year. Administrators of this survey will be the graduate
assistant and the program coordinator of the Department of Student Diversity and
Multicultural Affairs that lead the LGBTQIA+ initiatives. The sampling that we will be
using will be a census sampling method. Since there are about 45 participants that attend
Q Caf throughout the year its manageable to outreach to these folks and receive
feedback.
Q Caf is a program that is open to the general public. The survey population will
be gathered from sign in sheets that are passed around by facilitators at the end of the
each Q Caf session. Q Cafe attendees may include (but are not limited to) Rogers Park
and Edgewater community members, Loyola student participants, Loyola staff and
faculty participants, and participants friends and supporters.
QCAF
18
QCAF
19
Evaluation Design
The survey design that we will be working with will be a variation of quasiexperimental design that incorporates a cross sectional approach. A quasi-experimental
design is a type of evaluation, which aims to determine whether a program has the effect
that the facilitators were hoping to achieve (Wholey et. al., 2010).
Acknowledging the fact that quasi-experimental and cross sectional is mutually an
exclusive approach we know that this study is more looking towards a pre-experimental,
cross-sectional design. We believe a cross-sectional approach would be beneficial in that
it is also an observational study, but would allow us to view numerous factors at once and
how they correlate to specific outcomes (Wholey et. al, 2010). This will allow us to gain
more perspective not only regarding the programs effectiveness, but to see how it affects
certain identities in the interpretation of the data. Since this is the case, we would not be
using a randomized sample.
With this design, we hope to be able to gather information on the following three
goals: whether Q Caf is creating a space for LGBTQIA+ individuals to find community;
whether Q Cafe is a space that embraces identities through various forms of art and
expression; and whether it is able to generate conversations about LGBTQIA+ topics (i.e.
identity, politics, art, etc.) among its participants. These goals have never been assessed
to interpret whether the intended outcomes were being met. Our hope is that with this
tool we are able to gain more insight into the program.
Survey Timeline
The survey web link will be sent via email from the graduate assistant who runs
the program, Patrick Fina, following the final session of Q Cafe in April 2015. Reminder
QCAF
20
emails will be sent once a week for two weeks, at the end of which the survey will close,
14 days from when it was first launched.
Response Rate
We hope for a 50% response rate, given that many of the participants are repeat
attendees that have previously engaged in the program. This participation and high buyin of the participants to create a student-driven space that gives their experience voice can
arguably solicit a significant amount of feedback, as they have been invested in the
program as stakeholders from its inception.
Incentives
In a given year there are about 45 participants that attend Q Caf throughout the
year because of this we would be sufficiently content with 20-25 responses to the survey.
. As an incentive the participants will also all be entered into a drawing to win a $25 gift
card to Metropolis Coffee, where Q Cafe has taken place. Participants will need to
provide their email address within the survey (Appendix B) to be entered for the drawing.
This incentive would also further the relationship between Loyola University Chicago
and Metropolis Coffee by continuing patronage and support of the local business.
Survey Instrument
The survey instrument has a total of eighteen questions (Appendix B). The
majority of questions include Likert scales to gather information from participants about
their experience in the space. For instance, questions explore their experience with the
overall satisfaction of the program to whether or not this program fosters a sense of
community. The five demographic based questions utilize a response where multiple
QCAF
21
check boxes can be included. The options also include an other option where someone
can enter their own definition of their identity and/or perception. We wanted to create a
survey that honored the individual identities of Q Cafe participants, but still allowed for
data gathering and interpretation to happen in a smooth process. By providing some predetermined options, the coding and interpretation of data commands less clean-up work
on our part as researchers while allowing the participants to voice their individual
identities if they fall outside of the pre-determined categories.. The predetermined
options that would more than likely account for the majority of identities individuals
possess also makes the survey easier and faster to fill out. Our hope is that no one will
need to spend more than ten minutes on this quantitative aspect of the program
evaluation. This data, once collected, will be shared with Patrick Fina and the Program
Coordinator for LGBTQIA+ initiatives, Miguel Macias, who are the two individuals who
are most connected to the program in a staff capacity.
We do not believe the use of a control group would be necessary. A control group
would not be beneficial for this evaluation because we are not interested in collecting
data from individuals who do not attend Q Cafe. We have used the survey construct map
(Appendix C) as a way to ensure that the overarching goals of Q Cafe, as well as
demographic information that is pertinent to the concept of a space that focused on
aspects of identity are evaluated and assessed to an extent that does not overwhelm the
participant with an onslaught of questions. We also want the survey to gather a
significant amount of information to best understand where the program is succeeding,
and where it can be improved. These topics were specifically considered in the first
QCAF
22
several questions in the online survey form, and were specifically tied in the survey
construct map (Appendix C) with the three different goals, which were coded as A, B,
and C.
Pilot Testing
Pilot testing is used to uncover problematic elements in a survey in order to have
time to make any necessary adjustments before the implementation phase (Schuh, 2009).
We will be conducting a pilot test with administrators in the Student Diversity and
Multicultural Affairs office to ensure that the questions that we are asking in the survey as
not at all confusing or open to interpretational bias. This will be the first time that any
type of survey instrument of this nature will be used so it is important to find questions in
the survey that need to be altered. A pilot test will help make the survey as effective as
possible before sending it out to its respondents.
Statistical Analysis
Through this evaluative process we feel that the best way to look at our data and
make meaning of it is to use some simple data tables to compare means across different
populations. While anecdotal information alludes to a variety of identities feeling that Q
Cafe is a space for them, being able to explore multiple letters of the LGBTQIA+
acronym, as well as the multiplicity of other identities within the community, will help
paint a clearer picture as to what the respondents experiences are and if there is a
significance in different identities and their experience in the space.
To answer our evaluation questions from the survey we plan on using a
descriptive approach. Wholey et. al., defines descriptive statistics, when any
QCAF
23
QCAF
24
threats to significance since the sample size is so small, our final report could be changed
radically to reflect the data that was able to be gathered.
QUALITATIVE APPROACH
Qualitative Approach Rationale
After collecting the quantitative data from the survey, we plan on utilizing a focus
group approach to gain a richer understanding of participants experiences of Q Caf.
According to Schuh (2009), focus groups are best utilized when looking to uncover
factors that influence opinions, behaviors, or motivation (p. 69). A focus group would
be beneficial to ensure a number of other factors as well. Wholey et. al., (2010), noted
they are also valuable to gather information in the design phases of programs and
evaluations. For purposes of this evaluative study a focus group would help in evaluating
policy options in terms of how the program is structured or if respondents would be open
to changes to the program itself based on the data that will be collected.
The quantitative survey serves as the first step in gaining insight, but is limiting in
terms of understanding perceptions or opinions about the program. Wholey et. al. (2010),
suggest a focus group would help in understanding the quantitative findings from the
survey that would be sent out earlier to participants. For example, even though
participants might say that they would recommend this program to a friend it would help
in understanding what factors went into making that decision. This would be an instance
where the focus group would be used to expand on responses from the quantitative study.
This approach will address parts of the quantitative qualitative survey that we
would like to build upon such as the effect that art plays into the program and how
satisfied participants were with the structure of the program. This instrument will also
QCAF
25
serve in delving deeper into certain topic areas that might have not been clear in the
survey instrument. These topics of interest are included in the focus group protocol and
address the following topics (See Appendix D): whether or not the program fosters a
strong sense of community; the overall satisfaction of the program; and potential
improvements that can be implemented.
The qualitative data is going to focus more on process components related to the
art and satisfaction with the process. Most of the questions will deal with how
participants perceived the program after attending various sessions throughout the year.
By collecting this information we want to understand participants current view or
understanding of the purpose of Q Caf and hopefully by incorporating both the findings
from the quantitative and qualitative approach we can infer what steps to take to move
forward in improving Q Caf in the future.
Focus Group Participation
Ideally, we would like to have the entire population that attends Q Caf to
participate in the focus group process. Anyone who provides interest in attending the
focus group through the quantitative survey will be contacted through a series of emails
(see Appendix F). The initial email will be sent out by facilitators and ask for participants
availability. Based on the respondents availability, a date and time will then be set and
sent to the respondents and they will be sent a confirmation email. We would also send a
final email to respondents that were not able to be a part of the study due to conflicts with
the scheduled time. We believe this final email would be important as a common courtesy
QCAF
26
for those individuals who outreached and wanted to help in our study and show our
appreciation and thanks for being interested.
Schuh (2009) notes maximum variation samplingcommonly use to capture
general themes or patterns when a variety of experiences or opinions exists (p. 90). For
the purposes of this qualitative instrument we would employ a maximum variation
method in that we are looking to have both Loyola University Chicago members and
community members be a part of the focus group itself. In the event that we receive the
50 percent response rate that we were looking for which in this case would be 20 to 25
participants there are a number of factors to consider of who would be chosen to be in the
focus group. There are a number of factors that would need to be taken into
consideration. We would have to consider that some of these respondents who are
interested in joining a focus group might not be available during the actual
implementation of the focus group. In the event that we have approximately 25
participants we would then implement two focus groups. Each of these groups would
have 10-12 participants and we would then try to apply that maximum variation sample.
The hope is that we would be able to gain a deeper understanding of how they view the
program in their own points of view according to the community that they belong to. Of
course, this is all dependent on the number of participants who express interest in
attending the sessions.
Focus Group Protocol
The Q Caf focus group protocol will be used to collect data that is important in
understanding respondents overall experience (see Appendix D). The protocol gives the
QCAF
27
evaluator an overall look as to what they are trying to accomplish and what topics will be
covered throughout the hour and half session. The focus group protocol will be broken
up into 4 sections: welcome/introductions; framing the focus group experience;
disbursement of consent forms (see Appendix E); and explanation of purpose and intent
of the Q Caf program (see Appendix D). Participants will be informed that their
experience in the group will be completely confidential and only shared with staff
members of the Department of Student Diversity and Multicultural Affairs. The
participants will be asked to introduce themselves and answer questions that revolve
around three topic areas: general reactions to the program, sense of community, and
overall satisfaction with the program.
To gain insight as to how long the focus group will take and whether or not the
questions would need to be modified a pilot test of the qualitative study will be
implemented. We will use a mock group of participants from those that participated in
the program last year. Facilitators would outreach to these students and will provide
them with lunch. This will give us adequate time to adjust the questions if need be and
get a sense of how long the focus group would take. This is also dependent on the
number of participants who attend the final focus group and that is going to determine
how long the instrument will truly last.
Focus Group Implementation
This focus group would ideally occur in late April, after the final Q Caf session.
Two facilitators would be utilized one to lead the discussion, and one to take notes.
Having the folks who coordinate Q Caf removed from the space can also arguably create
QCAF
28
QCAF
29
The evaluators before entering the focus group constructed their own construct
map that developed a series of different priori codes and coding rubric these constructs
include: building community (COMM); programmatic improvements (IMPROVE);
overall satisfaction (SATIS); expression of art (ART) as described in Appendix G. These
are the predetermined codes that the evaluators have come up with, but we also have to
be cognizant that as the hour and a half session goes on there might be new ideas that
need to be addressed or added to this construct map. Even though there will be two
facilitators throughout the focus group each one will have their own designated role in the
process as one facilitates the other will be transcribing, but they would work together to
interpret the codes. Member checking by using the general themes collected in the focus
groups, we would share back with the participants what our large perceptions and
overarching themes were. We would also let them know that we would be open to
sharing the writing version of our notes and findings before the final result is given to the
Department of Student Diversity and Multicultural Affairs.
Wholey et. al., (2010), states strategies to overcome the potential bias that can
arise from the use of a single method, single data source, single observer, or single
theoretical base. (p. 446). The data will be triangulated with the initial quantitative
survey instrument in order to gain a richer understanding of their experience, but not
solely relying on one set of information and would be used to see where the similarities
and difference lie in each of the approaches.
QCAF
30
Limitations
Limitations to this survey include its timeline, as well as its number of
participants. With such a short time frame for the event finishing, surveys being started,
and focus groups (marketed as conversation circles to potential participants) there are
several moving parts that need to be aligned perfectly. As the potential entire population
of the program is about fifty people, even a fifty percent interest in participating in a
focus group would have us at a maximum of three focus groups, if we include
approximately eight people in each. As the last Q Caf occurs in April, and within a
month the school year is over and many of the participants leave the Chicago area, time is
a definite limiting factor in this study.
Results Presentation
For our final report presentation, we are hoping to use a few significant direct
quotes, but also hope to utilize tables that show the themes and codes that were captured
by the note taker during the focus group. The direct quotes will tie in and relate directly
to the tables that show the codes and themes (as well as the amount of times that a
potential snippet was coded into that code) to display significant overarching themes and
take-aways.
Budget
For the purposes of this evaluative study we have set a budget of $240 for the
year. The budget breakdown can be seen in Appendix H. We plan on setting aside $75
towards incentives that we plan on distributing to participants who plan on taking part in
the quantitative and qualitative survey. Then we allocated $150 towards food that will be
QCAF
31
used to feed our focus group participants. The remainder of the money will be used for
paper to print out copies of our protocol instrument and consent forms for the focus group
implementation.
Timeline
For the purposes of the study we have set a timeline (see Appendix I) that will
span over the months of November to May of 2015. The timeline will begin with
implementing the design tool and pilot testing it by November. December we will start
move towards finalizing the quantitative instrument and making the focus group protocol.
We would like to allot ourselves enough time to make sure that the questions being asked
make sense and will be beneficial towards the study. January to March will be used to
purchase incentives and start reserving space for the actually focus groups come April.
April will be the month where most of the implementation of the quantitative and
qualitative parts of the study will take place. Finally, May will be dedicated solely to just
analyzing the data and making next steps.
Next Steps
Towards the end of our evaluation process we will begin taking into consideration
everything that we have found throughout the study. Since the focus groups would take
place in the month of April we would use the month of May to analyze and put the final
report together. Based on our findings in both the quantitative and qualitative study we
would then infer and make appropriate suggestions to the program coordinator of the
Department of Student Diversity and Multicultural Affairs of what can be implemented or
changed if need be to make it a better experience for its participants.
QCAF
32
References
Loyola University Chicago - Student Diversity & Multicultural Affairs. (n.d.). Retrieved
December 1, 2014, from
http://www.luc.edu/diversity/programs/lgbtqia/safespaceworkshops/
Loyola University Chicago - Student Diversity & Multicultural Affairs. (n.d.). Retrieved
December 1, 2014, from http://www.luc.edu/diversity/programs/lgbtqia/qinitiatives/
Loyola university chicago petitioned to allow gay weddings on campus. (2013,
November). Huffington Post. Retrieved from
http://www.huffingtonpost.com/2013/11/13/loyola-university-gay-weddings_n_4269328.html
Rankin, S., Weber, G. N., Blumenfeld, W. J., & Frazer, S. (2010). 2010 state of higher
education for lesbian, gay, bisexual & transgender people.
Schuh, J.H. & Associates (2009). Assessment methods for student affairs. San Francisco,
CA: Jossey-Bass
Wholey, J.S., Hatry, H.P., & Newcomer, K.E. (Eds.) (2010). Handbook of practical
program evaluation (Third Edition). Jossey-Bass: San Francisco.
QCAF
33
APPENDICES
Outcomes
Outputs
Inputs
Activities
Participation
Short
Medium
QCAF
-Money/Budget
34
-Engaging in student
check ins.
-Teambuilding
-Community Building
-Identity/Art
Performers
-Speakers
-Staff
-Students
-Staff
-Faculty
-Venue (Metropolis)
-Structured Community
time once a month.
-For students
confident and
comfortable in
space.
-Making them
valued within
Loyola comm
-Faculty
-Metropolis/Venue
-Making them
the have a voi
space provide
-Students
-Voice
-Food (Pastries/Coffee)
Assumptions
Assume that the program is effective.
Assume that students enjoy/want it/need it/value it
Assume that students feel supported in the space.
Assume that students want to be there and engage in the program
Assume that Metropolis Coffee enjoys hosting this monthly event.
12/1/2014
External Factors
Weather might prohibit students
Attendance can be an issue.
Whether or not the larger Rogers
occur.
Q Cafe Feedback
This information will help us best understand if the current time, day, and structure of Q Cafe is
effective and where it can be improved.
Appendix
B: Q Caf Google Forms Survey
How many Q Cafes did you attend
this year? *
If you don't remember exactly, an estimate is sufficient.
https://docs.google.com/forms/d/1ZBiH5OwKP0NFT1ZrqI9ol2CnTIObUtdm5q6uSXrik
5M/viewform
I think the structure of Q Cafe was well planned. *
The time of the Q Cafe works well for my schedule. *
If you have any additional comments regarding the structure and planning of Q Cafe, please
enter it below.
Please provide your email address if you would like to be considered as a part of a focus group
to help us better understand how to improve Q Cafe.
There will be a drawing to receive a $50 gift card to Metropolis Coffee for focus group
participants. Your email will not be shared with anyone, and the subject line will not include
LGBTQI+related words or phrases.
Demographic Information
This information will help us understand who attended our Q Cafes and your individual
identities in regards to your experience. Q Cafe intends to be a space that values and embraces
all facets of identity.
https://docs.google.com/forms/d/1ZBiH5OwKP0NFT1ZrqI9ol2CnTIObUtdm5q6uSXrik5M/viewform
2/5
QCAF
12/1/2014
35
Q Cafe Feedback
Editthisform
QCafe Feedback
This survey is designed to help us deliver the most engaging and effective Q Cafe programs
possible. Your honest feedback will help us ensure that Q Cafe is meeting its goals and
aspirations to be a creative space for LGBTQI folks from the Loyola University Chicago,
Edgewater, and Rogers Park communities.
LGBTQIA+is an acronym that attempts to include all facets of the broader Lesbian, Gay,
Bisexual, Transgender/Trans*, Queer/Questioning, Intersex, Asexual, etc. community. We
recognize this acronym is imperfect and leaves out many identities. Please use the space in
this form to articulate and express your identities in their truest forms.
* Required
Logistics of Q Cafe
https://docs.google.com/forms/d/1ZBiH5OwKP0NFT1ZrqI9ol2CnTIObUtdm5q6uSXrik5M/viewform
1/5
QCAF
36
12/1/2014
Q Cafe Feedback
If you are unfamiliar with any of the words listed below, broad definitions can be found on this
12/1/2014
easy-to-navigate website from the University of Michigan.
http://internationalspectrum.umich.edu/life/definitions
What area do you currently live in? *
Q Cafe Feedback
Rogers Park
Edgewater
Please select as few or as many that fully validate your identity. If your identity is not
Other:
articulated accurately below, please enter it in the 'other' option on the bottom of this question.
Transgender woman / Transwoman
Transgender man / Transman
Genderqueer
No, I am not
Agender
Yes, I am a student
Cisgender man
Cisgender woman
Gender non-conforming
Yes, I am an alumni
Questioning
Other:
Other:
Queer
Website
Lesbian
Bisexual
Poster
Gay
Word of Mouth
Pansexual
Other:
Asexual
Questioning
Please provide your email address if you would like to be entered into a drawing for a $25
Metropolis Coffee Gift Card.
Other:
For filling out this survey, we would like to extend our thanks and offer you a chance in our giveaway!
Please select as few or as many that fully validate your identity. If your identity is not
articulated accurately below, please enter it in the 'other' option on the bottom of this question.
Submit
NeversubmitpasswordsthroughGoogleForms.
Poweredby
Other:
ThiscontentisneithercreatednorendorsedbyGoogle.
ReportAbuseTermsofServiceAdditionalTerms
https://docs.google.com/forms/d/1ZBiH5OwKP0NFT1ZrqI9ol2CnTIObUtdm5q6uSXrik5M/viewform
3/5
https://docs.google.com/forms/d/1ZBiH5OwKP0NFT1ZrqI9ol2CnTIObUtdm5q6uSXrik5M/viewform
4/5
QCAF
37
Goal/Purpose
A
Question #
1
A/P
A/P
A/D/P
A/D/P
9
10
11
A/B/C/D/P
12
D
D
D
D
D
13
14
15
16
17
D/P
A/B/C/D/P
18
19
Type of Question
Likert Scale
Likert Scale
Likert Scale
Likert Scale
Drop down menu
Likert Scale
Likert Scale
Likert Scale
Likert Scale
Likert Scale
Paragraph text
submission
Text submission
Checkboxes
Checkboxes
Checkboxes
Checkboxes
Checkboxes
Checkboxes
Text submission
QCAF
38
Purposeoffocusgroup:
1. Toassesssatisfactioninthe
implementationoftheprogram.
2. Tofindoutwhatimprovementscan
bedonetotheprogramtoenhance
theparticipantsexperience.
3. Whetherornottheprogramfosters
asenseofcommunity.
Wearelookingtogainperspectivetosee
whetherornottheseintendedgoalsare
actuallycomingtofruition.
FocusGroupScript
1. WelcomeandIntroductions:
Thankyouforjoiningustoday!Mynameis(insertnamehere)andIwillbe
facilitatingourconversationtoday.WeareexcitedtogainfeedbackaroundQ
Caf,andlearnwhatwecandotoenhanceyourexperienceintheprogram.
2. Framingtheexperience:
Wewillbespendingthenexthourandahalftogethertotalkaboutyour
experiencewiththeprogram.Throughoutthefocusgrouppleasefeelfreetoeat
yourfoodwejustaskthatwetryandstayfocused.Wewantthisexperiencetobe
veryconversationalsopleasefeelfreetobehonestandcandidwithyour
responsesasthiswillhelpinmakingQCafabetterexperienceforyouall.
3. Passoutconsentformsandspeakaboutaudiorecording:
Yourinformationwillbekeptcompletelyconfidentialandanythingthatisshared
inthisroomwillbepresentedtostaffmemberswithoutanyidentifying
informationassociatedwithit.Wealsowanttoaskallofyoutoholdeachothers
QCAF
39
experiencesandstoriesconfidentialaswell.Whatissharedinourconversations
todaywillremainhere.Ifyouhaveadditionalquestionsregardingconfidentiality,
wearehappytoanswerthosequestionsatthispoint.
Wewillnowbepassingouttheconsentformpleasetakeyourtimeandreaditon
yourownandsignifyouagreetothetermsinquestion.
Bysigningthisformyouwillallowustousedirectquotesandsummariesofyour
experiencestoimprovecurrentprogrammingwithinQCaf.Ifyouare
uncomfortablesigning,wecompletelyunderstand[Waitandseeifany
participantswouldliketoleave.Iftheydecidednottoparticipateinthestudy
pleasethankthemforcomingandexcusethemfromthespace].
Arethereanyquestionsbeforewebegin?(seeconsentform)
4. ExplainPurposeandIntentoftheprogram:
QCafhasthreegoals:1.tocreateaspaceforLGBTQIA+folkstofind
community;2.toembraceidentitiesthroughvariousformsofartandexpression;
3.togenerateconversationsaboutLGBTQIA+topics(i.e.identity,politics,art,
etc).Wearelookingtogainperspectivetoseewhetherornottheseintendedgoals
areactuallycomingtofruition.
5. Startthefocusgroup
Withthatbeingsaiditstimetobegin!Letsgoaroundandintroduceourselves.
Pleasestateyourname,youraffiliationwithLoyolaornotandwhat
neighborhoodyouresidein.
Questions
GeneralReactionsofProgram:Wearegoingtobeingtodaysconversationbystarting
withsomequestionsaboutyourgeneralreactionsandperceptionsofQCaf.
1. WhatsparkedyourinterestinattendingQCaf?
a. Probe:WhatdidyouexpecttolearnorgainfromattendingQCaf?
2. PleasetellusaboutyouroverallexperienceofQCafe.
a. Probe:WhataspectsofQCafdidyoulike?
b. Probe:Whataresomethingsthatyoudidntlike?
c. Probe:What,ifanything,didyoufindvaluableabouttheprogram?
d. Probe:What,ifany,oftheguestperformanceswerememorable?Why?
e. Probe:Didyouperform?Ifnot,canyoutellmewhynot?
QCAF
40
3. TellmeaboutthemostmemorableperformancesthatyourememberfromQ
Caf?(Makethisanopening)
a. Probe:Whataboutthoseperformanceswasmemorable?
b. Probe:Didyouperform?Ifnot,canyoutellmewhynot?
SenseofCommunity:ThankyouforsharingyourgeneralthoughtsonQCaf.Wenext
wanttotalktoyouaboutthetopicofcommunity,specificallywithintheQCafsetting.
1. Inwhatways,ifany,didQCafhelpyoutodevelopnewfriendships?
a. Probe:Whatwasyourexperiencelikeintermsofinteractingwith
differentpeoplewhoattendedQCaf?
2. WhattypesofconversationsdidyouhavewithotherpeoplewhoattendedQ
Caf?
a. Probe:Didtheyoccurinthespace?Outofthespace?
b. Probe:Towhatextent,ifany,didyourconversationscenteraround
identity?
SatisfactionwiththeProgram:Thankyouforsharingwithusyourexperiencesaround
findingcommunity,friendships,andrelationshipsinQCaf.Atthispointwewouldlike
toaskyouaboutyoursatisfactionwiththeprogram.
1. Overall,howsatisfiedwereyouwithQCafe?
a. Probe:Why/Whynot?
2. What,ifanything,canbeimprovedaboutQCaf?
a. Probe:Whataspectsoftheprogramdoyoufeelyoumightwanttosee
morestructurein?
b. Probe:Doyoubelieveitshouldberestructuredcompletely?
c. Probe:Whatshouldstaythesame?
3. WhatdidyouthinkabouttheprogramsreachtobothLoyolaandnonLoyola
affiliatedfolks?
a. Probe:Howdoyoufeelaboutcommunitymembers/faculty/staffbeing
partoftheprogram
4. Wouldyourecommendthisprogramtoafriend?
a. Probe:Whyorwhynot?
5. DoyoulikeattendingtheprogramatMetropolisCoffee?
a. Probe:Wouldyouliketoseethishostedsomewhereelse?
QCAF
41
Conclusion:Thankyouonceagainforsharingyourtimewithus.WeknowQCafthat
youarealltheexpertswhenitcomestowhetherornottheprogramismeaningfuland
effective.Itisveryimportanttoustohearyourhonestfeedbacksowecancontinually
improvetheprogram.Ifyouwouldliketostayandreviewwithuswhatwefoundthe
overallthemestobe,youaremorethanwelcometodosowecanalsoemailyouacopy
ofawrittenreport,ifyouwouldprefertoseeadraftoftheinformationthatwillbe
sharedwiththeprogramcoordinators.Youallarenowfreetogo!
QCAF
42
Ifyouagreetoparticipateinthisfocusgroupknowthatitwilllastapproximatelyanhour
andahalf.Youwillbeaskedaseriesofquestionsthataskyoutoreflectonyour
experienceswithintheprogram.Knowthatwehopethatyouarecontinuallyanswering
truthfullyandcandidlythroughtheprocess.Ifatanytimeyoufeeluncomfortable
answeringaquestionpleaseknowthatyoucanchoosetorefusetoansweratanytime.
However,wedoexpectyouactivelyparticipateasmuchaspossibleaslongasyoufeel
comfortable.Thisfocusgroupiscompletelyvoluntaryandifyouchoosetowithdraw
yourparticipationatanypointfeelfreetoletthefacilitatorknowandtheycanexcuse
yououtoftheroom.
Attheendofthisfocusgroupyouwillallbeenteredforadrawingtowinoneoftwo$25
giftcardsforMetropolisCoffee.Thewinnerswillbeemaileddirectly.
Confidentiality/Risks:
Thisfocusgroupwillbecompletelyanonymousandeverythingthatyousaytodaywill
nothaveyournameattributedtoit.Wehopethatbytakingthesestepsitminimizesany
riskstowardsrevealingyouridentity.Theinformationwillbecompletelyconfidential
andsharedwithstaffmembersfromtheDepartmentofStudentDiversityand
MulticulturalAffairs(SDMA)atLoyolaUniversityChicago.Weaskallparticipants
respectoneanothersconfidentialityandhopethatanythingthatissharedinthisstaysin
theroom.
ContactsandQuestions:
Ifyouhaveanyquestionsasyouprogressthroughthefocusgrouppleaseaskyour
facilitator,youmayalsocontactMiguelMacias,programcoordinatorforSDMAat
mmacias@luc.edu.
StatementofConsent:
Yoursignaturebelowindicatesthatyoutookthetimetoreadthisformandunderstand
thefullscopeofwhatthisfocusgroupwillentail.Wewillprovideyouwithacopyof
thisconsentformaswelltokeepforyourrecords.
ParticipantSignature:______________________
Date:___________________
EvaluatorsSignature:______________________
Date:___________________
QCAF
43
QCAF
44
(NAME)
Email #2: Confirming Attendance
Hello, and thank you for sharing your availability to participate in a feedback circle!
Your availability matches perfectly with one of our feedback circles, which will take
place from TIME to TIME on DAY, DATE, in LOCATION.
Please respond to this message to confirm that this time still works well for you.
We thank you so much for being a part of the process of creating a better Q Caf! As our
thanks, we will be offering FOOD ITEMS HERE AND DRINKS TOO during this
feedback circle, and will also be entering you to win one of two $25 gift cards to
Metropolis Coffee! If you win, we will contact you directly via this email address.
Best,
(NAME)
Email #3: Informing of Inability to Schedule
Hello, and thank you for sharing your availability to participate in a feedback circle.
Unfortunately, we were unable to find a time that fits your availability.
While we are unable to offer you a chance to be in our feedback circle, we do invite you
to submit any feedback or insights you feel are important regarding Q Caf to us at this
email address. We will do all in our power to keep messages anonymous and confidential.
Best,
(NAME)
QCAF
ConstructCode(s)
45
COMM
Meaning
BuildingCommunity
Definitions
QCafdidordidnotprovidea
spaceforindividualstobuild
communityamongsttheLoyola
communityandthelargerRogers
Park,Edgewatercommunities
IMPROVE
ProgrammaticImprovements
Participantssuggested
improvementsforQCafanywhere
fromprogrammaticsetuptovenue.
SATIS
OverallSatisfaction
ART
ExpressionofArt
Participantsexpressedtheiroverall
satisfactionwiththeprogramand
whetherornottheywouldcontinue
coming.
QCafdidordidnothavean
opportunitytoexpressthemselves
througharti.e.playinginstruments,
performingorhadanopportunityto
viewtheguestperformers.
QCAF
46
Appendix H: Budget
QCAF
47
Appendix I: Timeline
QCAF
48
QCAF
49
QCAF
50
QCAF
51
QCAF
52
QCAF
53
QCAF
54
QCAF
55
QCAF
56
QCAF
57
QCAF
58
QCAF
59