You are on page 1of 29

2021 Mentoring Program Evaluation

Marialice Mastronardi – STEM Education Doctoral student


Supervisor: Prof. Maura Borrego

Data Analysis:

Data have been separated in Mentors responses and Incoming PhD responses.
Pre- and post- survey questions have been kept consistent. A separate file with pre-survey questions is
attached.
The survey includes single items designed specifically for the evaluation of this mentoring program and
constructs of multiple items (factors: research competence, research interest, engineering interest,
sense of belonging and departmental climate) adapted from validated published scales. Items comprised
in each factor are shown in the table below.

Factors Survey Items


† Reverse coded
(1=Strongly Disagree/Not at All
Competent; 5=Strongly
Agree/Very Competent)
Research Interest I am interested in my research topic
(To what extent do you disagree I am interested in learning more about research
or agree with the following I enjoy research activities as part of my work week
statements?) In general, I find working on research interesting
I like doing research
Research Understanding and applying scientific and mathematical
Performance/Competence relationships based on the conditions
(How competent are you…) Applying math and science concepts to make new systems/models
Using calculations and equations to evaluate things
Understanding derivations and equations in journal papers
Understanding current research findings by using sufficient math,
science or engineering knowledge
Engineering Interest I think engineering is fun
(To what extent do you disagree I think engineering is interesting
or agree with the following I like to figure out how things work
statements) I feel good when I am doing engineering
I am interested in learning more about engineering
I enjoy engineering activities as part of my work week
I like doing engineering
Valued and Belonging I feel valued as an individual
(To what extent do you disagree I feel I belong
or agree with the following I have considered leaving because I felt isolated or unwelcome†
statements) I am treated with respect
I feel others don’t value my opinions†
I have found one or more communities or groups where I feel I
belong

1
Thriving & Growth My department is a place where I am able to perform up to my full
(To what extent do you disagree potential
or agree with the following I have opportunities for academic success that are similar to those of
statements) my peers
I have to work harder than others to be valued equally†
My experience has had a positive influence on my academic growth
The following analyses have been performed:
- Mentors comparison of pre- and post- surveys
- Incoming PhD students comparisons as follow:
o All pre- and post- comparison
o Pre- and post- comparison for students self-identified as other than male
o Pre- and post- comparison for students self-identified as not white and/or not Asian
o Post- only comparison over gender
o Post- only comparison over race/ethnicity
o Pre- and post- comparison of BME students
o Pre- and post- comparison of ECE students
For these analyses, the mean of pre- and post- survey responses are compared using the Mann-Whitney
test and effect size is reported. The Mann-Whitney test has been chosen to account for sample’s
variation between pre- and post-surveys. Statistical significance is reported.
In order to protect the identity of the respondents, students were asked two questions to build a code
that would allow to identify matching respondents for pre- and post- survey. Additional analyses
(reported as Matching in the tables) include:
- Incoming PhD students
o Pre- and post- comparison of matching sample
o Pre- and post- comparison of matching sample for students self-identified as other than
male
o Pre- and post- comparison of matching sample for students self-identified as not white
and/or not Asian
o Pre- and post- comparison of matching sample of BME students
o Pre- and post- comparison of matching sample of ECE students

For this analyses, we included only responses of pre- and post- surveys from the same individuals. The
mean of pre- and post- survey responses are compared using the Wilcoxon matched-pairs signed-ranks
test and effect size is reported. Statistical significance is reported.
Enough responses to perform separate analysis available only for BME and ECE.

2
Mentors: comparison of mentors pre- and post-survey responses

The mean of pre- and post- survey responses are compared using the Mann-Whitney test and effect size
is reported. The Mann-Whitney test has been chosen to account for sample’s variation between pre-
and post-surveys. Statistical significance is reported.

To what extent do you disagree or agree with the Pre Post Effect Size
following statements (26) (30)
(Peer mentors)
Factors
Research Competence 4.17 4.12 0.04
Research Interest 4.57 4.47** 0.35
Engineering Interest 4.54 4.33** 0.39
Valued & Belonging 4.01 3.86 0.13
Thriving & Growth 3.88 3.81 0.08
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01

3
Incoming PhD: comparison of Incoming PhD pre- and post-survey responses
The mean of pre- and post- survey responses are compared using the Mann-Whitney test and effect size
is reported. The Mann-Whitney test has been chosen to account for sample’s variation between pre-
and post-surveys. Statistical significance is reported.

To what extent do you disagree or agree with the following Pre Post Effect
statements (64) (36) Size
(Incoming PhD students)
I am well-informed about the advisor search process 3.38 3.94** 0.29
I know where to focus my effort during my first year in graduate 3.14 3.47 0.13
school
I know what to expect for my first year in graduate school 3.14 3.57 0.19
I know engineering graduate students who have successfully 3.91 4.33* 0.25
completed their first year of graduate school
I am excited about the research I will do at UT Austin 4.41 4.31 0.04
I received high-quality peer mentoring this summer (post only) 3.64
I received high-quality faculty mentoring this summer (post only) 3.36
Likert scale: 1= Strongly disagree, 5=Strongly Agree
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01

How competent are you… Pre Post Effect


(Incoming PhD students) (64) (36) Size
Understanding what is expected of me in graduate school 3.65 3.81 0.06
Finding a stimulating research group 3.53 3.97* 0.23
Doing good research at UT Austin 3.52 3.86 0.20
Settling into life in a new city 3.69 4.11* 0.21
Finding resources for living in Austin 3.89 4.06 0.04
Making friends or finding community in Austin, beyond the university 3.06 3.50 0.19
Finding help if struggling academically 3.33 3.33 0.01
Finding help if struggling with stress, mental health, or wellbeing 3.39 3.31 0.05
Performing well in graduate-level courses 3.89 4.06 0.13
Likert scale: 1= Not at all competent, 5=Very competent
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (64) (36) Size
Factors
Research Competence 4.02 3.96 0.05
Research Interest 4.39 4.46 0.07
Engineering Interest 4.47 4.59 0.18
Valued & Belonging 3.82 4.01 0.11
Thriving & Growth 3.66 3.91 0.14
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01

4
Incoming PhD from minoritized backgrounds:
Comparison of pre- and post- survey responses of Incoming PhD students who self-identified other
than white or Asian
The mean of pre- and post- survey responses are compared using the Mann-Whitney test and effect size
is reported. The Mann-Whitney test has been chosen to account for sample’s variation between pre-
and post-surveys. Statistical significance is reported.

To what extent do you disagree or agree with the following Pre Post Effect
statements (17) (10) Size
(Incoming PhD students)
I am well-informed about the advisor search process 3.00 3.70 0.34
I know where to focus my effort during my first year in graduate 2.94 3.90* 0.44
school
I know what to expect for my first year in graduate school 3.06 3.80 0.37
I know engineering graduate students who have successfully 3.65 4.50** 0.51
completed their first year of graduate school
I am excited about the research I will do at UT Austin 4.35 4.50 0.11
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Pre Post Effect


(Incoming PhD students) (17) (10) Size
Understanding what is expected of me in graduate school 3.76 4.00 0.10
Finding a stimulating research group 3.06 4.20** 0.52
Doing good research at UT Austin 3.65 3.90 0.17
Settling into life in a new city 3.94 4.10 0.13
Finding resources for living in Austin 4.12 4.30 0.12
Making friends or finding community in Austin, beyond the 3.12 3.60 0.21
university
Finding help if struggling academically 3.29 3.20 0.03
Finding help if struggling with stress, mental health, or wellbeing 3.24 3.60 0.17
Performing well in graduate-level courses 3.88 4.10 0.14
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Not at all competent, 5= Very competent

To what extent do you disagree or agree with the following Pre Post Effect
statements (17) (10) Size
(Incoming PhD students)
Factors
Research Competence 3.83 4.00 0.30
Research Interest 4.30 4.30* 0.43
Engineering Interest 4.47 4.61* 0.46
Valued & Belonging 3.77 3.93 0.15
Thriving & Growth 3.56 3.98 0.28
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

5
Incoming PhD post survey only:
Comparison of post-survey responses over race/ethnicity (1= white, Asian; 2= not white, not Asian)

The mean of pre- and post- survey responses are compared using the Mann-Whitney test and effect size
is reported. The Mann-Whitney test has been chosen to account for sample’s variation between pre-
and post-surveys. Statistical significance is reported.

To what extent do you disagree or agree with the following Race=1 Race=2 Effect
statements (25) (10) Size
(Incoming PhD students)
I am well-informed about the advisor search process 4.04 3.70 0.22
I know where to focus my effort during my first year in graduate 3.28 3.90 0.29
school
I know what to expect for my first year in graduate school 3.50 3.80 0.16
I know engineering graduate students who have successfully 4.28 4.50 0.26
completed their first year of graduate school
I am excited about the research I will do at UT Austin 4.24 4.50 0.17
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Race=1 Race=2 Effect


(Incoming PhD students) (25) (10) Size
Understanding what is expected of me in graduate school 3.72 4.00 0.15
Finding a stimulating research group 3.88 4.20 0.24
Doing good research at UT Austin 3.84 3.90 0.03
Settling into life in a new city 4.08 4.10 0.06
Finding resources for living in Austin 3.92 4.03 0.29
Making friends or finding community in Austin, beyond the 3.52 3.60 0.02
university
Finding help if struggling academically 3.40 3.20 0.11
Finding help if struggling with stress, mental health, or wellbeing 3.20 3.60 0.25
Performing well in graduate-level courses 4.08 4.10 0.02
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Not at all competent, 5= Very competent

To what extent do you disagree or agree with the following Race=1 Race=2 Effect
statements (25) (10) Size
(Incoming PhD students)
Factors
Research Competence 3.99 4.00 0.06
Research Interest 4.46 4.50 0.11
Engineering Interest 4.61 4.61 0.06
Valued & Belonging 4.04 3.93 0.11
Thriving & Growth 3.88 3.96 0.23
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

6
Incoming PhD not male:
Comparison of pre- and post- survey responses of Incoming PhD students who self-identified other
than male
The mean of pre- and post- survey responses are compared using the Mann-Whitney test and effect size
is reported. The Mann-Whitney test has been chosen to account for sample’s variation between pre-
and post-surveys. Statistical significance is reported.

To what extent do you disagree or agree with the following Pre Post Effect
statements (19) (14) Size
(Incoming PhD students)
I am well-informed about the advisor search process 3.58 3.86 0.18
I know where to focus my effort during my first year in graduate 3.58 3.43 0.07
school
I know what to expect for my first year in graduate school 3.37 3.71 0.20
I know engineering graduate students who have successfully 3.95 4.57* 0.38
completed their first year of graduate school
I am excited about the research I will do at UT Austin 4.42 4.43 0.10
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Pre Post Effect


(Incoming PhD students) (19) (14) Size
Understanding what is expected of me in graduate school 3.95 4.00 0.05
Finding a stimulating research group 3.63 4.29* 0.43
Doing good research at UT Austin 3.58 3.93 0.24
Settling into life in a new city 3.84 3.86 0.05
Finding resources for living in Austin 4.11 4.14 0.01
Making friends or finding community in Austin, beyond the 3.32 3.86 0.23
university
Finding help if struggling academically 3.26 3.50 0.13
Finding help if struggling with stress, mental health, or wellbeing 3.47 3.36 0.06
Performing well in graduate-level courses 3.95 4.29 0.24
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Not at all competent, 5= Very competent

To what extent do you disagree or agree with the following Pre Post Effect
statements (19) (14) Size
(Incoming PhD students)
Factors
Research Competence 3.92 4.10 0.13
Research Interest 4.35 4.43 0.24
Engineering Interest 4.47 4.52 0.15
Valued & Belonging 3.73 4.01 0.25
Thriving & Growth 3.74 3.93 0.15
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

7
Incoming PhD post survey only:
Comparison of post-survey responses over gender (1= male; 2= not male)

The mean of pre- and post- survey responses are compared using the Mann-Whitney test and effect size
is reported. The Mann-Whitney test has been chosen to account for sample’s variation between pre-
and post-surveys. Statistical significance is reported.

To what extent do you disagree or agree with the following Gender=1 Gender=2 Effect
statements (21) (14) Size
(Incoming PhD students)
I am well-informed about the advisor search process 4.00 3.86 0.06
I know where to focus my effort during my first year in graduate 3.48 3.43 0.02
school
I know what to expect for my first year in graduate school 3.50 3.71 0.09
I know engineering graduate students who have successfully 4.19 4.57 0.20
completed their first year of graduate school
I am excited about the research I will do at UT Austin 4.24 4.43 0.20
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Gender=1 Gender=2 Effect


(Incoming PhD students) (21) (14) Size
Understanding what is expected of me in graduate school 3.67 4.00 0.22
Finding a stimulating research group 3.76 4.29* 0.33
Doing good research at UT Austin 3.81 3.93 0.12
Settling into life in a new city 4.24 3.86 0.19
Finding resources for living in Austin 3.95 4.14 0.13
Making friends or finding community in Austin, beyond the 3.33 3.86 0.25
university
Finding help if struggling academically 3.24 3.50 0.13
Finding help if struggling with stress, mental health, or wellbeing 3.29 3.36 0.04
Performing well in graduate-level courses 3.95 4.29 0.18
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1=Not at all competent, 5=Very competent

To what extent do you disagree or agree with the following Gender=1 Gender=2 Effect
statements (21) (14) Size
(Incoming PhD students)
Factors
Research Competence 3.90 4.10 0.16
Research Interest 4.50 4.43 0.20
Engineering Interest 4.67 4.52 0.33
Valued & Belonging 4.02 4.01 0.03
Thriving & Growth 3.89 3.93 0.00
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

8
Incoming PhD enrolled at BME dept: Comparison of pre- and post-survey responses
The mean of pre- and post- survey responses are compared using the Mann-Whitney test and effect size
is reported. The Mann-Whitney test has been chosen to account for sample’s variation between pre-
and post-surveys. Statistical significance is reported.

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (19) (17) Size
I am well-informed about the advisor search process 3.74 4.12 0.27
I know where to focus my effort during my first year in graduate 3.26 3.82 0.28
school
I know what to expect for my first year in graduate school 3.21 3.76 0.27
I know engineering graduate students who have successfully 4.37 4.59 0.17
completed their first year of graduate school
I am excited about the research I will do at UT Austin 4.68 4.59 0.06
I received high-quality peer mentoring this summer (post only) 3.76
I received high-quality faculty mentoring this summer (post only) 3.35
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Pre Post Effect


(Incoming PhD students) (19) (17) Size
Understanding what is expected of me in graduate school 3.79 3.82 0.03
Finding a stimulating research group 3.74 4.06 0.27
Doing good research at UT Austin 3.53 4.06* 0.35
Settling into life in a new city 3.89 4.06 0.06
Finding resources for living in Austin 4.00 3.94 0.06
Making friends or finding community in Austin, beyond the university 3.16 3.76 0.24
Finding help if struggling academically 3.16 3.53 0.19
Finding help if struggling with stress, mental health, or wellbeing 3.63 3.29 0.18
Performing well in graduate-level courses 4.11 4.29 0.15
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Not at all competent, 5= Very competent

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (19) (17) Size
Factors
Research Competence 3.81 4.04 0.23
Research Interest 4.58 4.55 0.09
Engineering Interest 4.41 4.52 0.27
Valued & Belonging 3.90 4.08 0.29
Thriving & Growth 3.70 4.03 0.24
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

9
Incoming PhD enrolled at ECE dept: comparison of pre- and post-survey responses
The mean of pre- and post- survey responses are compared using the Mann-Whitney test and effect size
is reported. The Mann-Whitney test has been chosen to account for sample’s variation between pre-
and post-surveys. Statistical significance is reported.

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (30) (10) Size
I am well-informed about the advisor search process 3.10 3.22 0.01
I know where to focus my effort during my first year in graduate school 2.81 2.88 0.00
I know what to expect for my first year in graduate school 2.90 3.00 0.05
I know engineering graduate students who have successfully completed 3.71 3.56 0.02
their first year of graduate school
I am excited about the research I will do at UT Austin 4.19 3.67 0.27
I received high-quality peer mentoring this summer (post only) 2.67
I received high-quality faculty mentoring this summer (post only) 2.56
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Pre Post Effect


(Incoming PhD students) (30) (10) Size
Understanding what is expected of me in graduate school 3.47 3.44 0.01
Finding a stimulating research group 3.32 3.33 0.01
Doing good research at UT Austin 3.37 3.33 0.02
Settling into life in a new city 3.42 4.11 0.28
Finding resources for living in Austin 3.65 4.22 0.21
Making friends or finding community in Austin, beyond the university 2.94 2.56 0.13
Finding help if struggling academically 3.32 2.67 0.28
Finding help if struggling with stress, mental health, or wellbeing 3.06 3.22 0.04
Performing well in graduate-level courses 3.74 4.00 0.16
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Not at all competent, 5= Very competent

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (30) (10) Size
Factors
Research Competence 3.83 3.71 0.15
Research Interest 4.27 4.04 0.15
Engineering Interest 4.49 4.62 0.27
Valued & Belonging 3.66 3.57 0.03
Thriving & Growth 3.57 3.42 0.14
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

10
Matching Incoming PhD: comparison of pre and post survey responses of matching sample.
In order to protect the identity of the respondents, students were asked two questions to build a code
that would allow to identify matching respondents for pre- and post- survey. For this analysis, we
included only responses of pre- and post- surveys from the same individuals. The mean of pre- and post-
survey responses are compared using the Wilcoxon matched-pairs signed-ranks test and effect size is
reported. Statistical significance is reported.

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (17) (17) Size
I am well-informed about the advisor search process 3.24 3.94 0.42
I know where to focus my effort during my first year in graduate school 3.06 3.47 0.21
I know what to expect for my first year in graduate school 3.12 3.63 0.29
I know engineering graduate students who have successfully completed 4.00 4.24 0.15
their first year of graduate school
I am excited about the research I will do at UT Austin 4.53 4.35 0.11
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Pre Post Effect


(Incoming PhD students) (17) (17) Size
Understanding what is expected of me in graduate school 3.82 3.94 0.10
Finding a stimulating research group 3.47 3.94 0.34
Doing good research at UT Austin 3.82 3.94 0.20
Settling into life in a new city 4.24 4.29 0.05
Finding resources for living in Austin 4.12 4.35 0.20
Making friends or finding community in Austin, beyond the university 3.06 3.71 0.36
Finding help if struggling academically 3.18 3.47 0.21
Finding help if struggling with stress, mental health, or wellbeing 3.47 3.65 0.15
Performing well in graduate-level courses 3.88 4.12 0.25
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Not at all competent, 5= Very competent

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (17) (17) Size
Factors
Research Competence 3.82 4.01 0.30
Research Interest 4.55 4.49 0.23
Engineering Interest 4.66 4.69 0.19
Valued & Belonging 3.77 4.04 0.46
Thriving & Growth 3.50 3.84 0.35
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

11
Matching Incoming PhD: comparison of pre and post survey responses of matching sample of
students enrolled at BME dept.
In order to protect the identity of the respondents, students were asked two questions to build a code
that would allow to identify matching respondents for pre- and post- survey. For this analysis, we
included only responses of pre- and post- surveys from the same individuals. The mean of pre- and post-
survey responses are compared using the Wilcoxon matched-pairs signed-ranks test and effect size is
reported. Statistical significance is reported.

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (9) (9) Size
I am well-informed about the advisor search process 3.78 4.11 0.35
I know where to focus my effort during my first year in graduate school 3.33 3.78 0.30
I know what to expect for my first year in graduate school 3.44 3.89 0.30
I know engineering graduate students who have successfully completed 4.56 4.67 0.04
their first year of graduate school
I am excited about the research I will do at UT Austin 4.78 4.78 0.00
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Pre Post Effect


(Incoming PhD students) (9) (9) Size
Understanding what is expected of me in graduate school 4.22 4.00 0.19
Finding a stimulating research group 4.00 4.11 0.17
Doing good research at UT Austin 3.89 4.22 0.38
Settling into life in a new city 4.00 4.22 0.22
Finding resources for living in Austin 4.11 4.22 0.06
Making friends or finding community in Austin, beyond the university 3.44 3.89 0.28
Finding help if struggling academically 3.22 3.78 0.43
Finding help if struggling with stress, mental health, or wellbeing 3.56 3.67 0.21
Performing well in graduate-level courses 4.00 4.44 0.46
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Not at all competent, 5= Very competent

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (9) (9) Size
Factors
Research Competence 3.78 4.00 0.64
Research Interest 4.73 4.71 0.19
Engineering Interest 4.60 4.75* 0.65
Valued & Belonging 3.69 4.06 0.63
Thriving & Growth 3.69 4.06 0.56
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

12
Matching Incoming PhD: comparison of pre and post survey responses of matching sample
for students enrolled at ECE dept.
In order to protect the identity of the respondents, students were asked two questions to build a code
that would allow to identify matching respondents for pre- and post- survey. For this analysis, we
included only responses of pre- and post- surveys from the same individuals. The mean of pre- and post-
survey responses are compared using the Wilcoxon matched-pairs signed-ranks test and effect size is
reported. Statistical significance is reported.

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (5) (5) Size
I am well-informed about the advisor search process 2.00 3.20 0.86
I know where to focus my effort during my first year in graduate school 2.20 3.00 0.50
I know what to expect for my first year in graduate school 2.00 3.20* 0.95
I know engineering graduate students who have successfully completed 3.20 3.60 0.18
their first year of graduate school
I am excited about the research I will do at UT Austin 4.20 3.80 0.32
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Pre Post Effect


(Incoming PhD students) (5) (5) Size
Understanding what is expected of me in graduate school 3.20 3.80 0.32
Finding a stimulating research group 2.20 3.40 0.86
Doing good research at UT Austin 3.80 3.60 0.07
Settling into life in a new city 4.20 4.80 0.45
Finding resources for living in Austin 4.00 4.40 0.07
Making friends or finding community in Austin, beyond the university 2.60 3.20 0.50
Finding help if struggling academically 3.20 2.80 0.12
Finding help if struggling with stress, mental health, or wellbeing 3.40 3.60 0.07
Performing well in graduate-level courses 3.60 4.00 0.32
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Not at all competent, 5= Very competent

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (5) (5) Size
Factors
Research Competence 3.64 3.96 0.49
Research Interest 4.40 4.20 0.76
Engineering Interest 4.74 4.69 0.63
Valued & Belonging 3.53 3.90 0.71
Thriving & Growth 3.45 3.45 0.08
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

13
Matching not-male Incoming PhD:
Comparison of pre and post survey responses of matching sample for students who self-
identified as other than male.
In order to protect the identity of the respondents, students were asked two questions to build a code
that would allow to identify matching respondents for pre- and post- survey. For this analysis, we
included only responses of pre- and post- surveys from the same individuals. The mean of pre- and post-
survey responses are compared using the Wilcoxon matched-pairs signed-ranks test and effect size is
reported. Statistical significance is reported.

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (5) (5) Size
I am well-informed about the advisor search process 3.80 4.20 0.45
I know where to focus my effort during my first year in graduate school 3.80 4.00 0.19
I know what to expect for my first year in graduate school 3.80 4.20 0.25
I know engineering graduate students who have successfully completed 4.40 4.60 0.26
their first year of graduate school
I am excited about the research I will do at UT Austin 4.80 4.60 0.45
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Pre Post Effect


(Incoming PhD students) (5) (5) Size
Understanding what is expected of me in graduate school 4.20 4.40 0.45
Finding a stimulating research group 3.80 4.40 0.77
Doing good research at UT Austin 3.80 4.20 0.45
Settling into life in a new city 4.20 4.40 0.45
Finding resources for living in Austin 4.40 4.80 0.63
Making friends or finding community in Austin, beyond the university 3.40 4.40 0.63
Finding help if struggling academically 3.20 4.20 0.76
Finding help if struggling with stress, mental health, or wellbeing 3.40 3.60 0.19
Performing well in graduate-level courses 4.20 4.60 0.63
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Not at all competent, 5= Very competent

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (5) (5) Size
Factors
Research Competence 3.92 4.36* 0.91
Research Interest 4.64 4.48 0.87
Engineering Interest 4.74 4.60* 0.88
Valued & Belonging 3.87 4.37* 0.99
Thriving & Growth 3.90 4.05 0.46
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

14
Matching Incoming PhD from minoritized backgrounds:
Comparison of pre and post survey responses of matching sample for students who self-
identified as not white and/or not Asian.
In order to protect the identity of the respondents, students were asked two questions to build a code
that would allow to identify matching respondents for pre- and post- survey. For this analysis, we
included only responses of pre- and post- surveys from the same individuals. The mean of pre- and post-
survey responses are compared using the Wilcoxon matched-pairs signed-ranks test and effect size is
reported. Statistical significance is reported.

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (5) (5) Size
I am well-informed about the advisor search process 2.60 3.80 0.68
I know where to focus my effort during my first year in graduate school 2.80 4.00* 0.95
I know what to expect for my first year in graduate school 2.80 3.80 0.68
I know engineering graduate students who have successfully completed 3.80 4.20 0.31
their first year of graduate school
I am excited about the research I will do at UT Austin 4.60 4.60 0.00
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree, 5=Strongly Agree

How competent are you… Pre Post Effect


(Incoming PhD students) (5) (5) Size
Understanding what is expected of me in graduate school 3.40 3.80 0.32
Finding a stimulating research group 2.60 3.80 0.60
Doing good research at UT Austin 3.60 3.80 0.26
Settling into life in a new city 4.20 4.40 0.26
Finding resources for living in Austin 4.20 4.60 0.32
Making friends or finding community in Austin, beyond the university 2.60 3.60 0.55
Finding help if struggling academically 3.20 3.20 0.00
Finding help if struggling with stress, mental health, or wellbeing 3.60 3.80 0.26
Performing well in graduate-level courses 3.80 4.40 0.46
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Not at all competent, 5= Very competent

To what extent do you disagree or agree with the following statements Pre Post Effect
(Incoming PhD students) (5) (5) Size
Factors
Research Competence 3.48 3.84 0.30
Research Interest 4.44 4.44 0.06
Engineering Interest 4.71 4.77 0.45
Valued & Belonging 3.83 3.87 0.00
Thriving & Growth 3.80 3.95 0.63
Effect size: .1 to .29 small, .3 to .49 medium, and larger than .5 large
*p≤0.05; **p≤0.01
Likert scale: 1= Strongly disagree/Not at all competent, 5=Strongly Agree/Very competent

15
Comments
In the pre- and post-surveys, students were asked: “How can your engineering department or graduate
program be more inclusive for all students?”

Answers are reported in the tables below:

PRE-Survey

What is your How can your engineering department or graduate program be more
department? inclusive for all students?
ME First of all, many of these questions (and the Cockrell School in general) are
structured based on the assumption that we identify primarily as an engineer.
For many grad programs that are more math/ theory/ computer science
related, this may not be true. I am in the school of engineering, but I do not
consider myself an engineer, nor do I aspire to.

I also think UT should do much more to make life easier for its international
students. I'm a domestic student, and whenever I talk to my non-domestic
friends, I am amazed at the hoops that international students have to jump
through. For instance, when the Trump administration was threatening the
status of international, particularly Chinese students, UT should have been
much stronger and more vocal in their support of international students. I've
also heard that the international student office can take literally months to
respond to requests. This is egregious. UT should be embarrassed.

Some departments should also work towards hiring more women faculty. I am
in the [ME Program Area], and none of the core faculty are women. I don't
understand how this is acceptable, and multiple people have raised it as a
concern that made them consider alternative schools for their graduate
education.
ME I think just more activities that encourages people getting together and
working in smaller groups. At this point, I think the program does a decent job
at being inclusive and highlighting previous student's challenges and paths to
graduate school. I think allowing for more "mingling" so us students can better
know one another and become closer colleagues would be beneficial to
understand one another more.
ME Cross-cultural activities
ME faculty trainings/facilitations. open conversation between students and faculty
so they understand better what their students need from them and how to act
appropriately
BME More in-person actions that don't seem like "extra work"
BME provide resources for students that are first generation college students.
BME Consider activities for groups of students that share similar hobbies (e.g.,
gaming)
BME Ask minorities students how they feel in the program and how the program
can improve.
BME Hear what students have to say but take concrete steps to implement the
solutions in a timely manner

16
BME fewer abusive PIs
BME Understanding the background of each individual.
BME If I'm being really honest, get rid of professors that constantly act out
microaggressions on racial and gender minorities and support systemic
inequalities. We have them, and they terrorize students. We also want more
faculty engagement in graduate student activities (sponsoring and engaging
with student orgs, outreach events, attending recruitment and the graduate
student retreat), and we should have a departmental program that provides
fellowships for undergraduate student researchers to reduce trickle-down
inequalities in graduate school.
BME Having a club dedicated to appreciating different cultures around important
holidays in that culture.
BME I think community building is seriously lacking. This program is a great program
for incoming students, but I still feel isolated from other older students and
professors in the department. Social programming is left for students to plan
and fund, which is a large burden given our lagging compensation compared
to Austin's rising cost of living. I think a clear department prioritization of
community building would improve feelings of belonging.
BME More upper-level examples of inclusive identities (professors, postdocs, etc.),
specifically regards to race, gender identity, and sexuality
BME Creating a space, including a platform and time, for students from all
backgrounds to feel the urge to share and express their thoughts. The urge to
communicate and share needs to be kindled and nurtured - else only the
extroverted folks who want to talk will do so, or those who are at the cusp of
breakdown and can't take it anymore. Early diagnosis rather than symptom-
based initiative (like for breast cancer or diabetes). The voices of everyone,
even the reserved, left out and introverted/shy or even the folks who feel they
don't want to speak up, should be represented. In a world that is so fast paced
the element of sharing with our colleagues is becoming increasingly difficult,
especially due to Covid and the boom of social media. We need our
department to become a home where we want to share things voluntarily.
PGE I would like to know more about the advisor search process.
ECE Better advising communication that does not rely solely on emails. I.e., phone
calls, zoom meetings with graduate advisors and department staff.
ECE By moving Pickle campus closer to UT

By making available more parking space for students with A/C parking permits
or giving special permits for students working at pickle campus regularly

By hosting journal clubs/study groups

ECE A better orientation or follow up activities were you have opportunities to


perform activities and mingle with other incoming graduate students in your
same department and others.

17
POST- survey

What is your
department
or graduate How can your engineering department or graduate program be more
program? inclusive for all students?
CHE Hire more faculty from different universities. Hire non-white staff.
CHE Ensure proper representation for international students in seminars,
department events, etc.
CHE Honestly, we (ChE) need more supporting staff. This is both for
administration activities such as ordering and other logistics and also
personal assistants/lab managers for faculty. The high staff turnaround rate
in our department should be an indication that something needs to change.
Everyone is stretched so thin right now and it means a lot of tasks get
forgotten or pushed off. It's not the fault of the staff, it's that there are too
many tasks for them to complete. Please, please take this comment seriously
and hire more support staff. If there are more staff, then there are
opportunities for us to feel heard and supported more which is the most
tangible way to make this program/department more inclusive.
ECE already inclusive but should encourage more women to apply
ECE To be precise, the department has to strike a balance between the
toughness of the course and the easiness of the course. In some of the
courses, it is balanced but in one of the courses, the load is very high.
BME I think community building is the largest need right now i the department.
Small group activities are useful but I’d like larger gatherings sponsored by
the department as allowed by current covid restrictions
BME Increasing diversity throughout the department. I know this is an issue being
discussed currently and I think it is moving in the right direction. This can be
done through the diversity of the incoming graduate students and the
faculty.
BME It's a matter of making it a priority at leadership and institutional levels.
Professors, especially the "famous" professors get away with too much.
BME Do activities where everyone gets a chance to have conversations with
people from all backgrounds and with faculty. A social hour or something
along those lines where we can have honest conversations with each other.
The department is very good at advertising its diverse student body and yet
it does very few activities (if none at all) during the school year to show
appreciation and care for them.
BME Hire more diverse faculty and students
BME I think the BME department is already doing a great job at hiring diverse
faculty and having a diverse pool of students
BME Create physical spaces for students to congregate more, more social events
that were built to be more intentionally inclusive.
BME Professors during the rotation process need to be more transparent about
how many students they can accept, give feedback as to why a student was
rejected from the lab, and communicate with the students. Having to reach
out to multiple professors for weeks without a response is unacceptable and

18
causes unnecessary stress on incoming graduate students who already have
enough to worry about.
BME Consider students with specific hobbies outside of research or students who
would prefer spending time with others who like the same stuff.
BME Be more aware, at the ground level, why certain groups or categories of grad
students do not see any need/benefit in engaging with the rest of the
undergraduate/graduate/post-graduate/faculty communities. E.g.,
International students, post-docs, introverts/those less inclined towards
small-talk/non-research-socializing.
One platform that I envision could do wonders for building community is a
weekly/fortnightly informal work in progress seminar series amongst peers.
Wherein the entire community gather for a casual retreat-style hour of
presentation(s) of one/more work in progress talks by undergraduate,
graduate and even post-doc candidates - which would engender genuine
curiosity-driven discussion/questions after the talks. Volunteers who wish to
present/talk would sign up in advance for 15/30/45 min slots - and would
hopefully invite their entire research group to attend. The benefit/incentive
is that this would serve as a more diverse platform for practicing talks
beyond the comfort-zone of research groups - especially useful for
confidence boosting and feedback during the run up to milestones like
qualifying exam/proposal/dissertation/conference/investor-pitch etc.
While the seminar itself revolves around the research, I found that as an
attendee of a similar practice in my alma mater, this exercise helped me get
to know (and share with) others in the department better, meet more
frequently (in contrast to meeting them a few times an year, by chance), as
well as learn so many communication techniques from the diverse kinds of
questions (and answering paradigms) during/after the talks. This seemed to
me like a good way to reduce the gap between disconnected labs in different
(parts of) buildings.
This only sustains or grows when the push to participate in these comes from
within (and from peers) but also when the environment is made conducive
by the grad program/department and the faculty

19
Appendix A: 2021 Pre-survey 1

CSE Mentoring Program Pre-Survey 2021

Start of Block: Consent Form


Q1 You are invited to participate in this survey as a participant in a peer mentoring program within the
Cockrell School of Engineering. It should take approximately 10-15 minutes to complete.
You will receive no direct benefits from participating in this survey. However, your responses will help us
in the evaluation and assessment of this program. Your honest feedback is very important to us.
There are no foreseeable risks involved in participating in this study other than those encountered in
day-to-day life.
Your survey answers will be analyzed as aggregate, never individually examined. Your data will be kept
confidential. If you have questions at any time about the study or the procedures, you may contact us
via email at maura.borrego@austin.utexas.edu .
If you feel you have not been treated according to the descriptions in this form, or that your rights as a
participant in research have not been honored during the course of this project, or you have any
questions, concerns, or complaints that you wish to address to someone other than the investigator,
you may contact the Institutional Review Board at irb@austin.utexas.edu.

ELECTRONIC CONSENT:
Clicking on the “Agree” button indicates that
·You have read the above information
·You agree to participate
·You are 18 years of age or older

You may print a copy of this consent form for your records.

Agree (1)
Not Agree (2)

Skip To: End of Survey If You are invited to participate in this survey as a participant in a peer mentoring
program within... = Not Agree

Q2 This study involves data collection before and after the mentoring program. We would like to create
a unique participant ID to match your responses over time. We will ask you for the following information
in each survey.

1
In this report, we include only the pre-survey because pre- and post-survey are similar.

20
Please enter the date of the month you were born on. For example, if you were born on December 4th,
enter "04". If you were born June 30th, enter "30".

________________________________________________________________

Q3 This study involves data collection before and after the mentoring program. We would like to create
a unique participant ID to match your responses over time. We will ask you for the following information
in each survey.
Please enter the last four digits of your cell phone number. For example, your cell phone number is 555-
555-1234, enter “1234”

________________________________________________________________

Q4 What is your department?

Department of Aerospace Engineering and Engineering Mechanics (1)

Department of Biomedical Engineering (2)

McKetta Department of Chemical Engineering (3)

Department of Civil, Architectural and Environmental Engineering (4)

Department of Electrical and Computer Engineering (5)

Walker Department of Mechanical Engineering (6)

Hildebrand Department of Petroleum and Geosystems Engineering (7)

21
Q5 In 2021, are you involved in this program as a mentor or an incoming graduate student?

Incoming graduate student (1)


Peer Mentor (2)

Skip To: End of Block If In 2021, are you involved in this program as a mentor or an incoming graduate
student? = Incoming graduate student

Q6 What year did you enter your graduate program in the Cockrell School of Engineering at UT Austin?

2021 (12)
2020 (1)
2019 (2)
2018 (3)
2017 (4)
2016 (5)
2015 (6)
2014 (7)
2013 or earlier (8)

End of Block: Consent Form

Start of Block: Graduate Research & Engineering Identity Questions

Q7 How competent are you with the following tasks?

Rate from 1 to 5

2 - Somewhat
1 - Not at all 4 - Somewhat 5 - Very
not competent 3 - Neutral (3)
competent (1) competent (4) competent (5)
(2)

22
Understanding
and applying
scientific and
mathematical
relationships
based on the
conditions (1)

Applying math
and science
concepts to
make new
systems/models
(16)

Using
calculations and
equations to
evaluate things
(17)

Understanding
derivations and
equations in
journal papers
(18)

Understanding
current research
findings by using
sufficient math,
science or
engineering
knowledge (19)

Doing good
research at UT
Austin (22)

Understanding
what is
expected of me
in graduate
school (14)

Display This Question:


If In 2021, are you involved in this program as a mentor or an incoming graduate student? =
Incoming graduate student

23
Q8 How competent are you with the following tasks?

Rate from 1 to 5

2 - Somewhat
1 - Not at all 4 - Somewhat 5 - Very
not competent 3 - Neutral (3)
competent (1) competent (4) competent (5)
(2)

Finding a
stimulating
research group
(21)

Performing well
in graduate-
level courses
(28)

Finding
resources for
living in Austin
(24)

Settling into life


in a new city
(23)

Making friends
or finding
community in
Austin, beyond
the university
(25)

Finding help if
struggling
academically
(26)

Finding help if
struggling with
stress, mental
health, or
wellbeing (27)

Q9
To what extent do you disagree or agree with the following statements?

Rate from 1 to 5

24
1 - Strongly 5 - Strongly
2 Disagree (2) 3 - Neutral (3) 4 - Agree (4)
disagree (1) agree (5)

I like doing
engineering (11)

I am interested
in my research
topic (14)

I am interested
in learning more
about research
(28)

I enjoy research
activities as part
of my work
week (21)

I am interested
in learning more
about
engineering (19)

I enjoy
engineering
activities as part
of my work
week (22)

In general, I find
working on
research
interesting (24)

I like doing
research (25)

I think
engineering is
fun (39)

I think
engineering is
interesting (27)

I like to figure
out how things
work (26)

I feel good
when I am
doing
engineering (23)

25
Display This Question:
If In 2021, are you involved in this program as a mentor or an incoming graduate student? =
Incoming graduate student
Q10 To what extent do you disagree or agree with the following statements?

Rate from 1 to 5

1 - Strongly 5 - Strongly
2 Disagree (2) 3 - Neutral (3) 4 - Agree (4)
disagree (1) agree (5)

I am well-
informed about
the advisor
search process
(40)

I know where to
focus my effort
during my first
year in graduate
school (48)

I know what to
expect for my
first year in
graduate school
(49)

I know
engineering
graduate
students who
have
successfully
completed their
first year of
graduate study
(50)

I am excited
about the
research I will
do at UT Austin
(51)

End of Block: Graduate Research & Engineering Identity Questions


Start of Block: Michigan Climate questions

26
Q11 In your day-to-day graduate experience so far, to what extent do you agree with the following
statements?

Rate from 1 to 5

1 - Strongly 3 - Neither
5 - Strongly
disagree 2 - Disagree (2) agree nor 4 - Agree (4)
agree (5)
(1) disagree (3)

I feel valued as an
individual (1)

I feel I belong (2)

I have considered
leaving because I felt
isolated or
unwelcome (4)

I am treated with
respect (5)

I feel others don’t


value my opinions (6)

My department is a
place where I am
able to perform up
to my full potential
(7)

I have opportunities
for academic success
that are similar to
those of my peers (8)

I have found one or


more communities
or groups where I
feel I belong (9)

I have to work harder


than others to be
valued equally (12)

My experience has
had a positive
influence on my
academic growth
(13)

End of Block: Michigan Climate questions

27
Start of Block: Block 4

Q12 Please select the response that best describes your gender identity

Man (1)
Non-binary or third gender (2)
Trans* or Transgender (please specify) (3)
________________________________________________
Woman (4)
I prefer not to answer (5)
My gender identity cannot be wholly expressed in the above categories. I identify as... (6)
________________________________________________

Q13 Please select all the responses that apply to your racial/ethnic identity

African American/Black (1)

American Indian/Indian Subcontinent (2)

Hispanic or Latino/a/x (3)

Native Hawaiian/Pacific Islander (4)

Southeast Asian (5)

White/Caucasion (6)

I prefer not to answer (8)

Other (7) ________________________________________________

28
Q14 How can your engineering department or graduate program be more inclusive for all students?

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

________________________________________________________________

End of Block: Block 4

29

You might also like