Journal of Parallel and Distributed Computing 158 (2021) 151–163

Contents lists available at ScienceDirect

Journal of Parallel and Distributed Computing

www.elsevier.com/locate/jpdc

Developing parallel programming and soft skills: A project based

learning approach
Awad A. Younis a , Rajshekhar Sunderraman b , Mike Metzler c , Anu G. Bourgeois b,∗
a
Department of Computer Science, Northern Kentucky University, Highland Heights, KY, USA
b
Department of Computer Science, Georgia State University, Atlanta, GA, USA
c
Center for Excellence in Teaching and Learning, Georgia State University, Atlanta, GA, USA

a r t i c l e i n f o a b s t r a c t

Article history: Upon graduation, a computer science student should have a good understanding of the current
Received 29 October 2020 technology and have the soft skills necessary to secure a position in industry. Considering that typical
Received in revised form 11 June 2021 computers and even the common smartphone are multicore, students should be skilled in parallel
Accepted 18 July 2021
programming. Integrating parallel programming and soft skills within courses can help educate students
Available online 9 August 2021
on these essential skills. Our goal is to explore the effectiveness of using Project Based Learning (PBL)
Keywords: to teach these skills when classes are at content capacity. We divide 247 students into 51 diverse
Parallel computing groups and assigned five projects, each of two-week duration. We use pre- and post-surveys to measure
Open MP growth and found that incorporating PBL has a significant effect on the students’ parallel programming
Shared memory and soft skills. We show that through teamwork, students collaboratively learn and apply fundamental
Project based learning parallel programming and soft skills without direct guidance, thus demonstrating the effectiveness of PBL.
CS education The implementation was conducted in a course that does not traditionally teach parallel programming
concepts, but with the use of a PBL approach, students were able to acquire this new knowledge.
© 2021 Elsevier Inc. All rights reserved.

1. Introduction This study is to investigate ways to do so without sacrificing other

We live in a world where technology is evolving at a high rate.
This compels industry to seek out innovative people when hiring.
For this reason, computer science students should have a basic
understanding of today’s technology upon graduation. Without a
solid grasp of the technology, it is impossible to have innovation.
Therefore, we must ensure that our students understand concepts
of parallel programming, especially considering that the typical
computer is multicore, as are smartphones, which have become
pervasive in society [16]. They should be able to take advantage of
this shift toward multicore architectures [25].
Recognizing this trend, there was an update to the computer
science curriculum guidelines to include parallel programming as
a required topic area [20]. Learning parallel programming concepts,
such as multicore computer architectures, cloud distributed com-
puting, and general-purpose GPU will enable students to gain these
needed technical skills for the current state-of-the-art technology
and better prepare them for today’s workforce. Unfortunately, at
our institution, Georgia State University, we have not incorporated
parallel programming concepts as a requirement in our curriculum.
* Corresponding author at: 25 Park Place, Suite 700 Atlanta, GA 30302, USA.
E-mail address: abourgeois@gsu.edu (A.G. Bourgeois).
content.
While computer science students should learn the technical
concepts that make up the curriculum guidelines, it is also crit-
ical for students to develop professional soft skills in preparation
for the workforce. Research findings [8] show that senior engineer-
ing students are not often aware of soft skills, or the means to use
them through conflict resolution. They also emphasize the need for
embedding soft skills in a course early in the students’ program
as explicit instruction and emphasizing their importance and uti-
lization in future class projects. In other words, it is not sufficient
to put students in groups and simply ask them to work together.
Students must first learn the teamwork skills needed to function
successfully in group projects. Just as technical skills are critical
for success in the workplace, soft skills, such as teamwork, de-
cision making, cooperation, collaboration, conflict resolution, and
communication are recognized as crucial for computer science stu-
dents, as the industry is becoming more collaborative and interdis-
ciplinary [33,2]. It is for this reason, that we want to introduce and
incorporate having students work on teams and learn soft skills in
a class typically taken during the second year of the degree, rather
than waiting for their senior year or capstone course.
As a new approach in this area, we investigate the effective-
ness of using Project Based Learning (PBL) to teach parallel pro-

https://doi.org/10.1016/j.jpdc.2021.07.015
0743-7315/© 2021 Elsevier Inc. All rights reserved.
A.A. Younis, R. Sunderraman, M. Metzler et al.
Fig. 1. Timeline of project modules and surveys.
gramming concepts and soft skills early in the computer science
curriculum. PBL is an engaging instructional model that provides
more learning experience and leads to higher-level cognitive de-
velopment through students’ engagement with complex problems
[7]. PBL supports diverse groups of students in learning and prac-
ticing skills in problem solving, communication, collaboration, and
self-management. With collaborative learning, availability of the
required materials, and guidance from the instructor, we hypoth-
esize that students can collaboratively learn by themselves and
apply the fundamentals of parallel programming principles and
soft skills without the need for separate lectures, labs, or work-
shops, or by taking time in the regular lecture times and removing
other content. This leads to our research hypotheses that specif-
ically consider the first half of the semester as compared to the
second half and are formulated as follows:
• Hypothesis 1: There is a difference in emphasis on paral-
lel programming and soft skills between the first and second
parts of the semester.
• Hypothesis 2: By incorporating project-based learning, the stu-
dents acquire personal growth and improvement on their par-
allel programming skills and professional soft skills.
• Hypothesis 3: Students growth in parallel programming and
soft skills increase when greater emphasis is placed on these
areas.
To investigate the effectiveness of PBL, we developed and in-
corporated a semester-long PBL module in CSC 3210 (Computer
Organization and Programming), a core course taken by all majors
and a prerequisite to most of our senior-level classes. CSC 3210 is
taken close to the midpoint of the academic program. With our
changes to the course, it can now serve as a mini-capstone course
that provides an opportunity for students to learn critical parallel
programming skills early in the curriculum, in spite of it not being
part of the required course content. It also enables students to get
familiar with the logistics of team projects and soft skills. Fig. 1
provides a timeline of our PBL approach, including the topic areas
covered with each phase of the projects.
This paper is an extension of a preliminary study [43]. We
have since repeated the implementation for a second semester
and included these new results. We have also conducted a follow
up study on the students from the first semester in their soft-
ware engineering course taken in a later semester. These results,
as well as a more detailed analysis of the study results are pre-
sented here. The rest of our paper is organized as follows. Section 2
presents brief background on the hardware and software used in
our study to have students self-learn concepts of parallel program-
ming. Section 3 provides related work in the areas of teaching
parallel computing and soft skills. In Section 4, we describe the
overall approach and implementation of the Project Based Learn-
ing (PBL) module incorporated into our course. Section 5 addresses
the methods for grading and assessment of the PBL module, while
Section 6 presents the analysis and results of the PBL approach ef-
fectiveness. We provide a discussion of our findings in Section 7
and conclude the paper in Section 8.
152
Journal of Parallel and Distributed Computing 158 (2021) 151–163
2. Background
Single board computers (SBCs) including the Raspberry Pi are
small computing devices that can be used for different purposes
such as learning how to program, experimentation, drones and
webservers, and robotics and home automation. The most com-
mon SBC is the Raspberry Pi. The Raspberry Pi Model 3 features a
1.2 GHz quad-core ARM processor and 1 GB of RAM. Rasbian, the
Raspberry Pi operating system, comes pre-configured with Scratch
and Python. Due to it is multicore support, Raspberry Pi provides
a viable platform to teach parallel computing. It should be noted
that other affordable SBCs could be used as well.
Parallel computing is defined as the use of multiple resources,
in this case, processors, to solve a problem [4]. The main goal of
parallel programming is to decrease execution clock time. In paral-
lel programming, a problem is broken down into a series of smaller
steps, delivers instructions, and processors execute the solutions at
the same time. Raspberry Pi supports shared memory parallel sys-
tem. In a shared memory parallel system, multiple processors op-
erate independently but share the same memory resources. From a
parallel programming perspective, there are a number of program-
ming models including OpenMP [1] and MPI [42].
OpenMP is the new standard for shared memory programming
using compiler directives, known as an application programming
interface (APIs), which is available in C/C++ and Fortran implemen-
tations. OpenMP supports multi-platform including Solaris, AIX,
HP-UX, Linux, macOS, and Windows. OpenMP is very easy and
simple as it provides for incremental parallelism with serial code
[11]. In this research, we focus on using OpenMP with Rasp-
berry Pi to teach parallel programming. Although teaching parallel
programming as hard skills is critical and needed by employers
[20,36], teaching soft skills is also considered by business execu-
tives to be a very important attribute in the workplace [38].
Soft skills are interpersonal qualities and personal attributes
that a person possesses. Soft skills include teamwork, decision
making, cooperation, collaboration, conflict resolution, and com-
munication. There are multiple teaching and learning approaches
in computer science including lecture, problem-based learning,
lab-centered, computer simulation, and project work and direct
instruction [44]. In this research, we will explore using PBL as a
teaching and learning method for parallel programming and soft
skills.
3. Related work
In this section, we separately present previous research in the
areas of teaching parallel programming and soft skills for CS stu-
dents.
Teaching Parallel Programming: There have been many studies
and efforts at investigating ways to effectively teach parallel pro-
gramming that greatly vary in their approach. Modules have been
provided by CSinParallel [12] (supported by a grant from NSF-
TUES) to teach principles of parallelism and hands-on practice with
parallel computing. However, the effectiveness of teaching and
learning strategies for those modules have not been considered.
Other approaches are to use educational games [3,45], program-
ming competitions [15] or the use of programming patterns [9].
A.A. Younis, R. Sunderraman, M. Metzler et al.
Previous work has also taken the PBL approach to teaching paral-
lel computing [26], While these studies have shown to be effective
for student learning, they each require traditional lecture time to
be allocated to their teaching methodologies.
Recent research findings [29,28,27] have demonstrated that
teaching parallelism using single-board computers (SBCs), such as
the Raspberry Pi as a uniform work environment, is effective. Their
proposed approach, however, also used a lecture (in a workshop)
as a strategy for teaching. The drawback is that this requires find-
ing time, within the already packed computer science curriculum,
to cover additional topics. Our approach differs from previous re-
lated work in that we do not take away time in traditional lectures
to cover the concepts and instead, students self-learn in a guided
manner outside of the class.
Teaching Soft Skills: Recognizing the importance for developing
soft skills for computer science students has led to much research
on this topic. There are many different approaches that have been
presented in the literature.
One approach is to weave the soft skills concepts throughout
the curriculum [23]. This requires a coordinated effort to ensure
that the critical skills are distributed among the courses and to do
so in a manner to not sacrifice the technical content of the particu-
lar courses. Other approaches are to offer stand alone courses that
are usually referred to as professional development courses [37].
Considering that capstone and service learning courses usually
involve a long-term project and teamwork, there is motivation to
exploit these settings to weave in soft skills training [10]. This
same conjecture is used for studies that utilize the software en-
gineering course to teach soft skills [17,19,30]. This class is one
that typically has a semester-long team project and can serve as
a mechanism to cover these concepts. With our modifications to
incorporate PBL, we can consider the course to serve as a mini-
capstone course and implement this approach earlier in the cur-
riculum than these related studies.
4. Overall approach and methodology
In this section, we describe the process for the Project Based
Learning (PBL) module design and implementation. We first de-
scribe the course in which our study is conducted.
Computer Organization and Programming (CSC 3210) is usually
taken at the end of the second year of the CS curriculum. This
course introduces the principles of computer architecture as well
as assembly and machine language. Prior to adding these mod-
ules, we did not address concepts related to parallel architectures
and programming. However, with the adaption of PBL modules,
the CSC 3210 course now ensures that all students are exposed
to the topics and get an opportunity for hands-on learning. This
change addresses the challenge that parallel programming is not a
required course in our curriculum. We had a previous study [5] of
incorporating parallel and distributed computing topics throughout
our required curriculum, however, over the years, with change in
faculty and curriculum, those methods did not stay in place.
Fig. 2 provides a high-level view of the module design. Our
approach follows the methodology used to implement the PBL ap-
proach for an engineering course, where multiple “mini-projects”
were provided to students [14]. These mini-projects correspond to
what we call “modules” for our study. We first describe the overall
approach to the PBL module incorporated into our class and then
the details of the module design.
4.1. PBL module approach
Our courses are semester based and last a total of 15 weeks. In
the first week, we divide the students (247 total) into 51 diverse
groups (up to five per group). The team formation is made based
153
Journal of Parallel and Distributed Computing 158 (2021) 151–163
Fig. 2. PBL Module Design Overview.
on multiple criteria including gender, system and programming ex-
perience, experience in group work, GPA, and technical writing ex-
perience. These criteria are intended to help groups have a balance
in ability, of mixed gender and would avoid predetermined groups
of friends. After teams have been formed, five project assignments,
each of two-weeks duration, are provided to the students across
the span of the semester. (Refer to Fig. 1 for the timeline.)
With each assignment, we provide the required learning mate-
rials: Teamwork Basics [32]; Raspberry PI Multicore architecture
[29,28,27]; Shared Memory Parallel Patternlets in OpenMP [12];
Introduction to Parallel Computing [4]; CPU vs. SOC – The bat-
tle for the future of computing [46]; and Introduction to Parallel
Programming and MapReduce [24]. Along with the technical mate-
rial, students are provided the learning objectives, grading criteria,
team policies, peer rating form of team members’ contributions
to the team, and the specific tasks. By using the given materials,
students collaboratively learn and apply soft and critical thinking
skills in terms of teamwork basics, decision making, task identi-
fications, planning and scheduling, cooperation and collaboration,
conflict resolution, and communication. We required students to
utilize the following:
• Slack, a messaging application to communicate,
• GitHub, a social networking site for programmers to collabo-
rate, create customized workflows, and share code,
• Google Docs, an online word processor to collaborate and pro-
duce project assignments reports, and
• Videos and YouTube, to film, edit, and upload videos to a
YouTube channel to present the results.
All these technologies are free and available to all students.
Equipped with the given materials, students learn and apply
parallel programming concepts in terms of sequential and paral-
lel computation, multi-processor computer architectures (e.g. SISD,
SIMD, MISD, and MIMD), processes and threads, parallel program-
ming models, shared memory parallelism, data race, OpenMP and
C language, and MapReduce. In order to accomplish these goals,
each group is given a Raspberry PI kit (costs $59), a uniform work
environment, and a credit-card sized computer that plugs into a
computer monitor or TV and uses a standard keyboard and mouse.
The equipment has been paid by a grant funded by the Center for
Excellence in Teaching and Learning (CETL) at Georgia State Uni-
versity. The assignments ask the teams to explore the Raspberry
PI’s multicore architecture and use it to create programs for shared
memory parallelism using OpenMP and C. We provide details on
these assignments in the next subsection.
OpenMP has been chosen. because unlike complex parallel plat-
forms, it is designed to make it relatively easy to add parallelism
to existing sequential programs and write new parallel programs
A.A. Younis, R. Sunderraman, M. Metzler et al.
Fig. 3. Pre-requisite flowchart for required computer science courses.
from scratch. Raspberry PI, on the other hand, has been chosen
because components such as the processor, memory unit, storage
device, and others are clearly visible and that makes them readily
available for visual and tactile learners. Another motivation is that
our CSC 3210 teaches Intel X86 (CISC) architecture, and by using
the Raspberry PI, it gives students an exposure to ARM (RISC) ar-
chitecture. It is also similar to what they may encounter in today’s
ubiquitous mobile devices and help them explore its basic Instruc-
tion Set Architecture (IST) and compare it with Intel X86 in terms
of data movement, instruction encoding, immediate value repre-
sentation, and memory layout.
4.2. PBL module design
We now provide detail for four of the five stages of the PBL
module design, as shown in Fig. 2. This includes: 1) module de-
scription; 2) module integration; 3) team formation; and 4) mod-
ule development. The details for the fifth stage, module evaluation,
is provided in the following section.
Module Description: Each assignment includes a brief description
that includes the objectives and tasks to be carried out. Through
the series of five assignments, students will first learn and ap-
ply soft skills in terms of teamwork basics, decision making, task
identifications, planning and scheduling, cooperation and collab-
oration, conflict resolution, and communication. Then, equipped
with the basic soft skills, students will learn and apply parallel
programming concepts in terms of sequential and parallel com-
putation, multi-processor computer architectures (e.g. SISD, SIMD,
MISD, and MIMD), processes and threads, parallel programming
models, shared memory parallelism, data race, OpenMP and C lan-
guage.
PBL Module Integration: We integrate the PBL module into CSC
3210, Computer Organization and Programming, a core bridge
course and a pre-requisite to many of our senior-level courses
in the Computer Science Department at Georgia State University.
Fig. 3 depicts the sequence of required CS courses. CSC 3210 and
CSC 4350 are highlighted, as they are both utilized for this study,
with CSC 4350 providing longitudinal impact as described in Sec-
tion 6.3. Four sections of CSC 3210 were selected for this study,
two in Fall 2018 and two in Spring 2019. Each section had around
62 students enrolled: in Fall 2018, the first section had 16 females
and the second section had 10 females; in Spring 2019, the first
section had 16 females while the second section had 13 females.
We selected the sections so that each section was taught by the
154
Journal of Parallel and Distributed Computing 158 (2021) 151–163
same instructor and with the same instructional strategy incorpo-
rating the PBL module covering parallel programming principles
and soft skills. By keeping the instructor consistent, this eliminated
any variance due to teaching style or communication.
Team Formation: To form the teams, students in each section were
organized into thirteen diverse groups (up to five per group) based
on the following criteria: gender, system and programming experi-
ence, experience in group work, GPA, and technical writing expe-
rience. This information was collected via a survey at the start of
the class. These criteria are intended to balance groups in terms of
ability and assure a mixed gender and avoidance of predetermined
groups of friends. Having the instructor form teams based on pre-
determined criteria has been found to be more effective than when
students form their own team [34,13,39] especially when students
are not selecting their own project [41]. Once grouped, each team
elects a team coordinator and this role is to be rotated among
team members for each assignment. The team coordinator inter-
faces between the instructor and the team, turns in the documents,
reviews the returned assignments and ensures everyone under-
stands why any points were lost, how to correct any errors, and
identify, assign, and schedule tasks to the team members, as well
as monitor and report the progress of the assigned tasks.
PBL Module Development: After teams have been formed, five
project assignments, each of two weeks duration, are to be com-
pleted. In these assignments, we provide learning objectives, grad-
ing criteria, team policies, a peer rating form of team members’
contributions to the team, and the required tasks. Each assign-
ment includes the following components: Planning and Schedul-
ing (work breakdown structure: assignee name, email, assigned
task, duration in hours, dependency, due date, note), Collabora-
tion, Written Report, and Video Presentation. Each student must
participate in the group video, which must be 5-10 minutes long
and posted on YouTube. The following guide is used in each pre-
sentation to help students focus on their presentation: Introduce
yourself and your role; Identify your task for this assignment and
2-3 key things learned; How you will apply what you learned in
your next assignment, academic life (future classes), and in the
future job; What the best/most challenging/worst experience you
encountered.
For each assignment, the groups are provided with the one or
more of the following materials to do the assigned assignment:
• Teamwork Basics material,
• Raspberry PI Multicore architecture,
A.A. Younis, R. Sunderraman, M. Metzler et al.
• Shared Memory Parallel Patternlets in OpenMP,
• Introduction to Parallel Computing,
• CPU vs. SOC – The battle for the future of computing, and
• Introduction to Parallel Programming and MapReduce.
In the assignments that have programming in OpenMP and C
(Assignments 2–5), each group is required to include in the written
report what they have done and observed, including screenshots
and code snippets. Simply attaching screenshots and code snippets
without any explanation will not receive credits. They also need
to provide explanations for the observations that are interesting
or surprising. We now provide a brief description of the 5 assign-
ments shown in Fig. 1 and given over the course of the semester.
Assignment 1: Each group, using the Teamwork Basics mate-
rial [32], will first collaboratively-learn and apply the team Ground
Rules: work norms, facilitator norms, communication norms, meet-
ing norms, handling difficult behavior, and handling group prob-
lems. Next, they are to watch introductory/tutorial videos on
the teamwork technologies that will be utilized. The groups will
collaboratively-learn, apply and report how to utilize teamwork
technologies such as Slack, a messaging application to communi-
cate, GitHub, a social networking site for programmers to collab-
orate, create customized workflows, and share code, Online Word
processor (e.g. Google Docs) to collaborate and produce project as-
signments reports, and Videos and YouTube, to shoot, edit, and
upload videos to a YouTube channel to present their results.
Assignment 2: Using the Raspberry PI Multicore architec-
ture materials [29,28,27], Shared Memory Parallel Patternlets in
OpenMP [12], and Introduction to Parallel Computing [4], groups
will collaboratively-learn fundamental parallel computing princi-
ples based on the following questions: Identify the components on
the Raspberry PI B+? How many cores does the Raspberry Pi’s B+
CPU have? What is the difference between sequential and parallel
computation and identify the practical significance of each? Iden-
tify the basic form of data and task parallelism in computational
problems? Explain the differences between processes and threads?
What is OpenMP and what are OpenMP pragmas? What applica-
tions benefit from multi-core?
Next, each group will be given a Raspberry PI and a uniform
work environment. The groups are required to 1) download and
install the Operating System (RASPBIAN) Images on MicroSD, and
2) setup the Raspberry PI to connect with a monitor or a laptop.
Then, the groups will use the PI to create, compile, run, and modify
a parallel program that illustrates: 1) The fork-join programming
pattern, 2) A Single Program Multiple Data (SPMD) pattern for
shared memory parallelism using OpenMP and C language, and 3)
Shared memory concerns. In particular, students will see that by
sharing one bank of memory, programmers need to be a bit more
careful about declaring their variables (scope matter) to avoid the
data race problem.
Assignment 3: In addition to the materials in [4], [12], and [27],
groups will use CPU vs. SOC material [46] to answer the following
questions: What is: Task, Pipelining, Shared Memory, Communica-
tions, and Synchronization? Classify parallel computers based on
Flynn’s taxonomy (briefly describe each one of them)? What are
the Parallel Programming Models? List and briefly describe the
types of Parallel Computer Memory Architecture? What type is
used by OpenMP and why? Compare Shared Memory Model with
Threads Model? What is System On Chip (SOC)? Does Raspberry PI
use SOC? Explain what the advantages are of having a System on
a Chip rather than separate CPU, GPU, and RAM components?
Then, using the Raspberry PI, groups will create, compile, run,
and modify the following programs:
155
Journal of Parallel and Distributed Computing 158 (2021) 151–163
(1) Running Loops in Parallel: illustrates the use of OpenMP’s de-
fault parallel for loop in which threads iterate through equal
sized chunks of the index range.
(2) Scheduling of Parallel Loops (static and dynamic): illustrates
how to make OpenMP map threads to parallel loop iterations
in chunks of size one, two, and three.
(3) When Loops Have Dependencies: illustrates the OpenMP
parallel-for loop’s reduction clause.
Assignment 4: Using the materials in [4] and [12], groups will
first answer the following questions: What is the race condition?
Why race condition is difficult to reproduce and debug? How
can it be fixed? Compare the following: Collective synchronization
(barrier) with Collective communication (reduction), and Master-
worker with fork join. Next, using the Raspberry PI, groups will
create, compile, run, and modify the following programs:
(1) Integration Using the Trapezoidal Rule: illustrates the use of
parallel for loop, private, shared, and reduction clauses.
(2) Coordination: Synchronization with a Barrier: illustrates the
use of the OpenMP barrier command, using the command line
to control the number of threads.
(3) The Master-Worker Implementation Strategy: illustrates the
master-worker pattern in OpenMP.
Assignment 5: After reading the paper “Introduction to Parallel
Programming and MapReduce” [24], groups will answer the fol-
lowing questions: What are the basic steps (show all steps) in
building a parallel program (show at least one example)? What
is MapReduce? What is map and what is a reduce? Why MapRe-
duce (show an example for MapReduce)? Explain how MapReduce
model is executed? List and describe three examples that are ex-
pressed as MapReduce computations? When do we use OpenMP,
MPI, MapReduce (Hadoop), and why?
Given the material in [27], each group needs to first report their
understanding of the Drug Design and DNA problem and the algo-
rithmic strategy of the Drug Design and DNA in parallel using a
sequential, an OpenMP, and a C++11 Threads solutions. Using the
Raspberry PI, groups will create, compile, run, and modify the fol-
lowing programs:
(1) Apply a sequential, OpenMP, and a C++11 Threads solutions to
Drug Design and DNA problem.
(2) Measure the running time of each implementation, and an-
swer the following questions: Which approach is fastest? What
are the number of lines in each file (size of the program vs.
performance)? How does the C++11 implementation compare
to the OpenMP implementations?
(3) Increase the number of threads to 5 threads, what is the run
time for each?
(4) Increase the maximum ligand length to 7 and rerun each pro-
gram. What is the run time for each?
5. PBL module evaluation
In this section, we describe the overall approach for evaluation
of the PBL module. We provide the rubrics utilized for evaluating
individual modules. Fig. 4 provides a high-level view of the mod-
ule assessment, including the methods for data collection and data
analysis. We focus on the results from our data analysis in the next
section.
5.1. PBL grading for the course
The PBL module has been assigned 25% of the class overall
grade. The assigned grade weight has been equally distributed
A.A. Younis, R. Sunderraman, M. Metzler et al.
Fig. 4. PBL Module Assessment Overview.
Table 1
Rubric for grading an oral presentation.
Grade Criteria
Not meet basic requirements
Unsatisfactory (Points will be deducted according to the assignment
requirements)
Meet basic requirements, including but not limited to:
Fair 1. All members show up together (In the same room)
2. 3-8 minutes
3. Cover all the required contents in your video
4. Upload your video to channel correctly
5. The picture and voice are clear
1. Meet basic requirements
Good 2. Refer to notes less during your speech
3. More eye contacts with camera
4. Use diagrams whenever possible. A picture is worth a
thousand words.
1. Meet basic requirements
Excellent 2. Entirely or almost speak without notes
3. More eye contacts with camera
4. The expression is fluent and natural, the content is
detailed and related to your own experience
5. Good editing and music
6. All members are in a group discussion setting, rather than
speaking individually
7. Carry out the video content in the form of dialogue
across each of the five assignments. Every student who contributes
to the assignment will receive the team assigned grade. If a team
member refuses to cooperate or partially cooperated on an assign-
ment, a zero grade will be assigned for that assignment. If the
problem persists, the team should meet with the instructor so that
the problem can be resolved if possible, otherwise, the grade of
zeroes will be assigned for the remaining assignments.
Each assignment will be graded based on a rubric as shown
in Table 1, where criteria for grading an oral presentation is pro-
vided, and Table 2, where criteria for grading a written report is
illustrated. Again, the graded is performed for each assignment.
Besides the grade, feedback will be sent to the team coordinator
a week after the assignment due date. To assess an individuals’
performance, the students complete a quiz after each assignment
due date (five in total). Students also complete two tests, one in
the middle of the semester (midterm) and the other one at the
end of the semester (final exam).
5.2. PBL module assessment
A five-point scale survey proposed by [40] is used to collect
data and assess class emphasis and students’ growth in terms of
parallel programming and soft skills. We use surveys to analyze
the results of our study as grades are not always reflective of
156
Journal of Parallel and Distributed Computing 158 (2021) 151–163
knowledge gained [6]. The survey includes both “Class Emphasis”
and “Personal Growth” categories as a mechanism to judge how
effective the class is in developing specific skills. The survey con-
siders seven elements: teamwork, information gathering, problem
definition, idea generation, evaluation and decision making, imple-
mentation, and communication.
Class Emphasis scores are described as 1: Did not discuss, 2:
Minor emphasis, 3: Some emphasis, 4: Significant emphasis, and
5: Major emphasis. Personal Growth scores are described as 1: I
did not use this skill within this class, 2: I used previous skills and
had little growth, 3: I grew some and gained a few new skills, 4:
I experienced significant growth and added several skills, and 5: I
experienced tremendous growth and added many new skills. The
first item in each of the categories in the survey is the basic defini-
tion of that element. For instance, as shown in Fig. 4, teamwork is
defined as “Individuals participate effectively in groups or teams.”
The next items in a category are components or performance
indicators of that specific element. For teamwork, there are four
characteristic items (see Fig. 5): Individuals understand their own
and other member’s styles of thinking and how they affect team-
work, Individuals understand the different roles included in ef-
fective teamwork and responsibilities of each role, Individuals
use effective group communication skills: listening, speaking, vi-
sual communication, and Individuals cooperate to support effective
teamwork. The survey is to be taken twice, once at mid-semester
and the other at the end of the semester as shown in Fig. 1.
Once the data is collected, we use descriptive statistics, a T-test,
Cohen’s d (Effect Size) [21], Pearson Correlation, and Ranking of
students’ growth (Composite Score) [40] to determine the impact
of the intervention. Cohen’s d is used to measure how significant
the effect of the intervention is. Cohen suggested that d = 0.2 be
considered a ‘small’ effect size, 0.5 represents a ‘medium’ effect
size and 0.8 a ‘large’ effect size. In other words, if two groups’
means differ by less than 0.2 standard deviations, the difference
is trivial although it is statistically significant. We also used the
Composite Score to measure the Ranking of students’ growth. The
Composite Score is calculated by averaging the ‘definition’ and the
‘overall’ performance average of individual components. The Com-
posite Score is used because it provides information from two
different perspectives: global from the definition and focused on
the components.
6. Results
In this section, we will first present the descriptive statistics of
our sample data and then explain the analysis of the results. Next,
we assess the effectiveness of soft skills as learned through the PBL
approach.
6.1. Descriptive statistics
The statistical analyses are based on a sample of 247 computer
science students enrolled at a large public university, during Fall
2018 (124 students, 26 females) and Spring 2019 (123 students,
29 females). Each student surveyed was enrolled in a project based
in CSC 3210, Computer Organization and Programming course. The
survey was first distributed at the midpoint of the semester. The
same survey was then distributed to the students at the end of
the semester (see Fig. 1). Of the surveys collected during Fall 2018
and Spring 2019, 192 (77.73%) were from male students and 55
(22.26%) were from female students.
6.2. Results analysis and validation
Hypothesis 1. There is a difference in emphasis on parallel pro-
gramming and soft skills between the first and second parts of
A.A. Younis, R. Sunderraman, M. Metzler et al. Journal of Parallel and Distributed Computing 158 (2021) 151–163

Table 2
Rubric for Grading a Presentation.

Grade Criteria
Unsatisfactory 1. Not meet basic requirements (Points will be deducted according to the assignment requirements)
2. Plagiarism

Fair Meet basic requirements, including but not limited to:

1. Cover all the required contents (title page, report sections, appendix, links, etc.) PS: The cover page does not need to be numbered.
2. Correct format (text justified, fonts, page numbers, etc.)
3. Correct submission (no delay, PDF format, only coordinator need to submit, etc.)

Good 1. Meet basic requirements

2. Parallel Programming Foundation (Task 3) Answers are basically correct
Use your own words
3. Parallel Programming (Task 3) The experimental results are basically correct
Provide necessary screenshots, description of the phenomena and principles behind

Excellent 1. Meet basic requirements

2. Parallel Programming Foundation (Task 3)
3. Parallel Programming (Task 3)
The answers are exactly or almost correct
Use your own words
Answers are well-organized and easy to read
The original code and experimental results are exactly or almost correct
Detailed description of the phenomena and correct description of the principle behind
The experimental screenshots are clear and sufficient for every step

Fig. 5. Example of Team Design Skills Growth Survey [40].

Table 3
T-test: Class Emphasis and Personal Growth.

Mean Difference t N p-value

Fall 2018 Class Emphasis −0.10 −2.63 124 0.039
Personal Growth −0.20 −5.11 124 0.002
Mean Difference t N p-value
Spring 2019 Class Emphasis −0.40 −0.12 123 0.0000000887
Personal Growth −0.30 −19.33 123 0.0000012407

the semester. A T-test to determine if there was a difference be-
tween the first part of the semester and the second part regarding
class emphasis (variable1: class- emphasis1 and variable2: class-
emphasis2) on parallel programming principles and soft skills was
performed. The two variables were created by averaging all class
emphasis question scores on the two surveys respectively. The re-
sults of the T-test, as shown in Table 3, indicate that the class em-
phasis comparison was significant during the Fall 2018 and Spring
2019 semesters. This shows that the instructor did emphasize the
skills more during the second half of the semester.
To measure how big the effect of the intervention (course em-
phasis), Cohen’s d (Effect Size) was used. More details about Co-
hen’s d can be found in Section 5 in the Module Assessment sub-
section. Cohen’s d for course emphasis was calculated using the
following formula: Cohen’s d = (M2 - M1) / SDpooled, where, SD-
√
pooled = ((SD12 + SD22) / 2). It should be noted that Cohen’s d
= 0.2 is considered a ‘small’ effect size, 0.5 represents a ‘medium’
effect size and 0.8 a ‘large’ effect size. The results of the Cohen’s d
as shown in Table 4 indicate that the effect of the course emphasis
during Fall 2018 represents a ‘medium’ effect size and the group
means differ by 0.5 standard deviations whereas the effect of the
class emphasis during Spring 2019 represents a ‘high’ effect size
and the group means differ by 0.9 standard deviations. Thus, the
difference is not trivial for both semesters. This means that if the
means of two groups are within 0.2 standard deviations or more,
the difference is trivial, even if it is statistically significant.
Hypothesis 2. By incorporating project-based learning, the stu-
dents acquire personal growth and improvement on their parallel
programming and soft skills. A second T-test to determine if there
was a difference between the first part of the semester and the
second part regarding students acquiring personal growth on par-
allel programming and soft skills was performed. The two variables
were created by averaging all class emphasis question scores on
the two surveys respectively. The results of the T-test, as shown in
Table 3, indicate that students used more soft and parallel skills
after the first survey than before the first survey during the Fall
2018 and Spring 2019 semesters. We also performed a second
Cohen’s d (Effect Size) measure. The results of the Cohen’s d as

157
A.A. Younis, R. Sunderraman, M. Metzler et al. Journal of Parallel and Distributed Computing 158 (2021) 151–163

Table 4
Cohen’s d of Course Emphasis.

First Half Survey Second Half Survey

Fall 2018 Mean (M): 4.023068a 4.124365a
Standard deviation (s): 0.232416 0.172052
Sample size (n): 124 124
Cohen’s d = (4.124365 - 4.023068) / 0.204474 = 0.50
First Half Survey Second Half Survey
Spring 2019 Mean (M): 3.4391 3.839002
Standard deviation (s): 0.420598 0.434897
Sample size (n): 123 123
Cohen’s d = (3.839002- 3.439100) / 0.427747 = 0.93
a
(4: Significant emphasis and 5: major emphasis [40].)

Table 5
Cohen’s d (Effect Size) of Personal Growth.

First Half Survey Second Half Survey

Fall 2018 Mean (M): 3.81b 4.01b
Standard deviation (s): 0.262204 0.198497
Sample size (n): 124 124
Cohen’s d = (4.01 - 3.81) / 0.232542 = 0.86
First Half Survey Second Half Survey
Spring 2019 Mean (M): 3.33 3.65
Standard deviation (s): 0.428950 0.420807
Sample size (n): 123 123
Cohen’s d = (3.653991- 3.325122) /0.424878 = 0.77
b
(3: I grew some and gained a few new skills, and 4: I experienced a significant growth and
added several skills [40].)

Table 6
Final grade average (4 point scale) in Parallel and Distributed Computing (CSC 4310)
course.
Students that had the standard Students that had the PBL
approach for CSC 3210 approach for CSC 3210
GPA Number of students GPA Number of students
Fall 2019 3.243 20 3.947 16
Fall 2020 3.104 24 3.633 8 skills did increase when greater emphasis is placed on parallel pro-
shown in Table 5 indicate that the effect of incorporating project-
based learning on students soft and parallel programming skills
represents a ‘large’ effect size and the group means differ by 0.8
standard deviations during Fall 2018 semester and it represents a
‘large’ effect size and the group means differ by 0.7 standard de-
viations during Spring 2019 semester. Thus, the difference is not
trivial for both semesters.
To further validate the impact of the PBL approach, we wanted
to determine how it impacted student performance in an elective
course focused on PDC concepts. Specifically, we considered the
Parallel and Distributed Computing course, CSC 4310. This is a se-
nior elective course that is offered every fall semester and has the
CSC 3210 course as a prerequisite.
One component of Hypothesis 2 is that by incorporating the
PBL modules, students will improve their performance on paral-
lel computing skills. As the PBL modules were implemented in Fall
2018 and Spring 2019, we investigated student performance in the
Fall 2019 and Fall 2020 offerings of CSC 4310. Table 6 presents
a comparison of final grades (on a 4 point scale) in these two
sections for the undergraduate students that took the CSC 3210
prerequisite with the PBL modules incorporated as compared to
those with the standard approach. The data clearly shows that the
experience with the project modules had a significant impact on
their grasp of the PDC concepts. In particular, we found that the
16 students in Fall 2019 and 8 students in Fall 2020 that went
through the PBL approach for learning PDC topics earned an aver-
age final grade in their CSC 4310 that was above 0.5 points that of
the 20 and 24 students in Fall 2019 and Fall 2020, respectfully that
did not have the PBL approach in their CSC 3210 course. It is also
interesting to note that there were two separate instructors for the
CSC 4310 course across the two semesters considered in Table 6.
Hypothesis 3. Students’ growth on soft and parallel programming
gramming and soft skills. To determine if the class emphasis on
parallel programming and soft skills is associated with students’
application of these skills, a Pearson correlation was performed on
teamwork, information gathering, problem definition, idea gener-
ation, evaluation and decision making, implementation, and com-
munication between class emphasis and personal growth for the
first and second survey.
Each skill score was created by averaging all question scores
under each skill. Table 7 shows the Pearson correlation between
class emphasis and personal growth for the data collected in the
Fall 2018 and Spring 2019 semesters. As shown in the first part in
Table 7, all correlations in the Fall 2018 semester are positive and
highly statistically significant at p < 0.001 level. The correlations
strength falls within the moderate range (±0.40–±0.70) according
to Guilford [9], for all skills except for Evaluation, Decision Mak-
ing and Teamwork. The Evaluation and Decision Making skills fall
within the high range at r = 0.73 (±0.70–±0.90). Teamwork, at
only the first half of the semester, falls within the low range at r
= 0.38 (±0.20–±0.40), which is a measurable but small relation-
ship [21].
On the other hand, the second part in Table 6 shows all corre-
lations in the Spring 2019 semester are positive and highly statis-
tically significant at p < 0.001 level. The correlation strength falls
within the high range (±0.70–±0.90) for all skills except for Infor-
mation Gathering and Problem Definition that both fall within the
moderate range (±0.40–±0.70) and Teamwork falls within the low
range at r = 0.38 (±0.20–±0.40), which again is a measurable but
small relationship [21].

158
A.A. Younis, R. Sunderraman, M. Metzler et al. Journal of Parallel and Distributed Computing 158 (2021) 151–163

Table 7
Pearson Correlation Between Class Emphasis and Personal Growth.

First Half Survey Second Half Survey

r p-value N r p-value N
Fall 2018 Teamwork 0.38 p < 0.001c 124 0.47 p < 0.001 124
Information Gathering 0.66 p < 0.001 124 0.68 p < 0.001 124
Problem Definition 0.62 p < 0.001 124 0.61 p < 0.001 124
Idea Generation 0.64 p < 0.001 124 0.57 p < 0.001 124
Evaluation and 0.73 p < 0.001 124 0.73 p < 0.001 124
Decision Making
Implementation 0.59 p < 0.001 124 0.61 p < 0.001 124
Communication 0.67 p < 0.001 124 0.67 p < 0.001 124

Spring 2019 Teamwork 0.38 p < 0.001 123 0.73 p < 0.001 123
Information Gathering 0.69 p < 0.001 123 0.85 p < 0.001 123
Problem Definition 0.55 p < 0.001 123 0.74 p < 0.001 123
Idea Generation 0.89 p < 0.001 123 0.89 p < 0.001 123
Evaluation and 0.87 p < 0.001 123 0.90 p < 0.001 123
Decision Making
Implementation 0.92 p < 0.001 123 0.90 p < 0.001 123
Communication 0.91 p < 0.001 123 0.81 p < 0.001 123
c
As all P values are very small (e.g., p-value for teamwork, first half survey is 1.52716E-22), we reported
them using the inequality p < 0.001 [18].

Table 8
Ranking of Students’ Perception of the Course Emphasis.

Ranking First Half Survey (average) Second Half Survey (average)

Fall 2018 1 Teamwork: 4.38 Teamwork: 4.41
2 Implementation: 4.16 Implementation: 4.25
3 Problem Definition: 4.09 Problem Definition: 4.19
4 Idea Generation: 4.04 Idea Generation: 4.09
5 Communication: 4.02 Communication: 4.03
6 Information Gathering: 3.81 Evaluation and Decision Making 3.98
7 Evaluation and Decision Making: 3.66 Information Gathering: 3.91

Spring 2019 1 Teamwork: 3.99 Teamwork: 4.45

2 Problem Definition: 3.92 Problem Definition: 4.31
3 Information Gathering: 3.69 Information Gathering: 4.10
4 Implementation: 3.25 Implementation: 3.65
5 Communication: 3.14 Idea Generation: 3.51
6 Idea Generation: 3.12 Communication: 3.49
7 Evaluation and Decision Making: 2.96 Evaluation and Decision Making: 3.38

We also analyzed the students’ ranking of course emphasis and
perception of personal growth using the Composite Score [40]. The
Composite Score is calculated by averaging the ‘definition’ and the
‘overall’ performance average of individual components. More de-
tails about the Composite Score can be found in Section 5 in the
Module Assessment subsection. The data for Fall 2018 and Spring
2019 semesters were analyzed by ranking the items relative to
each other and looking at growth as compared to emphasis and
examining components that were not emphasized.
Table 8 shows the results for the Ranking of students’ percep-
tion of the course emphasis. Students’ responses show that the
module placed a heavy emphasis on Teamwork during both the
Fall 2018 and Spring 2019 semesters. They also perceived that
all of the elements had a strong class emphasis, but ranked the
emphasis for all of the elements higher in the second half. The el-
ements in the first half of the term during Fall 2018 were ranked
in a similar manner. During the second half, teamwork remains
significantly out in front of both the Fall 2018 and Spring 2019
semesters. Information gathering fell closely behind the Evalua-
tion and Decision Making in Fall 2018 whereas Evaluation and
Decision Making fell closely behind Communication during Spring
2019. However, there was not much change from the first half sur-
vey to the second half for Fall 2018. For this reason, we provided
extra support materials for the assignments in Spring 2019. The
impact of this adjustment resulted in a higher perceived emphasis
in the second half survey for Spring 2019.
The results of the skills Ranking in Table 9 show that students
indicate they had a more selective growth in each of the elements
during the first half of the term, which is demonstrated by a large
spread between the top and bottom scores of the elements and by
the significantly different values for each of the elements. Students
indicate that “Teamwork” skills were their highest growth while
“Evaluation and Decision Making” was the least. During the sec-
ond half of the term, the students perceived that the growth was
more equal with “Teamwork” and “Implementation” the top-rated
growth items.
6.3. Assessing the effectiveness of soft skills
To assess the improvement made by the addition of the group
project, a questionnaire was provided to 27 students in CSC 4350
software engineering students during the Summer 2019 semester.
Fig. 3 shows where CSC 4350 fits in the sequence of required CS
courses. Among the surveyed students, thirteen students who took
CSC 4350 software engineering had training in soft skills in the
CSC 3210 course whereas fourteen students did not have training
in soft skills. The results of this survey are discussed as follows.
(1) Students who took CSC 3210 and learned teamwork skills
needed to function successfully in group projects were asked
how useful they found the project work experience they had
in CSC 3210 while working on their Software Engineering
project during Summer 2019. The results are shown in Fig. 6.

159
A.A. Younis, R. Sunderraman, M. Metzler et al. Journal of Parallel and Distributed Computing 158 (2021) 151–163

Table 9
Ranking of Student Perception of Personal Growth.

Ranking First Half Survey (average) Second Half Survey (average)

Fall 2018 1 Teamwork: 4.14 Teamwork: 4.33
2 Implementation: 4.05 Implementation: 4.22
3 Problem Definition: 3.89 Problem Definition: 4.00
4 Idea Generation: 3.84 Idea Generation: 3.97
5 Communication: 3.83 Communication: 3.97
6 Information Gathering: 3.62 Information Gathering: 3.84
7 Evaluation and Decision Making: 3.36 Evaluation and Decision Making: 3.77

Spring 2019 1 Teamwork: 3.89 Teamwork: 4.26

2 Problem Definition: 3.79 Information Gathering: 4.06
3 Information Gathering: 3.61 Problem Definition: 3.91
4 Implementation: 3.13 Communication: 3.52
5 Idea Generation: 3.00 Implementation: 3.37
6 Communication: 2.98 Evaluation and Decision Making: 3.32
7 Evaluation and Decision Making: 2.85 Idea Generation: 3.14

Fig. 6. How useful did you find the project work experience you had in CSC 3210 while working on your Software Engineering project?

Fig. 7. How would you value the importance of the following soft skills? (For interpretation of the colors in the figure(s), the reader is referred to the web version of this
article.)

The used rating scaled is: Very Important, Important, Moder-
ately Important, Slightly Important, and Not important. Eleven
students have found the soft skill experience to be very impor-
tant or important whereas two have found it to be moderately
important.
When students were asked to value the importance of the fol-
lowing soft skills technologies: Collaboration using GitHub, Com-
munication using Slack, Writing using Google Doc or MS-Word,
Presentation using YouTube, Handling team conflicts using the
guidelines, their responses were as shown in Fig. 7. As can be seen,
all the selected technologies have been rated as either very impor-
tant or important. One student rated slack as not important and
one other student rated conflicts guideline as not important.
Students who took CSC 3210 and do not learn the teamwork
skills needed to function successfully in group projects were asked

160
A.A. Younis, R. Sunderraman, M. Metzler et al.
Fig. 8. Would exposure to soft skills such as giving presentations; writing skills; communication; and handling team conflicts prior to taking software engineering class be
important while working on your software engineering class project?
whether exposure to soft skills such as giving presentations, writ-
ing skills, communication, and handling team conflicts prior to
taking the Software Engineering course to be important while
working on your Software Engineering project. Fig. 8 shows that
thirteen out of fourteen students rated the exposure to the soft
skills earlier as very important or important and only one student
rated as moderately important.
7. Discussion
Our results show that incorporating Project Based Learning
(PBL) had a direct effect on a student’s personal growth and im-
provement on their parallel programming and soft skills. This was
shown with a significant p-value = 0.002, and a ‘large’ effect
size Cohen’s d = 0.86 during Fall 2018 case study, and p-value
= 0.0000012, and a ‘large’ effect size Cohen’s d = 0.77 during
Spring 2019 case study. Early exposure to Parallel and Distributed
Computing (PDC) using a PBL approach also directly impacted the
students’ performance in a subsequent PDC senior elective course.
These results are especially important as Computer Science pro-
grams prepare graduates for contemporary workplaces. If students
do not value Project Based Learning opportunities, it is likely that
they are not adequately prepared (particularly in terms of team
orientation) to successfully negotiate with others in their profes-
sional positions.
In addition, as the CSC 3210 class is taken at the midpoint of
the academic program, incorporating Project Based Learning can
serve as a mini-capstone that provides an opportunity for the stu-
dent to learn the soft skills required to be successful and then hone
these skills in the final capstone courses, such as software engi-
neering. Furthermore, with the provided material, guidance from
the instructor, and the use of collaborative learning, students can
learn by themselves and apply the fundamentals of parallel pro-
gramming and soft skills without the need for lectures, labs, or
workshops. This enables us to retain the full content of the CSC
3210 course.
The results of the T-test, as shown in Table 3, indicate that
the class emphasis comparison was more significant during Spring
2019 semester than Fall 2018 semester. This could be attributed
to the experience the course instructor and his teaching assistance
gained from the first experience in Fall 2018, as teaching experi-
ence is positively associated with student achievement gains [22].
Moreover, social interaction between students who took the course
in Fall 2018 and Spring 2019 could be another factor.
The Pearson correlation overall results between the class Em-
phasis and students’ Personal Growth in the first and second half
161
Journal of Parallel and Distributed Computing 158 (2021) 151–163
of the semester (Table 6) were positive and highly statistically sig-
nificant at p < 0.001 level for all the skills during Fall 2018 and
Spring 2019 case studies. The correlations strength for five skills
falls within a moderate range whereas Evaluation and Decision
Making as well as Teamwork at only the first half of the semester
skills fall within the high range and low range respectively dur-
ing the Fall 2018 case study. On the other hand, all correlations
during Spring 2019 semester the strength of the correlations falls
within the high range for all skills except for Information Gather-
ing and Problem Definition which fall within the moderate range
and Teamwork which fall within the low range at r = 0.38, which
is a definite but small relationship.
Overall, the correlation results indicate that the more the in-
structor emphasized those skills in class, the more students ap-
plied those skills. Compared to the other skills, it was observed
that the Teamwork skill scores the lowest in terms of correla-
tion strength in the first half of semester 0.38 and the second
half of semester 0.47. To improve this relationship, we provided
more material and incorporate one or two more exercises about
Teamwork basics in other assignments instead of having Teamwork
basics incorporated in only one assignment. We compared the re-
sults after this addition with the current results (Spring 2019) and
observed some improvements between Course Emphasis and Per-
sonal Growth, 0.38 first have and 0.73 the second half.
The results from skills ranking of student perception of the
course emphasis (Table 7) and personal growth (Table 8) show
as students’ knowledge of the process increased, they are better
able to judge the growth in their learning and are more discerning
about how their learning in the second half of the course relates to
the elements and components. This was shown by students’ per-
ception of a greater emphasis and a greater growth during the
second half of the term compared to the first half of the semester
even though the class teaching methodology used in each half
were similar.
It should be noted that students’ perception of course emphasis
is almost always higher than perceived growth and this has also
been found in other comparable results [40]. However, the one ex-
ception was in “Implementation” which had almost no differences
between perceived growth and class emphasis during the second
half (0.03). This can be attributed to the fact that there was a
larger growth in implementation skills during the second half of
the semester than in the first half. This is expected as students
had developed more parallel programs (four programs) in the sec-
ond half than in the first half where students had only developed
one program. According to [40], only if the difference between per-
ceived emphasis and perceived growth is larger than 0.2 should an
A.A. Younis, R. Sunderraman, M. Metzler et al.
effort be initiated to modify the course design and delivery to pro-
mote higher levels of skill development.
Threats to validity: We used a self-assessment five-point scale sur-
vey to collect data and assess class emphasis and students’ growth
in terms of parallel programming and soft skills. Although the use
of self-assessment will lead students to be able to learn inde-
pendently of the teacher, the accuracy of the data obtained from
self-assessment could threat the validity of the results. As much
of social science relies on self-assessment data, there has been
extensive research regarding the validity of self-reported data. Re-
searchers have found that self-assessment data are accurate when
individuals understand the questions and when there is a strong
sense of anonymity and little fear of reprisal [31]. To that end, stu-
dents in this experiment have been provided with training session
on how to use the survey in assessing their own learning. Besides,
all students in our experiments have filled in the survey anony-
mously.
One other concern is that we do not have another class to use
as a comparison study as to how students learn the parallel com-
puting and soft skills concepts without the PBL approach. As these
are not concepts typically taught in the CSC 3210 course, there is
not a means for comparison.
8. Conclusion
In this study, we investigated whether incorporating a Project
Based Learning module into the CSC 3210 Computer Organization
and Programming curriculum could help develop students’ parallel
programming and soft skills. In this module, comprising of five as-
signments, each of a two-week duration, 247 students were split
into 51 diverse groups (four or five per group). Working in groups,
they collaboratively learned and applied parallel programming con-
cepts and soft skills. Our results show that incorporating a Project
Based Learning module had a significant and direct effect on a
student’s personal growth and improvement on their parallel pro-
gramming and soft skills during two case studies, Fall 2018 and
Spring 2019. The impact of the project based approach to teach
parallel computing concepts carried over into a senior PDC course,
such that the average final grade for was higher by at least 0.5 out
of 4.0 points for the students in this study.
Our efforts show that practical projects, team learning and team
work are beneficial ways to cover parallel programming. The side
benefit of this approach demonstrates that this new topic can be
introduced outside of the lecture time and takes away the need to
reduce any existing content coverage in a course. By having stu-
dents work in groups outside of class time, they demonstrated the
ability to apply newly developed parallel programming skills and
soft skills. The results are promising, as it provides a means to
properly expose students to critical PDC concepts. With restric-
tions to the total number of credit hours we can require of our
students, the PBL approach would be a feasible way to revisit our
earlier work of attempting to interweave PDC concepts through-
out our curriculum [5] and not needing to add another required
course.
We have observed that the module emphasis on the Team-
work skills needed to be improved. Thus, in Spring 2019, we in-
corporated tasks about Teamwork basics in assignments two to
five. Our results show a better improvement compared to the
results from Fall 2018. We also developed project rubrics result-
ing in fewer emails and questions during office hours about the
project expectations. For future semesters, we also plan to incor-
porate other measures of assessment of student learning, including
quizzes and/or including questions on exams. This would provide
for a more objective measure of evaluation. Currently, we are us-
ing surveys, which is a subjective measure with possibly larger
162
Journal of Parallel and Distributed Computing 158 (2021) 151–163
variation due to the self-reporting. As students demonstrated a
strong ability to write parallel programs in multicore processors
and shared memory using OpenMP and C, we plan to extend
the module to include writing code for multicore processors and
distributed memory using Message Passing Interface (MPI) and C
language. This would provide students with more flexibility in de-
termining the correct memory architecture to use to solve a variety
of problems. Our intent is to use the material provided by CSinPar-
allel [12] and by [35] in future semesters.
CRediT authorship contribution statement
Awad A. Younis: Conceptualization, Funding acquisition, Inves-
tigation, Methodology, Writing – original draft. Rajshekhar Sun-
derraman: Validation, Writing – review & editing. Mike Metzler:
Formal analysis. Anu G. Bourgeois: Conceptualization, Validation,
Writing – review & editing.
Declaration of competing interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared to
influence the work reported in this paper.
Acknowledgment
We would like to thank the Center for Excellence in Teaching
and Learning (CETL) at Georgia State University for a Mini-Grant
that enabled the purchase of Raspberry PI kits that were a key
component to the success of developing this study.
References
[1] OpenMP, https://www.openmp.org/, 2021.
[2] F. Ahmed, L.F. Capretz, P. Campbell, Evaluating the demand for soft skills in
software development, IT Prof. 14 (1) (Jan.-Feb. 2012) 44–49, https://doi.org/
10.1109/MITP.2012.7.
[3] Ashish Amresh, Ryan Anderson, Parallel Programming Using Games: A Hands-
on Approach, AK Peters, Ltd., 2014.
[4] B. Barney, Introduction to parallel computing, Lawrence Livermore National
Laboratory, [Online]. Available, https://computing.llnl.gov/tutorials/parallel_
comp/. (Accessed 31 August 2020).
[5] A.G. Bourgeois, S. Prasad, Y. Zhu, J. Bhola, Y. Pan, Incorporating PDC topics
throughout the undergraduate computer science curriculum, in: Proceedings of
the 2nd NSF/TCPP Workshop on Parallel and Distributed Computing Education,
EduPar, 2012.
[6] Susan M. Brookhart, Thomas R. Guskey, Alex J. Bowers, James H. McMillan, Jef-
frey K. Smith, Lisa F. Smith, Michael T. Stevens, Megan E. Welsh, A century of
grading research: meaning and value in the most common educational mea-
sure, Rev. Educ. Res. 86 (4) (2016) 803–848.
[7] Buck Institute for Education (BIE), What is project based learn-
ing?, [Online]. Available, http://www.bie.org/images/uploads/general/
20fa7d42c216e2ec171a212e97fd4a9e.pdf. (Accessed 31 August 2020).
[8] Andrea C. Burrows, Mike Borowczak, Hardening Freshman engineering student
soft skills, in: Session W1A First Year Engineering Experience (FYEE) Confer-
ence, 2017, pp. 1–5.
[9] Manuel I. Capel, Antonio J. Tomeu, Alberto G. Salguero, Teaching concurrent
and parallel programming by patterns: an interactive ICT approach, J. Parallel
Distrib. Comput. 105 (2017) 42–52.
[10] Lori Carter, Ideas for adding soft skills education to service learning and cap-
stone courses for computer science students, in: Proceedings of the 42nd ACM
Technical Symposium on Computer Science Education, 2011, pp. 517–522.
[11] Rohit Chandra, Leo Dagum, David Kohr, Ramesh Menon, Dror Maydan, Jeff Mc-
Donald, Parallel Programming in OpenMP, Morgan Kaufmann, 2001.
[12] CSinParallel, Shared memory parallel patternlets in OpenMP [Online]. Avail-
able, http://selkie.macalester.edu/csinparallel/modules/Patternlets/build/html/
SharedMemory/OpenMP_Patternlets.html. (Accessed 31 August 2020).
[13] Huseyin Ergin, Instructor-formed capstone teams based on interest and techni-
cal experience: the road to success, J. Comput. Sci. Coll. 35 (5) (2019) 37–49.
[14] Moti Frank, Ilana Lavy, David Elata, Implementing the project-based learning
approach in an academic engineering course, Int. J. Technol. Des. Educ. 13 (3)
(2003) 273–288.
[15] D. Giménez, A practical parallel programming course based on problems of the
Spanish parallel programming contest, Proc. Comput. Sci. 80 (2016) 1978–1988.
A.A. Younis, R. Sunderraman, M. Metzler et al.
[16] Global smartphone operating system market share 2014-2023 | Statista,
https://www.statista.com/statistics/277048/global-market-share-forecast-of-
smartphone-operating-systems/. (Accessed 31 August 2020).
[17] Daniel González-Morales, Luz Marina Moreno De Antonio, José Luis Roda Gar-
cía, Teaching “soft” skills in software engineering, in: 2011 IEEE Global Engi-
neering Education Conference, EDUCON, IEEE, 2011, pp. 630–637.
[18] Sander Greenland, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles
Poole, Steven N. Goodman, Douglas G. Altman, Statistical tests, P values, con-
fidence intervals, and power: a guide to misinterpretations, Eur. J. Epidemiol.
31 (4) (2016) 337–350.
[19] William Gregory Johnson, Raj Sunderraman, Anu G. Bourgeois, Teaching Strate-
gies in Software Engineering Towards Industry Interview Preparedness, No.
4665, EasyChair, 2020.
[20] A.C.M. Joint, IEEE-CS task force on computing curricula, Comput. Sci. Curricula
(2013).
[21] Guilford Joy Paul, Fundamental statistics in psychology and education, 1950.
[22] Tara Kini, Anne Podolsky, Does teaching experience increase teacher effective-
ness? A review of the research, Learning Policy Institute, 2016.
[23] John Lamp, Chris Keen, Cathy Urquhart, Integrating professional skills into the
curriculum, in: ACM International Conference Proceeding Series, vol. 1, 1996,
pp. 309–316.
[24] Ren Li, Haibo Hu, Heng Li, Yunsong Wu, Jianxi Yang, MapReduce parallel pro-
gramming model: a state-of-the-art survey, Int. J. Parallel Program. 44 (4)
(2016) 832–866.
[25] Shaosong Li, Shivakant Mishra, Optimizing power consumption in multicore
smartphones, J. Parallel Distrib. Comput. 95 (2016) 124–137.
[26] Elavarasi Manogaran, Act-pbl: an adaptive approach to teach multi-core com-
puting in university education, in: 2013 IEEE Fifth International Conference on
Technology for Education, t4e 2013, IEEE, 2013, pp. 19–23.
[27] Suzanne J. Matthews, Joel C. Adams, Richard Brown, Elizabeth Shoop, Teaching
parallel computing with OpenMP on the Raspberry Pi, in: Proceedings of the
2017 ACM SIGCSE Technical Symposium on Computer Science Education, 2017,
p. 741.
[28] Suzanne J. Matthews, Joel C. Adams, Richard A. Brown, Elizabeth Shoop,
Portable parallel computing with the raspberry pi, in: Proceedings of the 49th
ACM Technical Symposium on Computer Science Education, 2018, pp. 92–97.
[29] Suzanne J. Matthews, Joel C. Adams, Richard A. Brown, Elizabeth Shoop, Ex-
ploring parallel computing with OpenMP on the Raspberry Pi, in: Proceedings
of the 50th ACM Technical Symposium on Computer Science Education, 2019,
p. 1234.
[30] Gerardo Matturro, Florencia Raschetti, Carina Fontán, Soft skills in software de-
velopment teams: a survey of the points of view of team leaders and team
members, in: 2015 IEEE/ACM 8th International Workshop on Cooperative and
Human Aspects of Software Engineering, IEEE, 2015, pp. 101–104.
[31] Junaidi Mistar, A study of the validity and reliability of self-assessment, Teflin
J. 22 (1) (2011) 45–58.
[32] MITOPENCOURSEWARE, Lecture notes, Sloan communication program, 15
8 2012. [Online]. Available, https://ocw.mit.edu/courses/sloan-school-of-
management/15-279-management-communication-for-undergraduates-fall-
2012/lecture-notes/MIT15_279F12_tmwrkBasics.pdf. (Accessed 31 August
2020).
[33] Philipp Neumann, Christoph Kowitz, Felix Schranner, Dmitrii Azarnykh, Inter-
disciplinary teamwork in HPC education: challenges, concepts, and outcomes,
J. Parallel Distrib. Comput. 105 (2017) 83–91.
[34] Barbara Oakley, Richard M. Felder, Rebecca Brent, Imad Elhajj, Turning student
groups into effective teams, J. Stud. Centered Learn. 2 (1) (2004) 9–34.
[35] Sushil Prasad, Anshul Gupta, Arnold Rosenberg, Alan Sussman, Charles Weems,
Topics in Parallel and Distributed Computing, Morgan Kaufmann, 2015.
[36] Sushil K. Prasad, Sheikh Ghafoor, Christos Kaklamanis, Ramachandran
Vaidyanathan, VSI: Edu*-2016-Keeping up with technology: teaching paral-
lel, distributed and high-performance computing, J. Parallel Distrib. Comput.
118 (1) (2018).
[37] Susan H. Pulko, Samir Parikh, Teaching ‘soft’ skills to engineers, Int. J. Electr.
Eng. Educ. 40 (4) (2003) 243–254.
[38] Marcel M. Robles, Executive perceptions of the top 10 soft skills needed in
today’s workplace, Bus. Commun. Q. 75 (4) (2012) 453–465.
[39] Rebecca H. Rutherfoord, Using personality inventories to help form teams for
software engineering class projects, ACM Sigcse Bull. 33 (3) (2001) 73–76.
[40] Brian Sauser, Using self assessment to evaluate the effectiveness of an engi-
neering management course with cross functional teams, in: Proceedings of
the 2005 American Society for Engineering Education Annual Conference & Ex-
position, 2005, pp. 10–1426.
163
Journal of Parallel and Distributed Computing 158 (2021) 151–163
[41] Bridget M. Smyser, B. Kris Jaeger, How did we end up together? Evaluating
success levels of student-formed vs. instructor-formed capstone teams, AGE 26
(2015) 1.
[42] Marc Snir, William Gropp, Steve Otto, Steven Huss-Lederman, Jack Dongarra,
David Walker, MPI–the Complete Reference: The MPI Core, vol. 1, MIT Press,
1998.
[43] Awad A. Younis, Rajshekhar Sunderraman, Mike Metzler, Anu G. Bourgeois,
Case study: using project based learning to develop parallel programing and
soft skills, in: 2019 IEEE International Parallel and Distributed Processing Sym-
posium Workshops, IPDPSW, IEEE, 2019, pp. 304–311.
[44] Andreas Zendler, Instructional methods in computing education judged by
computer science teachers and educational experts, IT, Inf. Technol. 60 (2)
(2018) 79–90.
[45] Jichen Zhu, Katelyn Alderfer, Anushay Furqan, Jessica Nebolsky, Bruce Char,
Brian Smith, Jennifer Villareale, Santiago Ontañón, Programming in game
space: how to represent parallel programming concepts in an educational
game, in: Proceedings of the 14th International Conference on the Foundations
of Digital Games, 2019, pp. 1–10.
[46] Nikola Zlatanov, CPU vs. SOC-the battle for the future of computing, in: Inter-
national System-on-Chip Conference at UCI, 2015.
Awad Younis is an Assistant Professor in the De-
partment of Computer Science at Northern Kentucky
University. He received his Ph.D. in Computer Science
from Colorado State University in 2016. His research
interests are in the areas of security vulnerabilities
(discovery process modeling; metrics), threats model-
ing (secure software engineering), human dimensions
of cybersecurity (end-users; defenders), cyber security
intelligence and operation, and computing and cyber-
security education (project based learning; AI based learning technology).
Rajshekhar Sunderraman is Professor and Asso-
ciate Chair of the Department of Computer Science
at Georgia State University. He received his Ph.D. in
Computer Science from Iowa State University in 1988.
His research expertise is in the areas of Databases,
Data Mining, Big Data, Knowledge Representation, Se-
mantic Web, and Reasoning with Incomplete and In-
consistent Information. He has published more than
150 research articles in these areas in leading com-
puter science journals and conference proceedings and is the author of a
popular textbook “Oracle 10g Programming: A Primer”, published by Pear-
son in 2007.
Mike Metzler is the Associate Director for Scholar-
ship of Teaching and Learning at Georgia State Univer-
sity (GSU). In 2015-2016 he was a GSU Faculty Teach-
ing Fellow in CETL, completing a research project on
instructional alignment with 20 faculty and gradu-
ate teaching assistants from various academic depart-
ments across the university. He is also currently a
professor in the Kinesiology and Health Department
at GSU.
Anu G. Bourgeois is an Associate Professor in the
Department of Computer Science at Georgia State
University. She received her Masters and Ph.D. in
Electrical and Computer Engineering from Louisiana
State University in 1997 and 2000, respectively. Her
research interests include parallel and distributed
computing, wireless networks, security and privacy,
and STEM education. She is a senior member of the
IEEE.