Proceedings of the Human Factors and Ergonomics Society 2019 Annual Meeting 437

OBSERVING CYBER SECURITY INCIDENT RESPONSE:

QUALITATIVE THEMES FROM FIELD RESEARCH
Megan Nyre-Yu, Purdue University
Robert S. Gutzwiller, PhD, Arizona State University
Barrett S. Caldwell, PhD, Purdue University

Cyber security increasingly focuses on the challenges faced by network defenders. Cultural and security-
driven sentiments about external observation, as well as publication concerns, limit the ability of
researchers to understand the context surrounding incident response. Context awareness is crucial to inform
design and engineering. Furthermore, these perspectives can be heavily influenced by the targeted sector or
industry of the research. Together, a lack of broad contextual understanding may be biasing approaches to
improving operations, and driving faulty assumptions in cyber teams. A qualitative field study was
conducted in three computer security incident response teams (CSIRTs) and included perspectives of
government, academia, and private sector teams. Themes emerged and provide insights across multiple
aspects of incident response, including information sharing, organization, learning, and automation. The
need to focus on vertical integration of issues at different levels of the incident response system is also
discussed. Future research will build upon these results, using them to inform technology advancement in
CSIR settings.

INTRODUCTION Professional analysts in cyber defense face a wide

Research in the intersection of cybersecurity and human
performance is a growing focus within the human factors
research community and at large (e.g., Aggarwal, Gonzalez, &
Dutt, 2018; Gutzwiller, Fugate, Sawyer, & Hancock, 2015;
Mancuso et al., 2014; Martin, Dubé, & Coovert, 2018;
Rajivan, Janssen, & Cooke, 2013; Vieane et al., 2016).
Considering the increasing importance of cybersecurity in
society, this timely development is critical to address projected
shortages in cybersecurity professionals and increasing fears
of cybersecurity threats (Bureau of Labor Statistics, 2016;
Coats, 2017).
As cyber is a dense socio-technical system, influences on
Copyright 2019 by Human Factors and Ergonomics Society. DOI 10.1177/1071181319631016
variety of human factors challenges (Gutzwiller et al., 2015).
Research addressing human performance of analysts,
however, is sparse though increasing (Gutzwiller, Hunt, &
Lange, 2016; Mancuso, Funke, Strang, & Eckold, 2015;
Vieane et al., 2016, 2017). Renewed emphasis on observation
and contextualization of analyst performance for cybersecurity
(Paul & Whitley, 2013) mostly focuses on intrusion detection,
but interest in studying analyst team activity is also emerging.
Computer Security Incident Response (CSIR) is based on
the need for a functioning team of professional cybersecurity
analysts to respond to cyber threats and handle their aftermath.
CSIR originates in the late 1980’s following the spread of a
computer worm which seriously disrupted connected systems

global cyber (and cyber-physical) system performance come
in many different forms and have many actors behind them.
From the intended users of commercialized software and
hardware, to the information technologists managing a
corporate environment, to a watch officer on board a ship,
each are part of a broader cyber ecosystem. Most users are on
the naïve end of a spectrum of control and responsibility for
cybersecurity; these users can attempt vigilance, carefully use
email, and protect their information and credentials through
various means such as encryption, complexity, and proper
handling. As both a major, continuing point of vulnerability
(Silberner, 2009) and as an easily-simulated and familiar task
which requires little expertise, research studies regarding
phishing exploits are numerous (Kumaraguru et al., 2010;
Martin et al., 2018; Sawyer & Hancock, 2018; Vishwanath,
Harrison, & Ng, 2016). Yet, elsewhere, in another part of the
system, cybersecurity professionals are dealing with cognitive
issues in performance as well (Bos et al., 2016; Conti,
Ahamad, & Stasko, 2005; Lemay & Leblanc, 2018; Libicki &
Pfleeger, 2004). Security is not a zero-sum game between
these groups, yet they each have different responsibilities; end
users are not directly responsible for a network, its defensive
posture, or responding to various cyber attacks.
(Schmidt & Darby, 2001) and imposed large costs. In
response, DARPA funded the establishment of the first CERT
- Computer Emergency Response Team (Goodwyn et al.,
1997). The team was tasked specifically with detecting and
responding to information security issues affecting the
networks of computers. In the last 30 years, the concept
caught on. CSIR teams (CSIRTs) are now present
internationally in large numbers, across governments, industry
and academic institutions. These teams can be defined, as in
Alberts et al. (2004), as “an organization or team that provides
services and support to a defined constituency for preventing,
handling, and responding to computer security incidents.”
(p.1).
Prior Research
A small body of research in CSIR was reviewed in
preparation for a series of interviews and on-site visits to
CSIRTs. The general roles of CSIRTs have been described in
detail elsewhere (e.g., Huis et al., 2017; Ruefle et al., 2014),
but generally include analysts performing proactive, reactive,
and maintenance tasks within an organization, and
coordinating and sharing information to balance the wide
array of skills and expertise in a given team. In general, the
Proceedings of the Human Factors and Ergonomics Society 2019 Annual Meeting
major CSIRT processes include preparation, detection,
analysis, containment, eradication, recovery, and post-incident
actions (Huis et al., 2017). The study of CSIRTs is different
than cognitive expertise studies of individual analysts because
CSIR is a distributed, team-based activity. A team-focused
perspective was pervasive in the literature, including the study
of situation awareness (e.g., Tyworth, Giacobe, & Mancuso,
2012). Work has examined CSIRTs in terms of large-scale
issues, such as workforce development, team effectiveness and
the social maturity of teams (Hoffman, Burley, & Toregas,
2012; Steinke et al., 2015; Tetrick et al., 2016).
Additionally, others have used ethnographic and
anthropological approaches for studying CSIRTs
(Sundaramurthy,et al., 2014), though they encountered access
problems in reaching their subjects. Others have applied more
traditional methods in examining knowledge structures in
cybersecurity, including cognitive task analysis for examining
team effectiveness and mental models (Chen et al., 2014), and
for understanding the diagnostic work process (Werlinger et
al., 2010).
CSIR capabilities are struggling in many areas, including
staffing of analysts and full-cycle response. Available
literature does not always provide context across CSIR
settings to focus design and improvement within these
organizations, or of the technology used in CSIR activities.
Where there was literature, it tended to focus on specific
aspects of CSIR operations, and did not adequately illustrate
vertical integration of issues affecting CSIR functions (such
as interpersonal, organizational and policy issues). Borrowed
from management literature, the term vertical integration in
this context is the understanding that, within the CSIRT, each
issue at each level of the organizational hierarchy must be
dealt with successfully, as it affects the others.
Mission
Policy
Organization
Process
Interpersonal
Technology
Figure 1. Vertical Integration of CSIR Issues
Human factors has long considered hierarchical and
systemic influences, particularly with respect to safety
(Shappell & Wiegmann, 2000). The vertical integration of
issues within CSIR (e.g., Figure 1) presents similar elements
and illustrates the complexity of this sociotechnical system as
more than just a set of usability issues in software, or of
technology development and deployment. Each of these
components represents perspectives and considerations for
human factors research.
Qualitative research can provide context and reveal new
aspects affecting security operations. Combined methods and
topics around ethnographic data collection can help create
richer contextual understanding (Schwartzman, 1993). This
438
study aimed to provide context-driven insights for analysts
and managers working in this area, and to allow researchers
better understanding of vertical issues in CSIR operations.
This article is the output of one analysis method which was
used in a multi-study dissertation by the first author.
Research Goals
The goal of this research was to gather context-informed
insights in CSIRTs to address both the contextual and vertical
assessment gaps. A long-term goal of the first author was to
determine technology design requirements to improve the
human-system integration in CSIRT environments. A variety
of CSIR settings were chosen to capture variation between
environments. Methods included observation and on-site
interviews designed to capture insights with high ecological
validity, and are described below.
METHOD
Ethnographic methods used included observations and
on-site interviews with three (3) active incident response
teams. Semi-structured observation (Gillham, 2008) and
interview techniques (Galletta, 2013) allowed the researcher to
gain insights, provided high ecological validity and a level of
consistency across participant teams.
CSIR teams. The study included three CSIRTs: one each
from a state government, a large company, and an academic
institution. Each team performed incident response functions,
and all teams had a tiered organization in which different
levels of response were broken into different tiers of
responders. In general, higher tiers correlated with higher
levels of operator expertise and specialization.
Data collection. The researcher spent 3-5 business days
with each team, observing daily operations, interviewing
analysts and managers, and walking through simulated
incidents with employees. Observations included sitting with
individual analysts during their daily activities and observing
shift handoff meetings. When direct observation was not
available, the researcher conducted simulated incident
response exercises with different tiers of analysts, using
manager-approved historical incident data from within the
respective organization. Simulations included Tier 1 (novices)
and Tier 2 (experienced generalist) analysts, as well as a lead
or manager if they were involved in the historical incident
being reconstructed.
Interviews with managers informed key attributes for
each team (Pelto, 2016), including: general work context,
work environment, mission-driven goals, operations,
organization, and general procedures during incident response.
Interviews with analysts further explored these topics, adding
information on specific practices regarding: information
sharing during incident response, automated tools, and the
roles and responsibilities of different tiers of analysts.
As many security operation centers (SOC) were secure
and sensitive to the potential of data breach, the researcher
was unable to use any recording devices. Data were generated
solely from observations, interviews, and interactions with
participants (recorded by the researcher’s handwritten notes).
Proceedings of the Human Factors and Ergonomics Society 2019 Annual Meeting
Analysis
Coding. Generated data from the field studies were
transcribed digitally for analysis. The researcher employed a
bottom-up qualitative coding procedure (Auerbach &
Silverstein, 2003), conducting three rounds of this
synthesizing technique. This process started with line-by-line
coding of all data generated from the ethnographic cases using
NVivo 11 software, resulting in 229 unique codes. The second
round used Microsoft Excel to group similar or related
statements, resulting in 99 second round codes.
Theme development. In the third round, second-round
codes were compared across all teams to understand higher-
level concepts. The researcher systematically and iteratively
grouped related or repeated codes into categories. This process
resulted in 11 themes. Developing themes using qualitative
methods (Merriam & Tisdell, 2016) is similar to developing
affinity diagrams (Holtzblatt, 2016) used in contextual inquiry.
An example of the theme development from two levels of
codes is depicted in Figure 2.
Automation is seen as a potential solution for low­level tasks
and coordination, but considered out of reach for teams who
don't have the support resources (20)
Automation as a Automation needs
Automation has
solution (10) additional support
disadvantages (4)
(6)
Automation of low Automation
level competes with considered more
training work
Automation not Automation can
prevalent or common create more noise
Figure 2. Example of Theme Development
RESULTS
The results of the qualitative data analysis included
eleven themes grouped around various aspects of incident
response settings (Table 1). Each of the thematic statements is
ranked by the volume of codes subsumed under them, with the
number of included codes indicated on the right side. Specific
thematic concepts addressed in the Discussion subsections are
shared awareness, organizational influences, and expertise.
DISCUSSION
The themes presented suggest a series of issues spanning
different levels of the security organizations, all of which
impact incident response operations. One challenge in
understanding CSIR operations is a lack of focus on system-
level analysis. Many other approaches consider improvements
in machine learning / artificial intelligence algorithms, or
enhancements to cybersecurity analyst education and skillsets
439
as solutions - yet rarely address the need for effective
integration and interaction between the software and human
components of CSIRTs. Again, these are not part of a zero-
sum game, and these gaps should be addressed; the findings in
this paper make strides toward a better, systems-level
understanding that includes context.
Table 1. Themes developed from interviews in three CSIRTs
Rank Theme Codes
Communication, feedback and accountability
are necessary for IR, awareness, and learning; If
1 38
lacking within or between levels of org, issues
arise
Organizational alignment on security priorities
2 and awareness of IR issues is important for 36
"full-cycle" IR process
3
Continuity of awareness and documentation
35
around incidents is important
IR requires a wide range of skills and
4 flexibility; Workforce may not be able to 23
maintain if not designed to do so
IR requires a wide range of activities, including
5 filtering and decision making; These can be 22
split based on expertise or authority of analysts
Automation is seen as a potential solution for
low-level tasks and coordination, but considered
6 20
out of reach for teams who don't have the
support resources
Knowledge sharing (in a repository, in person,
7 or through other channels) may be important for 14
learning and shared awareness
Formal and informal roles emerge to meet an
8 organizational need for management, 12
communication, and decision making
Identity and culture of the team affect
9 11
communications and responsibilities
Handoffs are varied in terms of procedure,
formality, and documentation; In whatever
10 10
form, they are important for continuity in
several contexts
Incident handling methods may be indicators of
11 organizational maturity; Maturity as a focus 8
may drive incident handling methods
Creating shared and consistent awareness
Shared awareness has often been identified as a key
component of effective cyber defense operations (Champion et
al., 2012; Gutzwiller et al., 2015; Mahoney et al., 2010;
Rajivan & Cooke, 2018). The data confirmed and identified
several factors that contribute to shared awareness, moving the
need in cybersecurity operations toward team-based,
distributed situation awareness (e.g., Salmon et al., 2009;
Stanton et al., 2006) when coupled with the geographic and
temporal distribution of analysts commonly seen in CSIR.
Facilitated by information sharing, the process of establishing
shared awareness is not a one-way information flow, but
rather a cycle that needs to include a feedback flow sometimes
lacking in organizations. Feedback ensures the information
Proceedings of the Human Factors and Ergonomics Society 2019 Annual Meeting
was received and understood; it can also be a channel for new
inputs from stakeholders in the communication loop. That is,
feedback can further establish an operational picture, guide
incident response decision making, and enable learning
between analysts. This finding also applies across the larger
organization. Points of contact in other parts of the
organization that receive ongoing incidents for further action
may not acknowledge receipt of the incident, which can create
uncertainty regarding action and resolution. Feedback is for
the analyst sending or escalating an incident to know that the
handoff was completed.
Mid- and senior-level managers, chief information
security officers (CISOs), and security personnel that liaise to
other divisions or groups are all included in the list of
individuals that may be notified during an incident to create
shared awareness among management. For the most part, their
role is to know (not necessarily acknowledge receipt of) the
information being shared, and use it for strategic decision
making or incident mitigation. Thus, much communication to
this class of people is one-way. Shared awareness may be
assumed at the risk of security.
One caveat is that shared awareness must remain
consistent throughout the incident response process, and is
typically maintained through documentation. Documentation,
as an additional and unique channel for information sharing,
plays a critical role in both real-time and historical analysis of
incident data, including threat data, key metrics of response,
who is or was involved, and so on. From this perspective,
documentation can ensure persistent awareness (Caldwell,
2011), especially if kept in a centrally accessible location, with
adequate mechanisms for real-time data input from multiple
parties. However, if not sufficiently maintained,
documentation (and access thereto) can inhibit accurate and
consistent awareness of a developing incident and hobble the
capability of a CSIRT.
Organizational influences and maturity
Consistent with systems approaches to human factors
(Proctor, 2008), recognizing the CSIRT as a part of a larger
system widens the scope of potential influences on
performance. The findings indicate that structure and policy
within an organization may greatly affect a group’s ability to
perform. In part, this is due to the constraints placed on the
agency and communication within and between subgroups.
The researcher noted that, in two of the three teams observed,
the authority of the security team to address incidents was
limited. Either the team was instructed to escalate the incident
to specialists (effectively making the security team a triage
step), or the team was instructed to escalate to the highest-
ranking security officer, who could manage the potential
political fallout of a security-driven decision. Either way,
actions driving decisions out of the hands of the CSIRT may
limit the team’s efficacy.
The organizational mission was an immediate and
significant driver in determining whether or not security was
an organizational priority. Prioritization at the organizational
level is critical, as security activities can directly compete with
operational uptime. In short, the goals of the organization
440
(such as security) can greatly affect the growth and maturity of
the security team in terms of human capital (how much the
team is allowed to expand in size and skill) and technology
(what investments can be made to advance the current state)
(and see Tetrick et al., 2016).
Expertise distribution and sharing
Many security operations centers are tiered organizations
across which expertise is distributed; capability is structured
into different levels of incident response. Lower (Tier 1)
analysts are typically responsible for filtering and triaging
incidents as they are detected, helping to determine whether or
not the alert is signal or noise, then directing the incident to
another analyst at a higher tier (Tier 2). Tier 2 analysts
typically handle the incident by conducting a short
investigation and taking actions to contain and even remediate
the threat. Should the incident require further investigation or
remediation actions, it is again escalated to a higher tier (Tier
3) of specialists who perform deeper analysis of the incident to
better understand origins and effects, then determine or
develop remediation actions and prevention activities.
The tiered nature of security operations creates
significant differences in the expertise available at each level,
which can diminish development and decision making at
lower levels of response. Exacerbating this effect was the
geographic, and sometimes temporal, separation of the tier
operators. Many teams had higher-tier analysts who worked at
a different site or worked from home. The separation of
expertise, coupled with different levels of authority and
clearance, were observed as hindrances to expertise sharing
between analysts. The different tiers sometimes maintain
separate knowledge sharing platforms that limit access for
team members of a different tier level. While there may be
mission-relevant reasons for these separations including
security itself, when CSIR is examined vertically, separations
reveal themselves as serious impediments to a team-based
response action. Essentially, these factors can disrupt
collaboration and knowledge sharing between tiers during
incident response, potentially degrading security.
CONCLUSIONS
Computer security incident response is an increasingly
important domain in providing defensive and offensive
support to firms around the globe. As the field advances
technologically, it is imperative that human factors efforts
keep pace in representing the human half of the human-
machine system. Research to strengthen contextual
understanding of this complex environment provides balance.
The work presented here reveals valuable insights grounded in
context from three different CSIRTs, and the findings indicate
that examining the vertical integration of issues uncovers
connections between factors across a variety of perspectives
and disciplines.
The findings in this study indicate there are a multitude
of areas in which research can expand to help build a more
complete understanding of CSIR environments. Further
investigation regarding distributed (but shared) situation
Proceedings of the Human Factors and Ergonomics Society 2019 Annual Meeting
awareness is warranted to better define information needs of
analysts, and development of strategies and interventions for
strengthening security operations may help alleviate issues
regarding staffing and process inefficiencies. Finally, this
paper encourages additional studies in defining distributed
expertise in CSIRTs, and how analyst teams gain and share
knowledge in secure, mission-critical task contexts.
References
Aggarwal, P., Gonzalez, C., & Dutt, V. (2018). HackIt: A real-time simulation
tool for studying real-world cyber-attacks in the laboratory. CNCS 2019,
(September).
Alberts, C., Dorofee, A., Killcrece, G., Ruefle, R., & Zajicek, M. (2004).
Management Processes for CSIRTs: A Work in Progress. Carnegie Mellon
Technical Report CMU/SEI-2004-TR-015.
Auerbach, C., & Silverstein, L. B. (2003). Qualitative data: An introduction to
coding and analysis. New York University Press.
Bos, N., Paul, C. L., Gersh, J. R., Greenberg, A., Piatko, C., Sperling, S., …
Burtner, R. (2016). Effects of gain/loss framing in Cyber defense decision-
making. Proceedings of the Human Factors and Ergonomics Society, 168–
172.
Bureau of Labor Statistics. (2016). Information Security Analysts.
Caldwell, B. S. (2011). Connection, Coupling, and Persistence in Online
Social Networks. In D. Haftor & A. Mirijamdotter (Eds.), Information and
Communication Technologies, Society and Human Beings: Theory and
Framework (Festschrift in honor of Gunilla Bradley) (pp. 346–354).
Hershey, PA: IGI Global.
Champion, M., Rajivan, P., Cooke, N. J., & Jariwala, S. (2012). Team-based
cyber defense analysis. Cognitive Methods in Situation Awareness and
Decision Support (CogSIMA), 218–221.
Chen, T. R., Shore, D. B., Zaccaro, S. J., Dalal, R. S., Tetrick, L. E., & Gorab,
A. K. (2014). An organizational psychology perspective to examining
computer security incident response teams. IEEE Security & Privacy, 12(5),
61–67.
Coats, D. R. (2017). Worldwide Threat Assessment of the US Intelligence
Community. Washington, DC, USA.
Conti, G., Ahamad, M., & Stasko, J. (2005). Attacking information
visualization system usability overloading and deceiving the human.
Proceedings of the 2005 Symposium on Usable Privacy and Security -
SOUPS ’05, 89–100.
Galletta, A. (2013). Mastering the Semi-Structured Interview and Beyond
From Research Design to Analysis and Publication. New York: NYU Press.
Gillham, B. (2008). Observation techniques : structured to unstructured.
London ; New York: Continuum International Pub.
Goodwyn, J. C., Moore, R. A., Jordan, W., Richardson, J., Roosild, S., Worch,
P., … Adams, D. (1997). Defense Advanced Research Projects Agency
Technology Transition. DARPA, 1–185.
Gutzwiller, R.S., Hunt, S. M., & Lange, D. S. (2016). A task analysis toward
characterizing cyber-cognitive situation awareness (CCSA) in cyber defense
analysts. In 2016 IEEE International Multi-Disciplinary Conference on
Cognitive Methods in Situation Awareness and Decision Support, CogSIMA
2016.
Gutzwiller, R. S, Fugate, S., Sawyer, B. D., & Hancock, P. A. (2015). The
human factors of cyber network defense. Proceedings of the Human Factors
and Ergonomics Society Annual Meeting, 59(1), 322–326.
Hoffman, L., Burley, D., & Toregas, C. (2012). Holistically building the
cybersecurity workforce. IEEE Security & Privacy, 10(2), 33-39.
Holtzblatt, K. (2016). Contextual design: design for life (2nd ed.). Elsevier.
Huis, M., van der Kleij, R., Kleinhaus, G., de Koning, L., Kort, J., Meiler, P.,
… Young, H. (2017). Human factors in cyber incident response: Needs,
collaboration and the Reporter. TNO Report 2017-R11575, 1–47.
Kumaraguru, P., Sheng, S., Acquisti, A., Cranor, L. F., & Hong, J. (2010).
Teaching Johnny not to fall for phish. ACM Transactions on Internet
Technology, 10(2), 1–31.
Lemay, A., & Leblanc, S. (2018). Cognitive biases in cyber decision-making.
Proceedings of the 13th International Conference on Cyber Warfare and
Security, 395.
Libicki, M., & Pfleeger, S. (2004). Collecting the dots: Problem formulation
and solution elements. Santa Monica, CA: RAND Corporation.
Mahoney, S., Roth, E., Steinke, K., Pfautz, J., Wu, C., & Farry, M. (2010). A
cognitive task analysis for cyber situational awareness. Proceedings of the
Human Factors and Ergonomics Society Annual Meeting, 54, 279–283.
441
Mancuso, V. F., Christensen, J. C., Cowley, J., Finomore, V., Gonzalez, C., &
Knott, B. (2014). Human factors in cyber warfare II: Emerging perspectives.
Proceedings of the Human Factors and Ergonomics Society Annual
Meeting, 58(1), 415–418.
Mancuso, V. F., Funke, G. J., Strang, A. J., & Eckold, M. B. (2015).
Capturing performance in cyber human supervisory control. Proceedings of
the Human Factors and Ergonomics Society Annual Meeting, 59(1), 317–
321.
Martin, J., Dubé, C., & Coovert, M. D. (2018). Signal detection theory (SDT)
is effective for modeling user behavior toward phishing and spear-phishing
attacks. Human Factors, 60(8), 1179–1191.
Merriam, S. B., & Tisdell, E. (2016). Qualitative research : a guide to design
and implementation (Fourth edi). Jossey-Bass.
Paul, C. L., & Whitley, K. (2013). A taxonomy of cyber awareness questions
for the user-centered design of cyber situation awareness. HCII 2013, 436–
448.
Pelto, P. J. (2016). Applied ethnography: Guidelines for field research. In
Applied ethnography: Guidelines for field research. Routledge.
Proctor, R. W. (2008). Human factors in simple and complex systems (2nd
ed..). Boca Raton, Fla.: CRC Press.
Rajivan, P., & Cooke, N. J. (2018). Information-pooling bias in collaborative
security incident correlation analysis. Human Factors, 60(5), 626–639.
Rajivan, P., Janssen, M. A., & Cooke, N. J. (2013). Agent-based model of a
cyber security defense analyst team. Proceedings of the Human Factors and
Ergonomics Society Annual Meeting, 57(1), 314–318.
Ruefle, R., Dorofee, A., Mundie, D., Householder, A. D., Murray, M., & Perl,
S. J. (2014). Computer security incident response team development and
evolution. IEEE Security & Privacy, 12(5), 16–26.
Salmon, P. M., Stanton, N. A., Walker, G. H., & Jenkins, D. P. (2009).
Distributed situation awareness: Theory, measurement and application to
teamwork. Boca Raton, FL: CRC Press.
Sawyer, B. D., & Hancock, P. A. (2018). Hacking the human: the prevalence
paradox in cybersecurity. Human Factors, 60(5), 597–609.
Schmidt, C., & Darby, T. (2001). The What, Why, and How of the 1988
Internet Worm.
Schwartzman, H. B. (1993). Ethnography in organizations (Vol. 27). Sage.
Shappell, S., & Wiegmann, D. A. (2000). The Human Factors Analysis
Classification System (HFACS). Federal Aviation Administration, 1–19.
Silberner, J. (2009, December). Don’t fall for CDC swine flu phishing scam.
NPR.
Stanton, N. A., Stewart, R., Harris, D., Houghton, R. J., Baber, C., McMaster,
R., … Green, D. (2006). Distributed situation awareness in dynamic
systems: theoretical development and application of an ergonomics
methodology. Ergonomics, 49(12–13), 1288–1311.
Steinke, J., Bolunmez, B., Fletcher, L., Wang, V., Tomassetti, A. J., Repchick,
K. M., … Tetrick, L. E. (2015). Improving cybersecurity incident response
team effectiveness using teams-based research. IEEE Security and Privacy,
13(4), 20–29.
Sundaramurthy, S. C., McHugh, J., Ou, X., Rajagopalan, S. R., & Wesch, M.
(2014). An anthropological approach to studying CSIRTs. IEEE Security &
Privacy, 12(5), 52-60.
Tetrick, L. E., Zaccaro, S. J., Dalal, R. S., Steinke, J. A., Repchick, K. M.,
Hargrove, A. K., … Wang, V. (2016). Improving Social Maturity of
Cybersecurity Incident Response Teams. Fairfax, VA: George Mason
University.
Tyworth, M., Giacobe, N. A., & Mancuso, V. (2012). Cyber situation
awareness as distributed socio-cognitive work. In Cyber Sensing 2012 (Vol.
8408, p. 84080F). International Society for Optics and Photonics.
Vieane, A. Z., Funke, G., Greenlee, E., Mancuso, V., Borghetti, B., Miller, B.,
… Boehm-Davis, D. (2017). Task interruptions undermine cyber defense.
Proceedings of the Human Factors and Ergonomics Society, 375–379.
Vieane, A.,Z. Funke, G., Mancuso, V., Greenlee, E., Dye, G., Borghetti, B.,
… Brown, R. (2016). Coordinated displays to assist cyber defenders.
Proceedings of the Human Factors and Ergonomics Society, 344–348.
Vieane, A. Z., Funke, G. J., Gutzwiller, R. S., Mancuso, V. F., Sawyer, B. D.,
& Wickens, C. D. (2016). Addressing human factors gaps in cyber defense.
Proceedings of the Human Factors and Ergonomics Society, 60, 770–773.
Vishwanath, A., Harrison, B., & Ng, Y. J. (2016). Suspicion, cognition, and
automaticity model of phishing susceptibility. Communication Research,
45(8), 1146-1166.
Werlinger, R., Muldner, K., Hawkey, K., & Beznosov, K. (2010). Preparation,
detection, and analysis: the diagnostic work of IT security incident response.
Information Management & Computer Security, 18(1), 26–42.