Boxing  Outside  the  Think:     Conduc8ng  Crea8ve  Vulnerability  Assessments
Roger G. Johnston, Ph.D., CPP Jon S. Warner, Ph.D. Vulnerability Assessment Team Argonne National Laboratory




Argonne National Laboratory

~$785 million annual budget 1500 acres, 3400 employees, 4400 facility users, 1100 students R&D and technical assistance for government & industry

Vulnerability Assessment Team (VAT)!



•     DoD   •     DOS   •     IAEA   •     Euratom  

The  VAT  has  done  detailed           vulnerability  assessments  on   hundreds  of  different  security      devices,  systems,  &  programs.  

•     DOE/NNSA   •     private  companies   •     intelligence  agencies   •     public  interest  organiza8ons  



The Top 5 Impediments to Good Security!
1.    Lack  of  Imagina/on     2.    Cogni/ve  Dissonance     3.    Security  Theater  &  Compliance-­‐Based  Security     4.    Poor  Insider  Threat  Mi/ga/on     5.    Weak  Security  Culture  

Problem: Lack of Research-Based Security Practice! A free, online,
peer-reviewed R&D journal


The Journal of Physical Security



Lack of Imagination!
I  don’t  think  that  anybody  could  have  predicted  that  these  people   would  take  an  airplane  and  slam  it  into  the  World  Trade  Center,   take  another  one  and  slam  it  into  the  Pentagon,  that  they  would   try  to  use  an  airplane  as  a  missile  ...  even  in  retrospect  there  was   nothing  to  suggest  that.                        -­‐-­‐  Tes8mony  of  Secretary  of  State  Condoleezza  Rice      to  the  9/11  Commission      

  The  purpose  is  to  find  exploitable  security  weaknesses   to  improve  security.  

Vulnerability Assessments!

Confused  a  lot  with  Threat  Assessments  (or  other   aspects  of  overall  Risk  Management).  

Should  include  sugges/ons  for  countermeasures.  



Adversarial Vulnerability Assessments!
•  Perform a mental coordinate transformation and pretend to be the bad guys (or VAers). (This is much harder than you might think.)

•  Be much more creative than the adversaries. They need only stumble upon 1 vulnerability, the good guys have to worry about all of them.

Adversarial Vulnerability Assessments!
•  Don’t let the good guys & the existing security infrastructure and tactics define the problem.

•  Gleefully look for trouble, rather than seeking to reassure yourself that everything is fine.



We need to be more like fault finders. They find problems because they want to find problems, and because they are skeptical:

  •     bad  guys   •     therapists   •     movie  cri8cs   •     computer  hackers   •     scien8fic  peer  reviewers   •     mothers-­‐in-­‐law  


Assembling Your Own VA Team: Seek…!
hackers q  narcissists q  trouble makers q  hands-on types q  creative people q  loop-hole finders q  independent thinkers q  questioners of authority q  people curious about how things work




Blunder: Thinking Engineers Understand Security"
  • ...work  in  solu8on  space,  not  problem  space   • …don’t  realize  that  mee8ng  standards  does  not  solve  the  problem     • …know  how  to  make  things  work,  but  not  how  to  make  them  break     • ...view  Nature  or  economics  as  the  adversary,  not  the  bad  guys       • …tend  to  think  technologies  fail  randomly,  not  by  deliberate,  intelligent,  malicious  intent       t, en • …are  not  typically  predisposed  to  thinking  like  bad  guys   gm
d ju e!   lue ibut a v ttr • …focus  on  user  friendliness—not  making  things  difficult  for  the  bad  guys   sa a ” i ct ity rodu r cu p • ...like  to  add  lots  of  extra  features  that  open  up  new  a_ack  vectors   igh Senot a H

• …make  products  simple  to  maintain/repair/diagnose—which  also  makes  them  easy  to  a_ack

The Creative VA Process!
•  Allow lots of time for individual analysis.

•  Individuals need to be given ownership of their ideas & should be personally recognized for their creativity.



The Creative VA Process!
•  The ideal group environment:
+  diverse & high energy +  people are a little tired +  urgent but not stressful +  free of authority figures +  humorous, joyful, & fun +  use the activation effect +  cohesive but not too cohesive +  competitive in a friendly & respectful way +  enthusiastic about individual differences & eccentricities

•  Every idea, no matter how wacky

or seemingly stupid, gets written down & treated as a gem, at least initially.

Delaying Judgment!
Nothing can inhibit and stifle the creative process more— and on this there is unanimous agreement among all creative individuals and investigators of creativity—than critical judgment applied to the emerging idea at the beginning stages of the creative process. ... More ideas have been prematurely rejected by a stringent evaluative attitude than would be warranted by any inherent weakness or absurdity in them. The longer one can linger with the idea with judgment held in abeyance, the better the chances all its details and ramifications [can emerge].
-- Eugene Raudsepp, Managing Creative Scientists and Engineers (1963). Keep the possibility phase completely separate from the practicality phase!



Where Vulnerability! Ideas Come From!

The Vulnerability Pyramid

Safety & Security are 2 Relatively Unrelated Problems!

Example: March 2012 Recall of 900,000 Safety 1st Push N’ Snap Cabinet Locks 140 reports of babies/toddlers defeating the locks, resulting in 3 poisonings

Security: All about intentional nefarious adversaries. Safety: No adversaries.  



Vulnerability Assessment Myths!
•  VA = problem solving. •  A vulnerability assessment should be done at the end. •  There are a small number of vulnerabilities. •  Most or all can be found & eliminated. •  A VA should ideally find zero vulnerabilities. •  Vulnerabilities are bad news.

Vulnerability Assessment (VA) Blunders!
•  Not using creative people with a hacker mentality who want to find problems and suggest solutions •  Conflicts of interest (economic & psychological) •  Shooting the messenger •  Sham rigor & the fallacy of precision •  Lack of skepticism



Vulnerability Assessment (VA) Blunders!
•  Focusing on high-tech attacks •  Letting attack methods define the vulnerabilities, not the other way around •  Arbitrarily constrained VAs (scope, time, effort, by modules or components) •  Fear of NORQ The…   Non-­‐Objec8ve   Non-­‐Reproducible   Non-­‐Quan8fiable  

For More Information...!



Sign up to vote on this title
UsefulNot useful