Getting Started with Data Visualization

Geoff McGhee
Tooling Up for Digital Humanities Seminar
May 6, 2011
Dealing with Data

Explosion of Electronic Information

2003 estimate: 5 exabytes/day* new info
Open government/transparency movements
E-commerce, electronic record-keeping
Digitization of media (photos, music, books...)
Remote sensors, RFID tags, POS systems
Plummeting Cost of Storage
Data formats (XML, JSON, RDF ), APIs
Social media

* Exabyte = 1 million terabytes

Dealing with Data
The Promise of Data Visualization

Using the Eye-Brain Connection

Bypass language centers, go direct to
the visual cortex
Leverage ability to recognize patterns,
visual sense-making
Powerful graphics chips enable
animation, live data processing possible

Map of New Brainland

by Unit Seven via Flickr
Roles of Data Visualization
Roles of Data Visualization

Making Sense of New Information

Data that reveals previously unknown

insights into patterns of life
Visualization as a way to throw things
on the wall and examine
Google N-Gram Viewer Things that used to be unknown,
unknowable, or impractical to know
Less about visualization than the data
Visualization
Visualizing as Mirror
New Information

Tourists vs. Locals, Eric Fischer, (2010)

http://www.flickr.com/photos/walkingsf/sets/72157624209158632/
Visualization as Mirror
Visualizing New Remix
Information

Flickr Flow, Fernanda Vigas and Martin Wattenberg (2009)

http://hint.fm/projects/flickr/
Visualization as Mirror
Visualizing New Information

GameDay, Major League Baseball (2011)

http://mlb.mlb.com
 Visualizing New Information

Good Morning, Jer Thorp (2009)

http://blog.blprnt.com/blog/blprnt/goodmorning
Roles of Data Visualization

Remix: The Familiar Through a New Lens

Innovations in graphic display can

change how we experience an idea
Less about data than the visualization
Now I see it
Visualization as Remix

Here and There, Berg Design (2009)

http://berglondon.com/projects/hat/
Visualization as Remix

River Maps, Daniel Huffman (2011)

http://somethingaboutmaps.wordpress.com/river-maps/
Visualization as Remix

River Maps, Daniel Huffman (2011)

http://somethingaboutmaps.wordpress.com/river-maps/
Visualization as Remix

The New York Times (2009)

http://www.nytimes.com/interactive/2009/11/06/business/economy/unemployment-lines.html
Roles of Data Visualization

Environment for Exploration

Tool for individual or collective exploration

Can show same data in multiple
dimensions, like time/space
Search, filter, drill down to details
Analyzing OCR Quality of Newspapers
Ideally, mark and share discoveries
within the tool
Visualization as
Environment for Mirror
Exploration

Mapping America: Every City, Every Block, The New York Times (2010)
http://projects.nytimes.com/census/2010/explorer
Visualization as
Environment for Mirror
Exploration

Assessing Digitization Quality, Bill Lane Center for the American West/
University of North Texas (2011)
http://mappingtexts.org
Life Cycle of Visualizations

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization
Life Cycle of Visualizations

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research
Spreadsheets
Databases
Digitized Media

Other Downloads
Public Data
Archives/Libraries
Academic Partners
Purchase

Scraping
Junar
Outwit Hub
ScraperWiki
Life Cycle of Visualizations

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research
Spreadsheets
Databases
Digitized Media

Other Downloads
Public Data
Archives/Libraries
Academic Partners
Purchase

Scraping
Junar
Outwit Hub
ScraperWiki
Life Cycle of Visualizations

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research
Spreadsheets
Databases
Digitized Media

Other Downloads
Public Data
Archives/Libraries
Academic Partners
Purchase

Scraping
Junar
Outwit Hub
ScraperWiki
Life Cycle of Visualizations

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization,

Spreadsheets Format
Databases Conversion
Digitized Media Google Refine
Data Wrangler
Other Downloads Mr. Data Converter
Public Data
Archives/Libraries
Academic Partners
Purchase

Scraping
Junar
Outwit Hub
ScraperWiki
Life Cycle of Visualizations

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization,

Spreadsheets Format
Databases Conversion
Digitized Media Google Refine
Data Wrangler
Other Downloads Mr. Data Converter
Public Data
Archives/Libraries
Academic Partners
Purchase

Scraping
Junar
Outwit Hub
ScraperWiki
Google Refine demo
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization,

Spreadsheets Format
Databases Conversion
Digitized Media Google Refine
Data Wrangler
Other Downloads Mr. Data Converter
Public Data
Archives/Libraries
Academic Partners
Purchase

Scraping
Junar
Outwit Hub
ScraperWiki
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization,

Spreadsheets Format
Databases Conversion
Digitized Media Google Refine
Data Wrangler
Other Downloads Mr. Data Converter
Public Data
Archives/Libraries
Academic Partners
Purchase

Scraping
Junar
Outwit Hub
ScraperWiki
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization,

Spreadsheets Format
Databases Conversion
Digitized Media Google Refine
Data Wrangler
Other Downloads Mr. Data Converter
Public Data
Archives/Libraries
Academic Partners
Purchase

Scraping
Junar
Outwit Hub
ScraperWiki
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization, Web Services

Spreadsheets Format Google Spreadsheets
Databases Conversion Google Fusion Tables
Digitized Media Google Refine IBM ManyEyes
Data Wrangler
Other Downloads Mr. Data Converter Applications
Public Data
Tableau/Tableau Public
Archives/Libraries
MS Office, OpenOffice
Academic Partners
Gephi
Purchase
Node XL (plug-in for Excel)
Spotfire
Scraping
R
Junar
Processing
Outwit Hub
ScraperWiki
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization, Web Services

Spreadsheets Format Google Spreadsheets
Databases Conversion Google Fusion Tables
Digitized Media Google Refine IBM ManyEyes
Data Wrangler
Other Downloads Mr. Data Converter Applications
Public Data
Tableau/Tableau Public
Archives/Libraries
MS Office, OpenOffice
Academic Partners
Gephi
Purchase
Node XL (plug-in for Excel)
Spotfire
Scraping
R
Junar
Processing
Outwit Hub
ScraperWiki
Getting Started with Visualization
Free and Web-Based Applications

IBM ManyEyes
http://manyeyes.alphaworks.ibm.com

Pros: Easy as pie

Many different chart forms
Interactivity
Bring your own data, or
use existing data set

Cons: Java applets are slow, clunky

Little design control
Easy but unpolished
No control over style
Little control over functionality
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization, Web Services

Spreadsheets Format Google Spreadsheets
Databases Conversion Google Fusion Tables
Digitized Media Google Refine IBM ManyEyes
Data Wrangler
Other Downloads Mr. Data Converter Applications
Public Data
Tableau/Tableau Public
Archives/Libraries
MS Office, OpenOffice
Academic Partners
Gephi
Purchase
Node XL (plug-in for Excel)
Spotfire
Scraping
R
Junar
Processing
Outwit Hub
ScraperWiki
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization, Web Services Static Visualizations

Spreadsheets Format Google Spreadsheets Previous tools +
Databases Conversion Google Fusion Tables Adobe Illustrator
Digitized Media IBM ManyEyes Adobe Photoshop
Google Refine
Data Wrangler
Other Downloads Animated Visualizations
Mr. Data Converter Applications
Public Data Processing
Tableau/Tableau Public
Archives/Libraries Adobe Flash
MS Office, OpenOffice
Academic Partners Adobe After Effects
Gephi
Purchase
Node XL (plug-in for Excel) Interactive/Web Visualizations
Spotfire
Scraping HTML5
R
Junar Protovis
Processing
Outwit Hub D3
ScraperWiki http://processingjs.org/
Adobe Flash or Flex
Processing
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization, Web Services Static Visualizations

Spreadsheets Format Google Spreadsheets Previous tools +
Databases Conversion Google Fusion Tables Adobe Illustrator
Digitized Media IBM ManyEyes Adobe Photoshop
}REFINEMENT
Google Refine
Data Wrangler
Other Downloads Animated Visualizations
Mr. Data Converter Applications
Public Data Processing
Tableau/Tableau Public
Archives/Libraries Adobe Flash
MS Office, OpenOffice
Academic Partners Adobe After Effects
Gephi
Purchase
Node XL (plug-in for Excel) Interactive/Web Visualizations
Spotfire
Scraping HTML5
R
Junar Protovis
Processing
Outwit Hub D3
ScraperWiki http://processingjs.org/
Adobe Flash or Flex
Processing
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization, Web Services Static Visualizations

Spreadsheets Format Google Spreadsheets Previous tools +
Conversion Google Fusion Tables Adobe Illustrator
Databases
Digitized Media IBM ManyEyes Adobe Photoshop
} refinement
Google Refine
Data Wrangler
Other Downloads Animated Visualizations
Mr. Data Converter Applications
Public Data Processing COMMONLY USED
Tableau/Tableau Public
Archives/Libraries Adobe Flash FOR NEWS
MS Office, OpenOffice
Academic Partners Adobe After Effects
Gephi
Purchase
Node XL (plug-in for Excel) Interactive/Web Visualizations
Spotfire
Scraping HTML5
R
Junar Protovis
Processing
Outwit Hub D3
ScraperWiki http://processingjs.org/
Adobe Flash or Flex
Processing
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization, Web Services Static Visualizations

Spreadsheets Format Google Spreadsheets Previous tools +
Conversion Google Fusion Tables Adobe Illustrator
Databases
Digitized Media IBM ManyEyes Adobe Photoshop
} refinement
Google Refine
Data Wrangler
Other Downloads Animated Visualizations
Mr. Data Converter Applications
Public Data Processing COMMON FOR
Tableau/Tableau Public
Archives/Libraries Adobe Flash ART
MS Office, OpenOffice
Academic Partners Adobe After Effects
Gephi
Purchase
Node XL (plug-in for Excel) Interactive/Web Visualizations
Spotfire
Scraping HTML5
R
Junar Protovis
Processing
Outwit Hub D3
ScraperWiki http://processingjs.org/
Adobe Flash or Flex
Processing
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization, Web Services Static Visualizations

Spreadsheets Format Google Spreadsheets Previous tools +
Conversion Google Fusion Tables Adobe Illustrator
Databases
Digitized Media IBM ManyEyes Adobe Photoshop
} refinement
Google Refine
Data Wrangler
Other Downloads Animated Visualizations
Mr. Data Converter Applications
Public Data Processing WONT WORK
Tableau/Tableau Public
Archives/Libraries Adobe Flash ON IPHONE/IPAD
MS Office, OpenOffice
Academic Partners Adobe After Effects
Gephi
Purchase
Node XL (plug-in for Excel) Interactive/Web Visualizations
Spotfire
Scraping HTML5
R
Junar Protovis
Processing
Outwit Hub D3
ScraperWiki http://processingjs.org/
Adobe Flash or Flex
Processing
Data Visualization Workflow

Gathering Data Analyzing It Sharing Findings

Discovery/ Cleaning/ Analysis/Exploratory Publication
Acquisition Munging Visualization

Original Research Normalization, Web Services Static Visualizations

Spreadsheets Format Google Spreadsheets Previous tools +
Conversion Google Fusion Tables Adobe Illustrator
Databases
Digitized Media IBM ManyEyes Adobe Photoshop
} refinement
Google Refine
Data Wrangler
Other Downloads Animated Visualizations
Mr. Data Converter Applications
Public Data Processing
Tableau/Tableau Public
Archives/Libraries Adobe Flash
MS Office, OpenOffice
Academic Partners Adobe After Effects
Gephi
Purchase
Node XL (plug-in for Excel) Interactive/Web Visualizations
Spotfire
Scraping HTML5
R
Junar Protovis
Processing
Outwit Hub D3
ScraperWiki http://processingjs.org/
Adobe Flash or Flex
Processing
Video Documentary
Video Documentary

datajournalism.stanford.edu
 Thanks!

gmcghee@stanford.edu

@mcgeoff