Measuring Performance

,
JavaScript Games,
and Distributed Testing
John Resig
http://ejohn.org - http://twitter.com/jeresig 
Measuring
Performance 
Analyzing Performance
✦ Optimizing performance is a huge
concern: Faster code = happy users!
✦ Measure execution time
✦ Loop the code a few times
✦ Measure the difference:
✦ (new Date).getTime(); 
Stack Profiling
✦ jQuery Stack Profiler
✦ Look for problematic methods and plugins
✦ http://ejohn.org/blog/deep-profiling-
jquery-apps/ 
Accuracy of
JavaScript Time
We’re measuring the performance of
JavaScript from within JavaScript!

http://ejohn.org/blog/accuracy-of-javascript-time/ 
15ms intervals ONLY!

Error Rate of 50-750%! 
Performance Tools
✦ How can we get good numbers?
✦ We have to go straight to the source: Use
the tools the browsers provide.
✦ Tools:
✦ Firebug Profiler
✦ Safari Profiler
✦ (Part of Safari 4)
✦ IE 8 Profiler 
Firebug Profiler 
Safari 4 Profiler 
IE 8 Profiler 
FireUnit
✦ A simple JavaScript test suite embedded in
Firebug.
✦ http://fireunit.org/ 
FireUnit Profile Data

{
fireunit.getProfile(); "time": 8.443,
"calls": 611,
"data":[
{
"name":"makeArray()",
"calls":1,
"percent":23.58,
"ownTime":1.991,
"time":1.991,
"avgTime":1.991,
"minTime":1.991,
"maxTime":1.991,
"fileName":"jquery.js (line 2059)"
},
// etc.
http://ejohn.org/blog/function-call-profiling/ ]} 
Complexity Analysis
✦ Analyze complexity rather than raw time
✦ jQuery Call Count Profiler (uses FireUnit)
Method Calls Big-O
.addClass("test"); 542 6n
.addClass("test"); 592 6n
.removeClass("test"); 754 8n
.removeClass("test"); 610 6n
.css("color", "red"); 495 5n
.css({color: "red", border: "1px 887 9n
solid red"});
.remove(); 23772 2n+n2
.append("<p>test</p>"); 307 3n 
Complexity Analysis
✦ Reducing call count helps to reduce
complexity
✦ Results for 1.3.3:
Method Calls Big-O

.remove(); 298 3n

.html("<p>test</p>"); 507 5n

.empty(); 200 2n

http://ejohn.org/blog/function-call-profiling/ 
Why JavaScript
Games Are HARD
to Build 
Browser-Based Games
✦ No installation required
✦ (Will be able to play at work!) 
Why not Flash?
✦ JavaScript works everywhere
✦ Desktop
✦ iPhone / Mobile Device
✦ Wii
✦ OLPC

✦ JavaScript is an open technology 
Goals of a game
✦ Should be Multiplayer
✦ Can’t be casually cheated
✦ Work well everywhere
✦ Addictive 
Why Multiplayer?
✦ Incredibly addictive
✦ No longer “alone”
✦ Enticed to continue playing 
3 Styles of Games
✦ Strategy
✦ Intelligence
✦ Accuracy 
Strategy Games
✦ Require a lot of time to think
✦ Generally last over a couple hours or days
✦ Replayability
✦ Hard to cheat 
WarFish

✦ http://warfish.net/ 
Nile Online

✦ http://www.playnileonline.com/ 
Strategy Games
✦ Very server-side heavy
✦ Most logic hidden from the user

✦ Hard to cheat
✦ Casual cheaters can’t change values
✦ Dedicated cheaters have to write full AI 
Intelligence Games
✦ Player’s intelligence/knowledge challenged
✦ Games could be quick or slow-paced
✦ Easy to cheat
✦ Casual cheaters can open dictionary /
encyclopedia for answers 
Word Twist

✦ http://wordtwist.org/ 
Babble

✦ http://playbabble.com/ 
Iron Sudoku

✦ http://ironsudoku.com/ 
Speed/Accuracy Games
✦ Require low latency
✦ Fast-paced and addictive
✦ JavaScript completely fails
✦ Garbage Collection cycles freeze the
browser
✦ None, or few, Accuracy-centric JavaScript
games
✦ Experienced coders can easily cheat
✦ (A bot to hit the keys at the right time) 
Guitar Hero

http://ejohn.org/apps/hero/ 
Guitar Hero
✦ Heavily dependent upon accuracy
✦ (Hit the right notes at the right time)

✦ Garbage collection cycles absolutely kill
the game
✦ Rendering the play area is also difficult
✦ And impossible in all browsers.
✦ (Use HTML 5 Canvas Element) 
Failures on All Ends
✦ Strategy: Slow, secret server-side code
✦ Intelligence: Easily cheatable
✦ Accuracy: Too hard to implement
✦ Optimal solution would be a combination
of the above.
✦ JavaScript games can’t be like other games,
have to be unique. 
What can make a fun game?
✦ Quick play
✦ Points
✦ High Scores
✦ Head-to-head competition 
Wordsplay

✦ Real-time Boggle
✦ Head-to-head with other players
http://www.wordsplay.net/ 
Tringo

✦ Tetris + Bingo (based on a Second Life
game)
http://ejohn.org/tringo/ 
DeepLeap

✦ Fast-paced Scrabble-like game
✦ Speed + Intelligence + Strategy

http://deepleap.org/ 
vs. Cheating
✦ All words are recorded with exact time
information
✦ Game is “played back” on the server to
verify score integrity using Server-Side JS

✦ This data can be used to simulate a multi-
player experience! 
DeepLeap
✦ Works in multiple languages
✦ Dictionaries pulled from OpenOffice,
can build a Spanish version in < 5sec
✦ Players can challenge each other head-to-
head
✦ Score multiplier (carry over from Guitar
Hero) 
Distributed Testing 
Choose Your Browsers 
Cost / Benefit

IE 7 IE 6 FF 3 Safari 3 Opera 9.5
Cost Benefit

Draw a line in the sand. 
Graded Support

Yahoo Browser Compatibility 
Browser Support Grid
IE Firefox Safari Opera Chrome

Previous 6.0 2.0 3.0 9.5

Current 7.0 3.0 3.2 9.6 1.0

Next 8.0 3.1 4.0 10.0 2.0

jQuery Browser Support 
Browser Support Grid
IE Firefox Safari Opera Chrome

Previous 6.0 2.0 3.0 9.5

Current 7.0 3.0 3.2 9.6 1.0

Next 8.0 3.1 4.0 10.0 2.0

jQuery 1.3 Browser Support 
The Scaling Problem
✦ The Problem:
✦ jQuery has 6 test suites
✦ Run in 11 browsers
✦ (Not even including multiple platforms!)

✦ All need to be run for every commit,
patch, and plugin.
✦ JavaScript testing doesn’t scale well. 
Distributed Testing
✦ Hub server
✦ Clients connect and help run tests
✦ A simple JavaScript client that can be run
in all browsers
✦ Including mobile browsers!

✦ TestSwarm 
FF 3.5 FF 3.5 FF 3.5
IE 6
IE 6
FF 3 IE 6
Op 9
FF 3

IE 7

TestSwarm
IE 7

Test Suite Test Suite Test Suite 
Manual Testing
✦ Push tests to users who follow pre-defined
steps
✦ Answer ‘Yes’/’No’ questions which are
pushed back to the server.
✦ An effective way to distribute manual test
load to dozens of clients. 
TestSwarm.com
✦ Incentives for top testers (t-shirts, books)
✦ Will be opening for alpha testing very soon
✦ Help your favorite JavaScript library
become better tested!
✦ http://testswarm.com 
Questions
✦ Contact:
✦ John Resig
✦ http://ejohn.org/
✦ http://twitter.com/jeresig