Web Scalability &

Performance
TechWeekend #3
5th September 2009

Mukul Kumar
Co-Founder & VP Engineering at
PubMatic
mukul@pubmatic.com

http://twitter.com/mukulneetika
 Agenda
• Application Scalability
• System Scalability
• Application Performance
• System Performance
• Hard Learnt Lessons
 I will not talk about 
• Cloud Computing
• MySQL tuning
 Ideas for Scalability
• Keep it simple
• Make it light
• Cache Aggressively
• Don’t invalidate cache too often
• Should you keep cache centralized?
– Single point of failure
– Bottleneck
 Loosely Coupled
Distributed Architecture
• Shared nothing
• Stateless apps
• MySQL
– Use replication extensively
– Master Slave
– Master Slave Slave
– Don’t use your master DB for running all
queries
– Use new features and versions with care
– or don’t use them
 Tiered architecture
• Memcached
• Read Only DB
• Master-DB

• Think global
– X servers in San Jose, Y in Texas, Z in
London
• Think – Horizontal scaling vs. vertical
scaling
 Hardware load balancer
• Get a hardware load balancer
• Algorithms
• Session persistence
• Header passing
• Timeout values
Recipe for handling rapid
growth
Source – http://highscalability.com/youtube-architecture
 
while (true) {
identify_and_fix_bottlenecks();
eat_drink();
sleep();
notice_new_bottleneck();
}
 
This loop runs many times.

Getting it, 60%-70%-80% perfect

Testing
• Real load
• Real environment
• Real hardware
• Real software

Test with real load, in a real environment, with real

hardware and software. Lab tests are only indicative.
You will know about the real problems only when you
run the real load.
 HLL – Hard Learnt Lessons
• Independent servers
• Keep spare servers all the time
• Keep all commands in a script
– you may have to run them at 2:12AM
– then at 3:25AM
– then at 7:42AM
• Keep a fault identification, communication and fault
resolution “protocol”
• Monitor your web services from 12 cities around the
world
• Be paranoid
• Servers can go down anytime
• DDOS attacks can happen anytime
• NIC cards can become slow or go down anytime
Image Source: http://bose.infopop.cc/eve/forums/a/tpc/f/6806048934/m/1171031006
 Thanks!

Questions & Comments

mukul@pubmatic.com
twitter.com/mukulneetika