Talking About Twitter's

Architecture

Grupo 9:
54807 João Reis
55455 Jaime Bernardo
57743 António Sousa
Talking About Twitter's
Architecture

The importance of scalable

architectures!
What is twitter?
The Need
The Need
The Need
 The Need
How do we keep everyone
updated about our life?
How do we keep updated
about everyone's life?
The Solution
Microblogging
A Great Idea
Hammer View
What Happened?
Twitter got popular
As it often happens...
twitter's Bottlenecks
 twitter's Bottlenecks
• no monitoring, no graphs, no statistics
 twitter's Bottlenecks
• no monitoring, no graphs, no statistics

• memcached

• single mySQL database

 twitter's Bottlenecks
• no monitoring, no graphs, no statistics

• memcached

• single mySQL database

• Abuses
 twitter's Bottlenecks
• no monitoring, no graphs, no statistics

• memcached

• single mySQL database

• Abuses

• Plan to partition in the future

how can they
solve this?
 Solution

completely re-write the

application
how are they
doing it?
Databases

• more machines

• index everything

• denormalize
Starling

• Distributed Queuing

• Transactional Playback

• Fast

• Simple

• Speaks Memcache's Language

• 100% Pure Ruby

API

• Representational State Tranfer (REST)

• HTTP Requests

• XML

• RSS

• ATOM
Use The Community

Open Source

Starling
API

Feedback
Abuses

• Build tools to detect

• Be ruthless. Delete them as users.

Performance Monitoring
Has it worked?
 New services

Hundreds of updates per minute

Besides normal usage
Is it finished?
It's a work in progress
Still has some downtime, from time
to time.

Regular updates.
Lessons to learn
 Lessons to learn
• Scale early.
 Lessons to learn
• Scale early.
• Create performance monitors.
 Lessons to learn
• Scale early.
• Create performance monitors.
• Involve the community.
 Lessons to learn
• Scale early.
• Create performance monitors.
• Involve the community.
• Foresee abusive uses.
 Lessons to learn
• Scale early.
• Create performance monitors.
• Involve the community.
• Foresee abusive uses.
• Don't use inadequate
architectures.
QUESTIONS?