Why?

Data storage applications (caches, queues, databases, etc) implemented in Ruby suffer two major problems

You cannot use multiple CPU cores

Pausing GC causes serious delays for large data sets

What does Arabesque Promise?

Scalability to multiple CPU cores Constant performance profile regardless of queue size Can be used stand alone or embedded

Secret Sauce?

Berkeley DB

 

Single writer multiple reader concurrency in a multi-process or multi-threaded environment Data is stored in a shared memory area away from Ruby's GC Master-slave data replication Create data structure exclusively in memory

Benchmarks

Multi CPU Scaling
20000 18000 16000 14000

12000

Operations / Sec

10000

8000

6000

4000

2000

0 Single Process Dual Processes

Number of Processes

GC Time
12.00%

% of GC Time to application time

10.00%

8.00%

6.00%

4.00%

2.00%

0.00%

Zero

100K

1M

10M

Queue Size

Deployment Options

In-Process Library

Pros
  

Very fast Much easier to setup and use Less moving parts

Cons

Does not scale to multiple physical servers If you do that you lose ordering (might not be an issue)

Standalone Server

Pros

Allows multiple front end servers to use the same queue Allows the queue to scale up or scale out independently from the front end

Cons
 

More moving parts Added latencies of TCP/HTTP communication overhead Loses strict queue ordering in scale out scenarios

Thank You!