The Tornado IOLoop

More than just a pretty web framework:
the Tornado IOLoop
Gavin M. Roy
About myYearbook
According to comScore, myYearbook is one of the 25 mosttrafficked sites in the United States as measured by page views, by minutes, and by minutes per visitor per month.
http://www.myyearbook.com/careers
PyCon 2012 Follow me on Twitter: @Crad
Tornado @ myYearbook
At myYearbook, we used Tornado in highrequest-velocity applications.
PyCon 2012
Web apps Finite State Machine Game Server

Take what you need? We do.
Follow me on Twitter: @Crad
Async Network Programming

Core revolves around an I/O loop Let the Operating System indicate when
to read, when to write and when there are errors on sockets.
Callback-Passing Style
PyCon 2012
select, poll, epoll (Linux), kqueue (BSD)

Async Network Programming
PyCon 2012
Callback Passing Style

def start(foo): # Do stuff with foo and when done call the next function stuff(callback=next_step, data=foo); def next_step(bar): # Call more_stuff to parse bar more_stuff(callback=last_step, data=bar); def last_step(baz): # Send the response send_response(baz) # Let our handler know we are done finish()
PyCon 2012
tornado.IOLoop
Core of Tornados network stack Fast and easy to use Single instance per process Cross-platform Easy to read source: http://goo.gl/VFeAF Used for client libraries & server
applications
PyCon 2012
Works on windows but not supported.
C-Based epoll, not required
tornado.IOStream
Convenient utility class for dealing with the
IOLoop
Does most of the work for you This too is for use by clients and server
applications
IOStream: How to read

read_until_regex read_until read_bytes
(regex, callback) (delimiter, callback)
(num_bytes, callback, streaming_callback=None)
read_until_close
(callback, streaming_callback=None)
PyCon 2012
IOStream: Whats Going On?
reading(), writing(), closed()

Serious Business Uses SSL

And thats why SSLIOStream Exists.
tornado.netutil.TCPServer
PyCon 2012
Hello Async World

from tornado import ioloop from tornado import netutil class EchoServer(netutil.TCPServer): def handle_stream(self, stream, address): self._stream = stream self._read_line() def _read_line(self): self._stream.read_until('\n', self._handle_read) def _handle_read(self, data_in): self._stream.write('You sent: %s' % data_in) self._read_line() if __name__ == '__main__': server = EchoServer() server.listen(2007) ioloop.IOLoop.instance().start()
PyCon 2012
Source: http://goo.gl/eB7z4
What just happened?

tornado.stack_context.StackContext
Slight Magic
PyCon 2012
Diving Deeper
PyCon 2012
tornado.IOLoop
IOLoop.instance() ioloop.add_handler
(fd, handler, events) (fd, handler, events)
ioloop.update_handler
ioloop.remove_handler(fd)
Events?
READ, WRITE, ERROR

Example
import socket from tornado import ioloop def on_events(fd, events, error=None): if events & ioloop.IOLoop.READ: print 'Socket read: %r' % fd.recv(1024) if events & ioloop.IOLoop.ERROR: print 'Error received: %r' % error if events & ioloop.IOLoop.WRITE: pass _ioloop = ioloop.IOLoop.instance() fd = socket.socket() # Other stuff needs to be done here events_desired = ioloop.IOLoop.READ | ioloop.IOLoop.ERROR _ioloop.add_handler(fd, on_events, events_desired) _ioloop.start()
Timers
add_timeout
(timeout)
add_timeout returns a reference deadline can be unix timestamp or a datetime.timedelta
(deadline, callback)
remove_timeout
PyCon 2012
add_callback(callback)
Pythonic Simplication
PyCon 2012
tornado.gen
for coding not in Callback Passing Style:
@gen.engine def foo(): http_client = AsyncHTTPClient() response1, response2 = yield [gen.Task(http_client.fetch, url1), gen.Task(http_client.fetch, url2)]
PyCon 2012
tornado.platform.twisted
Last Thoughts
Single threaded, use processes Watch out for resource contention Benchmark your application Fork work to remote workers multiprocessing.reduction
multiprocessing.reduction:
# Main process from multiprocessing.reduction import reduce_handle h = reduce_handle(client_socket.fileno()) pipe_to_worker.send(h) #instance of multiprocessing.Pipe # Worker process from multiprocessing.reduction import rebuild_handle h = pipe.recv() fd = rebuild_handle(h) client_socket = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM) client_socket.send("hello from the worker process\r\n")
Questions?

The Tornado IOLoop

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The Tornado IOLoop

Uploaded by

Copyright:

Available Formats

More than just a pretty web framework:

the Tornado IOLoop

Web apps Finite State Machine Game Server

Async Network Programming

select, poll, epoll (Linux), kqueue (BSD)

Async Network Programming

Follow me on Twitter: @Crad

Callback Passing Style

Follow me on Twitter: @Crad

C-Based epoll, not required

Follow me on Twitter: @Crad

IOStream: How to read

(num_bytes, callback, streaming_callback=None)

Follow me on Twitter: @Crad

IOStream: Whats Going On?

reading(), writing(), closed()

Serious Business Uses SSL

Follow me on Twitter: @Crad

Hello Async World

What just happened?

Follow me on Twitter: @Crad

Follow me on Twitter: @Crad

(fd, handler, events) (fd, handler, events)

READ, WRITE, ERROR

add_timeout returns a reference deadline can be unix timestamp or a datetime.timedelta

Follow me on Twitter: @Crad

Follow me on Twitter: @Crad

Follow me on Twitter: @Crad

You might also like