Asynchronous programming with Tornado

Asynchronous programming can be tricky for beginners, therefore I think it’s useful to iron some basic concepts to avoid common pitfalls. For an explanation about generic asynchronous programming, I recommend you one of the many resourcesonline. I will focus solely on asynchronous programming in Tornado.

From Tornado’s homepage:

FriendFeed’s web server is a relatively simple, non-blocking web server written in Python. The FriendFeed application is written using a web framework that looks a bit like web.py or Google’s webapp, but with additional tools and optimizations to take advantage of the non-blocking web server and tools. Tornado is an open source version of this web server and some of the tools we use most often at FriendFeed. The framework is distinct from most mainstream web server frameworks (and certainly most Python frameworks) because it is non-blocking and reasonably fast. Because it is non-blocking and uses epoll or kqueue, it can handle thousands of simultaneous standing connections, which means the framework is ideal for real-time web services. We built the web server specifically to handle FriendFeed’s real-time features every active user of FriendFeed maintains an open connection to the FriendFeed servers. (For more information on scaling servers to support thousands of clients, see The C10K problem.)

The first step as a beginner is to figure out if you really need to go asynchronous. Asynchronous programming is more complicated that synchronous programming, because, as someone described, it does not fit human brain nicely.

You should use asynchronous programming when your application needs to monitor some resources and react to changes in their state. For example, a web server sitting idle until a request arrives through a socket is an ideal candidate. Or an application that has to execute tasks periodically or delay their execution after some time. The alternative is to use multiple threads (or processes) to control multiple tasks and this model becomes quickly complicated.

The second step is to figure out if you can go asynchronous. Unfortunately in Tornado, not all the tasks can be executed asynchronously.

Tornado is single threaded (in its common usage, although it supports multiple threads in advanced configurations), therefore any “blocking” task will block the whole server. This means that a blocking task will not allow the framework to pick the next task waiting to be processed. The selection of tasks is done by the IOLoop, which, as everything else, runs in the only available thread.

For example, this is a wrong way of using IOLoop:

	import time
	from tornado.ioloop import IOLoop
	from tornado import gen
	def my_function(callback):
	print 'do some work'
	# Note: this line will block!
	time.sleep(1)
	callback(123)
	@gen.engine
	def f():
	print 'start'
	# Call my_function and return here as soon as "callback" is called.
	# "result" is whatever argument was passed to "callback" in "my_function".
	result = yield gen.Task(my_function)
	print 'result is', result
	IOLoop.instance().stop()
	if __name__ == "__main__":
	f()
	IOLoop.instance().start()

view rawasync_generic.py hosted with ❤ by GitHub

Note that blocking_call is called correctly, but, being blocking (time.sleep blocks!), it will prevent the execution of the following task (the second call to the same function). Only when the first call will end, the second will be called by IOLoop. Therefore, the output in console is sequential (“sleeping”, “awake!”, “sleeping”, “awake!”).

Compare the same “algorithm”, but using an “asynchronous version” of time.sleep, i.e. add_timeout:

	# Example of non-blocking sleep.
	import time
	from tornado.ioloop import IOLoop
	from tornado import gen
	@gen.engine
	def f():
	print 'sleeping'
	yield gen.Task(IOLoop.instance().add_timeout, time.time() + 1)
	print 'awake!'
	if __name__ == "__main__":
	# Note that now code is executed "concurrently"
	IOLoop.instance().add_callback(f)
	IOLoop.instance().add_callback(f)
	IOLoop.instance().start()

view rawasync_sleep_1.py hosted with ❤ by GitHub

In this case, the first task will be called, it will print “sleeping” and then it will ask IOLoop to schedule the execution of the rest of the routine after 1 second. IOLoop, having the control again, will fire the second call the function, which will print “sleeping” again and return control to IOLoop. After 1 second IOLoop will carry on where he left with the first function and “awake” will be printed. Finally, the second “awake” will be printed, too. So, the sequence of prints will be: “sleeping”, “sleeping”, “awake!”, “awake!”. The two function calls have been executed concurrently (not in parallel, though!).

So, I hear you asking, “how do I create functions that can be executed asynchronously”? In Tornado, every function that has a “callback” argument can be used with gen.engine.Task. Beware though: being able to use Task does not make the execution asynchronous! There is no magic going on: the function is simply scheduled to execution, executed and whatever is passed tocallback will become the return value of Task. See below:

	import time
	from tornado.ioloop import IOLoop
	from tornado import gen
	def my_function(callback):
	print 'do some work'
	# Note: this line will block!
	time.sleep(1)
	callback(123)
	@gen.engine
	def f():
	print 'start'
	# Call my_function and return here as soon as "callback" is called.
	# "result" is whatever argument was passed to "callback" in "my_function".
	result = yield gen.Task(my_function)
	print 'result is', result
	IOLoop.instance().stop()
	if __name__ == "__main__":
	f()
	IOLoop.instance().start()

view rawasync_generic.py hosted with ❤ by GitHub

Most beginners expect to be able to just write: Task(my_func), and automagically execute my_func asynchronously. This is not how Tornado works. This is how Go works! And this is my last remark:

In a function that is going to be used “asynchronously”, only asynchronous libraries should be used.

By this, I mean that blocking calls like time.sleep or urllib2.urlopen or db.query will need to be substituted by their equivalent asynchronous version. For example, IOLoop.add_timeout instead of time.sleep, AsyncHTTPClient.fetchinstead of urllib2.urlopen etc. For DB queries, the situation is more complicated and specific asynchronous drivers to talk to the DB are needed. For example: Motor for MongoDB.