While working on topka, I've used the twisted framework. Twisted tutorials are really good with lots of practical examples, anyway after using the framework for weeks, I gain enough experience to write a post about some tricks I've discovered.
Disclaimer: you will not find here something that is not already in the twisted docs.
While using twisted, it happens that you can be a situation where you can either return a
final value or do the computation asynchronously and return a
Deferred. The code looks like:
import twisted.internet.defer as defer def myFunc(): def treatRet(): ... ret = functionThatProcess(...) if isinstance(ret, defer.Deferred): ret.addCallback(treatRet) else: ret = treatRet(ret) return ret
I figured that usually it's a good idea to change the code so that the result of
is always a
Deferred. It's generally a good idea because otherwise all the callers of
also have to handle the two cases of a final result or a ̀
And there comes
>>> import twisted.internet.defer as defer >>> d = defer.succeed('value') >>> def cb(v): ... print "value=", v ... return v ... >>> d.addCallback(cb) value= value <Deferred at 0x7fda5d44b9e0 current result: 'value'> >>>
When the result is
defer.succeed, if you add a callback (or an errback for
defer.fail), it will be called immediately.
And so the original code becomes:
import twisted.internet.defer as defer def myFunc(): def treatRet(): ... ret = functionThatProcess(...) ret.addCallback(treatRet) return ret
functionThatProcess if the return value can be computed directly, we will just return
callbacks will be executed as soon as they are added. Or it's an asynchronous computation and the callbacks will be proceeded
when the value will become available. The bonus is that now we have only one code path.
Sometime you can have complex workflows that requires to launch things, wait for event, ...
And so you can be in a situation where you're in a callback but you would still need to wait
for another event to complete. Of course you can't wait in a callback because it would busy-lock the
reactor. A possibility is to execute things in a thread using
reactor.callInThread. Anyway the most
elegant solution is when the callback returns a
Deferred, the callback chain execution will be stopped
Deferred has completed:
from twisted.internet import defer, reactor def cb1(v): print "in callback 1 with value=", v d = defer.Deferred() reactor.callLater(1.0, d.callback, "modified") return d def cb2(v): print "in callback 2 with value=", v reactor.stop() delayed = defer.Deferred() delayed.addCallback(cb1).addCallback(cb2) reactor.callLater(2.0, delayed.callback, 'val') reactor.run()
The output is:
(2 seconds delay) in callback 1 with value= val (1 second delay) in callback 2 with value= modified
We can see that the calback chain is suspended until the returned
Deferred has completed.
You can also notice that it's the return value of the
Deferred that is passed to the callback
Using twisted with its callback style programming can quickly lead to a spaghetti program. Anyway one can find some piece of elegance in this way of doing.