tag:blogger.com,1999:blog-4195135246107166251.post255580926374362830..comments2023-05-08T07:04:09.641-07:00Comments on Neopythonic: Asynchronous RPC in App Engine TodayGuido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-4195135246107166251.post-28208879538343115522011-10-24T12:26:28.689-07:002011-10-24T12:26:28.689-07:00How to determine what the URL fetch RPC object?How to determine what the URL fetch RPC object?masterhttps://www.blogger.com/profile/05925909018149264400noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-81950032721994432022011-01-29T15:15:27.810-08:002011-01-29T15:15:27.810-08:00Thank you for the reply. This is great information...Thank you for the reply. This is great information which is not apparent from the docs. I really appreciate it.<br /><br />Thank You,<br />VishUnknownhttps://www.blogger.com/profile/06658949550934707184noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-49867174467963947362011-01-28T09:49:45.862-08:002011-01-28T09:49:45.862-08:00Vish:
- Making your own async versions of standar...Vish:<br /><br />- Making your own async versions of standard App Engine APIs is not recommended; it would be possible by reverse engineering the synchronous API code (the source code is all in the SDK) but we cannot guarantee the stability of such reverse-engineered solutions -- when we change the underlying protocol we make sure to change the client library too, but your reverse-engineered solution might break. Please file bugs in the App Engine issue tracker for specific APIs you'd like to see grow an async variant.<br /><br />- The "functional programming" solution you mention looks like it would be based on threads; this is not available in App Engine. In Python itself, though, you might be interested in PEP 3148.<br /><br />- You are not paying for CPU time while wait_any() is blocked. Of course the real time clock ticks on, but this does not affect what yuo are charged. The real time taken by requests may factor into how future requests will be scheduled, but using async APIs most likely makes your total elapsed real time smaller, which will improve your scheduling. :-)<br /><br />- The async calls do not create threads. At the lowest level we receive callbacks from the infrastructure -- in fact at the lowest level even synchronous calls are implemented using a fundamentally asynchronous implementation!<br /><br />Hope this answers all your questions,<br /><br />--GuidoGuido van Rossumhttps://www.blogger.com/profile/12821714508588242516noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-73764148673591126152011-01-28T06:04:12.726-08:002011-01-28T06:04:12.726-08:00Hi Guido,
Is there any documentation on how to ma...Hi Guido,<br /><br />Is there any documentation on how to make your own functions/modules support async? I would love to read those. Your new Datastore API might solve the problem for Datastore calls, but i have other requirements like having to generate thumbnails using the PIL for multiple images. I would ideally love to do those calls async. But there is nothing available for me to do it currently. Another example is also, when I am constructing a large KML file, i would ideally love to do this in chunks aync and merge them together. So, docs on how to do this would be very helpful.<br /><br />I am not an advanced Python user, but i thought one of the great things about functional programming is it makes async programming simple. That is, i had expected to just call any function async by calling a helper function and passing in the function to executed async and a callback. Is this not possible in python?<br /><br />Also, in the url_fetch examples you have shown, during the wait_any & the wait_all calls, are we paying appengine costs when the handler is waiting on external HTTP requests? If so, is there a way around. Can there be a concept of an AsyncRequestHandler like from other programming frameworks which would allow the thread to do something else while it is waiting on the request and thus avoid appengine costs.<br /><br />If the url_fetch HTTP async calls are creating new threads, is there a limit to the number of async calls that can be made? Is the limit per handler or for all concurrent handlers summed together?<br /><br />Thank You,<br />Vish<br /><br />Thank You,<br />VishUnknownhttps://www.blogger.com/profile/06658949550934707184noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-63725940220706248382011-01-24T13:47:40.198-08:002011-01-24T13:47:40.198-08:00I know of asynctools -- it currently works but it ...I know of asynctools -- it currently works but it uses undocumented internal APIs that we cannot guarantee will keep working. The groundwork we did for Datastore Plus included providing a more maintainable API (mainly, datastore_rpc.py and datastore_query.py). While this new API is not yet documented, we feel comfortable that we can promise not to break 3rd party tools built on top of this API.<br /><br />However, Datastore Plus will eventually make asynctools unnecessary so there may be no need to develop another 3rd party library for async datastore interaction.Guido van Rossumhttps://www.blogger.com/profile/12821714508588242516noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-42453647027727820182011-01-24T13:40:39.939-08:002011-01-24T13:40:39.939-08:00A friend of mine put together AsyncTools a while t...A friend of mine put together <a href="http://code.google.com/p/asynctools/" rel="nofollow">AsyncTools</a> a while to do datastore retrieval async and in parallel, but I don't know if they're still being maintained.Tony Arkleshttps://www.blogger.com/profile/01288738871727960441noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-90294074722473230672011-01-24T13:34:12.145-08:002011-01-24T13:34:12.145-08:00If you dig deep enough (nearly) every App Engine A...If you dig deep enough (nearly) every App Engine API can be called asynchronously. Eventually we may publish and document stable APIs for more of them. Datastore Plus will do this for the datastore.Guido van Rossumhttps://www.blogger.com/profile/12821714508588242516noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-58486560949656850172011-01-24T12:31:37.318-08:002011-01-24T12:31:37.318-08:00Is it possible that all services (over google prot...Is it possible that all services (over google protocoll buffer) are asynchroc callable in the app engine? I found the first tries on <a href="http://blog.notdot.net/2010/09/Under-the-hood-with-App-Engine-APIs" rel="nofollow">Nick Johnson blog</a> and it make sense for many use cases.wingihttps://www.blogger.com/profile/18044191896934550541noreply@blogger.com