tag:blogger.com,1999:blog-4195135246107166251.post466875216353064816..comments2023-05-08T07:04:09.641-07:00Comments on Neopythonic: Compare-And-Set in MemcacheGuido van Rossumhttp://www.blogger.com/profile/12821714508588242516noreply@blogger.comBlogger17125tag:blogger.com,1999:blog-4195135246107166251.post-31780546269482235872016-01-27T23:45:01.411-08:002016-01-27T23:45:01.411-08:00Boris, the memcache.add() function will fail if th...Boris, the memcache.add() function will fail if the key exists already, so you could add with the base value and if that fails you know the key exists and you should increment it atomically.Coda Highlandhttps://www.blogger.com/profile/18353997842256813621noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-21612600285545097492015-08-17T14:45:03.304-07:002015-08-17T14:45:03.304-07:00Hey Guido,
Great post. I'm stuck on the situa...Hey Guido,<br /><br />Great post. I'm stuck on the situation that Michael described above, where a key does not yet exist in the cache. For a key that may not be in the cache yet, how can we atomically check that key's value and set it?<br /><br />Thanks,<br />BorisAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-15861552997622809902012-11-07T10:31:35.113-08:002012-11-07T10:31:35.113-08:00Late to the comment party, but wanted to say thank...Late to the comment party, but wanted to say thanks for a very excellent explanation of a very important topic. Not sure if this has made it into the GAE docs, but it really should. Could this somehow be leveraged in a way that would support the holy grail of a monotonically increasing app counter? Everything seems to be here except for the issue of memcache ephemerality. Again, thanks. -stevepAnonymoushttps://www.blogger.com/profile/08491520235058102238noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-11540363565817576862012-04-20T14:45:51.219-07:002012-04-20T14:45:51.219-07:00There was a typo in my previous comment. I meant &...There was a typo in my previous comment. I meant "if counter is *not* in the memcache, then reset=True"Staffhttps://www.blogger.com/profile/15386914080060491430noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-53174024978425704052012-04-12T21:55:15.399-07:002012-04-12T21:55:15.399-07:00Another problem I see with this. How do we set cou...Another problem I see with this. How do we set counter to 0 the first time? cas only works if we've set the memcache value once before. What happens if we want to increment the counter, but we're not sure if there is anything in the memcache. We want to do something like this:<br /><br />A: if counter is in the memcache, then reset=True<br />A: if reset, set counter to 0<br />A: cas(counter+1)<br /><br />But if we have two requests running, something like this could occur<br /><br /><br />A: if counter is in the memcache, then reset=True<br />B: if counter is in the memcache, then reset=True<br />A: if reset, set counter to 0<br />A: cas(counter+1)<br />B: if reset, set counter to 0<br />B: cas(counter+1)<br /><br />Now counter will be 1 instead of 2Staffhttps://www.blogger.com/profile/15386914080060491430noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-81475621710123020522012-04-11T20:11:28.324-07:002012-04-11T20:11:28.324-07:00Isn't a major problem with using the memcache ...Isn't a major problem with using the memcache for this that the memcache can be cleared at any point in time? If we test and then set a value, what's to stop the memcache from being cleared?Staffhttps://www.blogger.com/profile/15386914080060491430noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-12533864787771355782012-01-17T00:14:00.582-08:002012-01-17T00:14:00.582-08:00Hi Guido,
Remembers me (kind of sentimental) of t...Hi Guido,<br /><br />Remembers me (kind of sentimental) of the MC68000 Test-And-Set instruction back in the 80-ies:<br /><a rel="nofollow">MC 68000 Advanced instructions</a> which should be used for the same purpose.<br /><br />cheers,<br />Fredamazinghttps://www.blogger.com/profile/09468128238390306114noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-75543291561903925452011-08-28T20:32:50.751-07:002011-08-28T20:32:50.751-07:00I was obsessing over this a few weeks back when I ...I was obsessing over this a few weeks back when I was wondering about implementing transactional writes to memcache. Another simpler solution presented itself.<br /><br />Also a small typo - there's a "gest()" that should be a "gets()".Anonymoushttps://www.blogger.com/profile/15739907007464929214noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-35610825167214322102011-08-25T14:44:50.786-07:002011-08-25T14:44:50.786-07:00Yes, years too late.
It does exist in Java, I eve...Yes, years too late.<br /><br />It does exist in Java, I even gave a link to the docs: http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/memcache/MemcacheService.html#putIfUntouched%28java.lang.Object,%20com.google.appengine.api.memcache.MemcacheService.IdentifiableValue,%20java.lang.Object%29 . You also need to use http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/memcache/MemcacheService.html#getIdentifiable%28java.lang.Object%29 .Guido van Rossumhttps://www.blogger.com/profile/12821714508588242516noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-9279563580471998592011-08-25T14:41:54.044-07:002011-08-25T14:41:54.044-07:00Waiting for java verison!Waiting for java verison!Araminoshttps://www.blogger.com/profile/03387932583803366505noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-32005175534450110812011-08-25T14:41:20.397-07:002011-08-25T14:41:20.397-07:00This is (strong) load-linked/store-conditional. CA...This is (strong) load-linked/store-conditional. CAS is weaker. Too late to rename your API?Chrishttps://www.blogger.com/profile/07405761543847905584noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-53967532816401994662011-08-25T14:24:30.558-07:002011-08-25T14:24:30.558-07:00Yes, it solves the ABA problem (that's why I e...Yes, it solves the ABA problem (that's why I explained that it works by using timestamps).<br /><br />(For readers like me who had never heard of that problem before: http://en.wikipedia.org/wiki/ABA_problem )Guido van Rossumhttps://www.blogger.com/profile/12821714508588242516noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-25968168823867296082011-08-25T13:59:58.089-07:002011-08-25T13:59:58.089-07:00Does the library handle the ABA problem, or is tha...Does the library handle the ABA problem, or is that left to the user?mmocnyhttps://www.blogger.com/profile/00585754106557615292noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-50447466038448775542011-08-25T12:44:10.262-07:002011-08-25T12:44:10.262-07:00@Guido: Yeah, I bet in real life it's not that...@Guido: Yeah, I bet in real life it's not that bad. I just wanted to know from theoretical point of view :-) Anyway, this reminds me of an university course about synchronized programming and using cas operations to synchronize some tasks. Ah, I'm getting sentimental :-)Filip Gruszczyńskihttps://www.blogger.com/profile/11422797638747568555noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-55614895768238028482011-08-25T11:45:31.627-07:002011-08-25T11:45:31.627-07:00Thanks for the update. I've done some fun thi...Thanks for the update. I've done some fun things with memcached CAS, but it seems to be difficult to explain to people how to use it correctly. :)<br /><br />In most languages where I've implemented memcached CAS, I've found it useful to provide a functional abstraction over it similar to this (which I'm kind of sketching in this comment thing, so it quite possibly has a bug):<br /><br />def do_cas(k, f, initial=None):<br />. client = memcache.Client()<br />. while True:<br />. . ob = client.gets(k)<br />. . if ob:<br />. . if client.cas(k, f(ob)):<br />. . . break<br />. . elif initial:<br />. . . if client.add(k, initial):<br />. . . . break<br />. . else:<br />. . . raise Something("Uninitialized...")Dustinhttps://www.blogger.com/profile/10950273921525189499noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-65235966145821543072011-08-25T11:38:49.222-07:002011-08-25T11:38:49.222-07:00@Filip: You're right it is active waiting, alt...@Filip: You're right it is active waiting, although it is moderated by the time required for memcache server roundtrips (typically 1-5 msec). That's why there should be a retry limit (e.g. max 3 tries).<br /><br />You could also use exponential back-off, but very quickly you'd hit the point where it is probably more important to get on with handling the user's request than trying to bump the counter; while you're waiting to bump the counter you're not doing anything else.<br /><br />In the end this is still just a simplified example to illustrate what cas() does rather than a recommendation on how to implement counters; cas() really is a low-level tool for building more reliable caches.Guido van Rossumhttps://www.blogger.com/profile/12821714508588242516noreply@blogger.comtag:blogger.com,1999:blog-4195135246107166251.post-54342549064417603762011-08-25T11:32:34.020-07:002011-08-25T11:32:34.020-07:00Isn't code snippet for increasing counter usin...Isn't code snippet for increasing counter using active waiting? Isn't it bad?Filip Gruszczyńskihttps://www.blogger.com/profile/11422797638747568555noreply@blogger.com