For a project i wanted to keep one of my table in memcache. I tried to do it using Django ORM but it was taking a hell lot of memory.
Let me tell u some details.
Table Structure : This table has almost 1,300,000 rows
All I wanted was to keep them in cache(memcached), with following format:
memcache Key : movie-id
memcache Data : [ (user1:rating1), (user2:rating2) ]
First Approach : Directly from DB
for r in FlixsterUserRating.objects.all(): ratingDict = cache.get(r.movie.id) if ratingDict is None: cache.set(r.movie.id, [(r.user.id, r.rating)], 86400) else: ratingDict.append((r.user.id, r.rating)) cache.set(r.movie.id, ratingDict, 86400) del ratingDict
In this approach for processing every 1000 it was taking some 5.1 secs. Read More …
Read More …