Why Django ORM Sucks : It takes a hell lot of memory in processing.

For a project i wanted to keep one of my table in memcache. I tried to do it using Django ORM but it was taking a hell lot of memory.

Let me tell u some details.

Table Structure : This table has almost 1,300,000 rows

id movie_id user_id rating
1 1 1 5.0
2 1 2 4.5
3 1 3 4.5
4 2 1 3.5
5 3 3 2.0

All I wanted was to keep them in cache(memcached), with following format:
memcache Key : movie-id
memcache Data : [ (user1:rating1), (user2:rating2) ]

First Approach : Directly from DB

for r in FlixsterUserRating.objects.all(): 
	ratingDict = cache.get(r.movie.id) 
	if ratingDict is None:  
		cache.set(r.movie.id, [(r.user.id, r.rating)], 86400) 
	else:  
		ratingDict.append((r.user.id, r.rating))  
		cache.set(r.movie.id, ratingDict, 86400) 
	del ratingDict

In this approach for processing every 1000 it was taking some 5.1 secs.

Read More …

Installing Memcached for Django Application on Windows XP

For a project I needed Memcached. But the problem was that I have my development system on WindowsXP.

When I Google I got various links to install Memcached on windows, but there wasn’t any link that tell all about Memcached with Django on Windows

So Here we go:

  • Download memcached 1.2.4 Win32.
  • Unzip the binaries in your desired directory (eg. c:\memcached)
  • Install the service using the command: ‘c:\memcached\memcached.exe -d install’ from either the command line
  • Start the server from the Microsoft Management Console or by running the following command: ‘c:\memcached\memcached.exe -d start’
  • Use the server, by default listening to port 11211
  • Now we have to get something so that python can talk with our memcached server.
  • Read More …