Django ORM : What to do when the memory requirement goes high?

As the title suggest this post is regarding “Memory Requirement of django ORM”.
Actually some days back I myself faced this problem and wrote “Why Django ORM Sucks : It takes a hell lot of memory in processing“. From various comments and a thread which started on google groups, I came up with few points one should check while operating on large amount of data.

For showing points i’ll use following example:

for r in UserRating.objects.all():
	ratingDict = cache.get(r.movie.id)
	if ratingDict is None:
		cache.set(r.movie.id, [(r.user.id, r.rating)], 86400)
	else:
		ratingDict.append((r.user.id, r.rating))
		cache.set(r.movie.id, ratingDict, 86400)

So here are the mistakes in previous code.

Read More

Django Example: FileField and ImageField

The syntax to add it in Model is:

class MyModel(models.Model):
	video = FileField(upload_to=None[, max_length=100, **options])

Here upload_to is local filesystem path that will be appended to your MEDIA_ROOT setting to determine the value of the url(mymodel.video.url) attribute.

Setting MEDIA_ROOT and upload_to

If,
MEDIA_ROOT = ‘/home/myname/files/
upload_to=’videos’
file: abc.flv
results in: /home/myname/files/videos/abc.flv

where as if,
MEDIA_ROOT = ‘/home/myname/files/’
upload_to=’/videos‘
and file : abc.flv
result will be saved in : ‘/videos/abc.flv’ (i.e., root directory of filesystem)

You can also set upload_to=’videos/%Y/%m/%d’.

Read More

Filmfare nominations 2009 : Bollywood

After months of speculation, the Filmfare nominations are finally out. Time to see who will take the ‘beautiful dark lady’ home this year! Will act in Rab ne.., happy and ‘gay’ Dostana or 8-packed Aamir Khan?
Even as the stars get ready for the big night, stay glued to get all the inside dope and starry controversies surrounding the event. So, who are you betting on this year?

THE NOMINATIONS ARE…

Best Film

Dostana
Ghajini
Jaane Tu… Ya Jaane Na
Jodhaa Akbar
Rab Ne Bana Di Jodi
Rock On!!

Read More

Why Django ORM Sucks : It takes a hell lot of memory in processing.

For a project i wanted to keep one of my table in memcache. I tried to do it using Django ORM but it was taking a hell lot of memory.

Let me tell u some details.

Table Structure : This table has almost 1,300,000 rows

id movie_id user_id rating
1 1 1 5.0
2 1 2 4.5
3 1 3 4.5
4 2 1 3.5
5 3 3 2.0

All I wanted was to keep them in cache(memcached), with following format:
memcache Key : movie-id
memcache Data : [ (user1:rating1), (user2:rating2) ]

First Approach : Directly from DB

for r in FlixsterUserRating.objects.all(): 
	ratingDict = cache.get(r.movie.id) 
	if ratingDict is None:  
		cache.set(r.movie.id, [(r.user.id, r.rating)], 86400) 
	else:  
		ratingDict.append((r.user.id, r.rating))  
		cache.set(r.movie.id, ratingDict, 86400) 
	del ratingDict

In this approach for processing every 1000 it was taking some 5.1 secs.

Read More

Installing Memcached for Django Application on Windows XP

For a project I needed Memcached. But the problem was that I have my development system on WindowsXP.

When I Google I got various links to install Memcached on windows, but there wasn’t any link that tell all about Memcached with Django on Windows

So Here we go:

  • Download memcached 1.2.4 Win32.
  • Unzip the binaries in your desired directory (eg. c:\memcached)
  • Install the service using the command: ‘c:\memcached\memcached.exe -d install’ from either the command line
  • Start the server from the Microsoft Management Console or by running the following command: ‘c:\memcached\memcached.exe -d start’
  • Use the server, by default listening to port 11211
  • Now we have to get something so that python can talk with our memcached server.

    • Read More

    Dictionary on double click : BookMarklet

    While visiting various sites (specially those fucking Raja Sen Movie reviews), I at times need dictionary. When I don’t know meaning of any word I have to open dictionay.com and than search for it.

    So, to get a solution of this is came with a Bookmarklet, which opens information bubble on double-clicking on any word in the page.
    But to launch dictionary you have to click on this – Dictionary : DblClick

    Bookmark above link for future use.

    Example : The meaning of “Reference” is shown in a small bubble.

    Read More

    WTF : What the Framework bookmarklet : Find the javascript framework being used

    Yesterday I was hacking the source code of a website. While doing I needed the javascript library being used in that site.
    Although on accessing various javascript files i came to know that Prototype was used in that site.

    Later I came to know about a bookmarklet : WTFramework. Just add this to on your bookmark.

    Just click this bookmark whenever there is a need to know the JavaScript frameworks (MooTools, Yahoo User Interface, jQuery, the infamous Scriptaculous, Dojo Toolkit, Mochikit, etc..

    Read More

    Happy Valentine’s Day : History Valentine’s Day


    Valentine’s Day or Saint Valentine’s Day is a holiday celebrated on February 14 by many people throughout the world. In the West, it is the traditional day on which lovers express their love for each other by sending Valentine’s cards, presenting flowers, or offering confectionery. The day was originally a pagan festival that was renamed after two Early Christian martyrs named Valentine.


    The day became associated with romantic love in the circle of Geoffrey Chaucer in the High Middle Ages, when the tradition of courtly love flourished.

    Read More