Blog entries on April 2008

  • Internet lifestream with Django

    By Nuno Mariz, on 4 April 2008 @ 01:30
    My goal was to archive and display my internet lifestream. My first approach was writing a client for each API of the social networks that I'm in.
    This turned out to be a complete waste of time and effort. All that I needed after all was a FriendFeed account that would centralize all my feeds.

    Archiving and displaying your entries with Django is quite simple.
    First of all, you need to download the Python FriendFeed API client. Then start a new application in your project, lets call it lifestream:

    ./manage.py startapp lifestream

    On the settings.py add the lifestream project to the INSTALLED_APPS and a variable to store your FriendFeed username:

    FRIENDFEED_USERNAME = 'your_username'

    In the models.py add a model named Entry:

    from django.db import models
    
    class Entry(models.Model):
        id = models.CharField(max_length=255, primary_key=True)
        service_id = models.CharField(max_length=50, null=True, blank=True)
        service_name = models.CharField(max_length=50, null=True, blank=True)
        service_icon = models.URLField(max_length=255, verify_exists=False, null=True, blank=True)
        service_profile = models.URLField(max_length=255, verify_exists=False, null=True, blank=True)
        title = models.CharField(max_length=255, null=True, blank=True)
        link = models.URLField(max_length=255, verify_exists=False, null=True, blank=True)
        updated = models.DateTimeField(null=True, blank=True)
        published = models.DateTimeField(null=True, blank=True)
        media_title = models.CharField(max_length=255, null=True, blank=True)
        media_link = models.URLField(max_length=255, verify_exists=False, null=True, blank=True)
        media_thumbnail = models.URLField(max_length=255, verify_exists=False, null=True, blank=True)
        created = models.DateTimeField(auto_now_add=True)
    
        def __unicode__(self):
            return self.title
    
        class Meta:
            ordering = ['-published']
            verbose_name = 'Entry'
            verbose_name_plural = 'Entries'
    
        class Admin:
            list_display = ['title', 'service_name', 'published']
            list_filter = ['service_name']
            date_hierarchy = 'published'

    Create an url.py on the lifestream folder:

    from django.conf.urls.defaults import *
    from lifestream.models import Entry
     
    entry_list_dict = {
        'queryset' : Entry.objects.all(),
        'paginate_by' : 30,
    }
    
    urlpatterns = patterns('',   
        (r'^$', 'django.views.generic.list_detail.object_list', entry_list_dict),
    )

    As you can see, I've used a generic view. You can also use the date based generic views and pagination to build an archive like mine.

    Add to your project root urls.py:

    (r'^lifestream/', include('lifestream.urls'))

    Create a template lifestream/entry_list.html:

    {% for entry in object_list %}
    <div class="source">
      <a href="{{ entry.service_profile }}" title="{{ entry.service_name }}"><img src="{{ entry.service_icon }}" alt="{{ entry.service_name }}" alt="{{ entry.service_name }}" /></a>
    </div>
    <div class="details">
      <ul>
        <li><a href="{{ entry.link }}">{{ entry.title }}</a></li>
        <li>{{ entry.published|timesince }} ago</li>
        {% if entry.media_thumbnail %}<li><a href="{{ entry.media_link }}"><img src="{{ entry.media_thumbnail }}" alt="{{ entry.media_title }}" /></a></li>{% endif %}
      </ul>
    </div>
    {% endfor %}

    Finally, create a script to synchronize your feeds:

    #!/usr/bin/env python
    # -*- coding: utf-8 -*-
    
    import sys
    import os
    
    ROOT_PATH = os.path.realpath(os.path.dirname(__file__))
    PROJECT_PATH, PROJECT_DIR = os.path.split(ROOT_PATH)
    
    sys.path.insert(0, ROOT_PATH)
    sys.path.insert(1, PROJECT_PATH)
    
    os.environ['DJANGO_SETTINGS_MODULE'] = '%s.settings' % PROJECT_DIR
    
    from friendfeed import FriendFeed
    from django.conf import settings
    from lifestream.models import Entry
    
    ff = FriendFeed()
    feed = ff.fetch_user_feed(settings.FRIENDFEED_USERNAME)
    
    for e in feed.get('entries'):
        entry, created = Entry.objects.get_or_create(id=e.get('id'))
        if created:
            service = e.get('service')
            entry.service_id = service.get('id')
            entry.service_name = service.get('name')
            entry.service_icon = service.get('iconUrl')
            entry.service_profile = service.get('profileUrl')
            entry.title = e.get('title')
            entry.link = e.get('link')
            entry.updated = e.get('updated')
            entry.published = e.get('published')
            media = e.get('media')
            if media:
                entry.media_title = media[0].get('title')
                entry.media_link = media[0].get('player') or entry.link
                thumbnails = media[0].get('thumbnails')
                entry.media_thumbnail = thumbnails[0].get('url')
            entry.save()

    If you want, you can add a job in your crontab:

    # synchronize every 15 mins
    */15 * * * *   root   /path/to/your/application/lifestream_cron.py

    See my lifestream as the working example.

    UPDATE: Friendfeed sends the time in UTC, if you want to use your timezone you have do some hacking:

    Install pytz:

    easy_install pytz

    Import and assign your timezone to a variable:

    import pytz
    tz = pytz.timezone(settings.TIME_ZONE)

    And replace entry.updated and entry.published with:

    updated = e.get('updated')
    updated = updated.replace(tzinfo=pytz.utc).astimezone(tz)
    published = e.get('published')
    published = published.replace(tzinfo=pytz.utc).astimezone(tz)
    if settings.DATABASE_ENGINE == 'mysql': # http://code.djangoproject.com/ticket/5304
        updated = updated.replace(tzinfo=None)
        published = published.replace(tzinfo=None)
    entry.updated = updated
    entry.published = published

    Thanks to Chris Kelly that send me an email reporting this.