Using Python to Compare Document IDs in Two CouchDB Databases

I’m doing a bit of research into what may or may not be an issue with a specific database in our BigCouch cluster, but regardless of the outcome of that side of things I thought I’d share how I used Python and couchdb-python to dig into the problem.

In our six-server BigCouch cluster we noticed that on the database for one of our most heavily trafficked applications the document counts displayed in Futon for each of the cluster members don’t match. As I said above this may or may not be a problem (I’m waiting on further information on that particular point), but I was curious which documents were missing from the cluster member that has the lowest document count. (The interesting thing is the missing documents aren’t truly inaccessible from the server with the lower document count, but we’ll get to that in a moment.)

BigCouch is based on Apache CouchDB but adds true clustering as well as some other very cool features, but for those of you not familiar with CouchDB, you communicate with CouchDB through a RESTful HTTP interface and all the data coming and going is JSON. The point here is it’s very simple to interact with CouchDB with any tool that talks HTTP.

Dealing with raw HTTP and JSON may not be difficult but isn’t terribly Pythonic either, which is where couchdb-python comes in. couchdb-python lets you interact with CouchDB via simple Python objects and handles the marshaling of data between JSON and native Python datatypes for you. It’s very slick, very fast, and makes using CouchDB from Python a joy.

In order to get to the bottom of my problem, I wanted to connect to two different BigCouch cluster members, get a list of all the document IDs in a specific database on each server, and then generate a list of the document IDs that don’t exist on the server with the lower total document count.

Here’s what I came up with:

>>> import couchdb
>>> couch1 = couchdb.Server(‘http://couch1:5984/’)
>>> couch2 = couchdb.Server(‘http://couch2:5984/’)
>>> db1 = couch1[‘dbname’]
>>> db2 = couch2[‘dbname’]
>>> ids1 = []
>>> ids2 = []
>>> for id in db1:
…     ids1.append(id)
… 
>>> for id in db2:
…     ids2.append(id)
… 
>>> missing_ids = list(set(ids1) – set(ids2))

What that gives me, thanks to the awesomeness of Python and its ability to subtract one set from another (note that you can also use the difference() method on the set object to achieve the same result), is a list of the document IDs that are in the first list that aren’t in the second list.

The interesting part came when I took one of the supposedly missing IDs and tried to pull up that document from the database in which it supposedly doesn’t exist:

>>> doc = db2[‘supposedly_missing_id_here’]

I was surprised to see that it returned the document just fine, meaning it must be getting it from another member of the cluster, but I’m still digging into what the expected behavior is on all of this. (It’s entirely possible I’m obsessing over consistent document counts when I don’t need to be.)

So what did I learn through all of this?

  • The more I use Python the more I love it. Between little tasks like this and the fantastic experience I’m having working on our first full-blown Django project, I’m in geek heaven.
  • couchdb-python is awesome, and I’m looking forward to using it on a real project soon.
  • Even though we’ve been using CouchDB and BigCouch with great success for a couple of years now, I’m still learning what’s going on under the hood, which for me is a big part of the fun.

CouchDB Tip: When You Can’t Stop the Admin Party

I was setting up a new CouchDB 1.2 server today on Ubuntu Server, specifically following this excellent guide since sudo apt-get install couchdb still gets you CouchDB 0.10. Serious WTF on the fact that the apt installation method is years out of date — maybe I should figure out who to talk to about it and volunteer to maintain the packages if it’s just a matter of people not having time.

The installation went fine until I attempted to turn off the admin party, at which point after I submitted the form containing the initial admin user’s name and password things just spun indefinitely. And apparently adding the admin user info manually to the [admin] section of the local.ini file no longer works, since it doesn’t automatically encrypt the password you type into the file on a server restart like it used to.

Long and short of it is if you see this happening, chances are there’s a permission problem with your config files, which are stored (if you compile from source) in /usr/local/etc/couchdb. In my case that directory and the files therein were owned by root and I’m not running CouchDB as root, so when I tried to fix the admin party the user that’s running CouchDB didn’t have permission to write to the files.

A quick chown on that directory structure and you’re back to being an admin party pooper.

Three Approaches to Handling Static Files in Django

I had a really great (and lengthy) pair programming session today with a coworker during which we spent a bit of time going over a couple of different approaches for dealing with static files in Django, so I thought I’d document and share this information while it’s fresh in my mind.

First, a little background. If you’re not familiar with Django it was originally created for a newspaper web site, specifically the Lawrence Journal-World, so the approach to handling what in the Django world are called “static files” — meaning things like images, JavaScript, CSS, etc. — is based on the notion that you might be using a CDN so you should have maximum flexibility as to where these files are located.

While the flexibility is indeed nice, if you’re used to a more self-contained approach it takes a little getting used to, and there are a few different ways to configure your Django app to handle static files. I’m going to outline three approaches, but using different combinations of things and other solutions of which I may be unaware there are certainly more ways to handle static files than what I’ll outline here. (And as I’m still relatively new to Django, if I’m misunderstanding any of this I’d love to hear where I could improve any of what I’m sharing here!)

One other caveat — I’m focusing here on STATIC_URL and ignoring MEDIA_URL but the approach would be the same.

Commonalities Across All Approaches

First, even though it may not strictly be required depending on which approach you take for handling static files, since you wind up needing to use this for other reasons, we’ll use django.template.RequestContext in render_to_response as opposed to the raw request object. This is required if you want access to settings like MEDIA_URL and STATIC_URL in your Django templates. For more details about RequestContext, the TEMPLATE_CONTEXT_PROCESSORS that are involved behind the scenes, and the variables this approach puts into your context, check the Django docs.

I’m also operating under the assumption that the static files will live in a directory called static that’s under the main application directory inside your project directory (i.e. the directory that has your main settings.py file in it). Depending on the approach you use you may be able to put the static directory elsewhere, but unless stated otherwise, that’s where the directory is assumed to be. (Note that if you store static files on another server entirely, such as using a CDN, STATIC_URL can be a full URL as opposed to a root-relative URL like /static/)

Also in all examples it’s assumed that the STATIC_URL setting in the main settings.py file is set to ‘/static/’

Approach One (Basic): Use STATIC_URL Directly in Django Templates

This is the simplest approach and may be all you need. With STATIC_URL set to ‘/static/’ in the main settings.py file, all you really have to worry about is using RequestContext in your view functions and then referencing {{ STATIC_URL }} in your Django templates.

Here’s a sample views.py file:

from django.shortcuts import render_to_response
from django.template import RequestContext

def index(request, template_name=’index.html’):
    return render_to_response(template_name, context_instance=RequestContext(request))

By using RequestContext the STATIC_URL variable will then be available to use in your Django templates like so:

<html>
<head>
    <script src=”{{ STATIC_URL }}scripts/jquery/jquery-1.8.1.min.js”></script>
</head>
<body>
    <img src=”{{ STATIC_URL }}images/header.jpg” />
</body>
</html>

That’s all there is to it. Again, since /static/ will be relative to the root of the main application directory in your project it’s assumed that the static directory is underneath your main application directory for this example, and obviously in the case of the example above that means that underneath the static directory you’d have a scripts and images directory.

Approach Two: Use a URL Pattern, django.views.static.serve, and STATICFILES_DIRS

In this approach you leverage Django’s excellent and hugely flexible URL routing to set a URL pattern that will be matched for your static files, have that URL pattern call the django.views.static.serve view function, and set the document_root that will be passed to the view function to a STATICFILES_DIRS setting from settings.py. This is a little bit more involved than the first approach but gives you a bit more flexibility since you can place your static directory anywhere you want.

The approach I took with this method was to set a CURRENT_PATH variable in settings.py (created by using os.path.abspath since we need a physical path for the document root) and leverage that to create the STATICFILES_DIRS setting. Here’s the relevant chunks from settings.py:

import os
CURRENT_PATH = os.path.abspath(os.path.dirname(__file__).decode(‘utf-8’)).replace(‘\’, ‘/’)

STATICFILES_DIRS = (
    os.path.join(CURRENT_PATH, ‘static’),
)

Note that the replace(‘\’, ‘/’) bit at the end of the CURRENT_PATH setting is to make sure things work on Windows as well as Linux.

Next, set a URL pattern in your main urls.py file:

from django.conf.global_settings import STATICFILES_DIRS

urlpatterns = patterns(”,
    url(r’^static/(?P<path>.*)$’, ‘django.views.static.serve’, {‘document_root’: STATICFILES_DIRS}),
)

And then in your Django templates you simply prefix all your static assets with /static/ as opposed to using {{ STATIC_URL }} as a template variable. Even though you’re specifying /static/ explicitly in your templates, you still have flexibility to put these files wherever you want since the URL pattern acts as an alias to the actual location of the static files.

Approach Three: Use staticfiles_urlpatterns and {% get_static_prefix %} Template Tag

django.contrib.staticfiles was first introduced in Django 1.3 and was designed to clean up, simplify, and create a bit more power for static file management. This approach gives you the most flexibility and employs a template tag instead of a simple template variable when rendering templates.

First, in settings.py we’ll do the same thing we did in the previous approach, namely setting a CURRENT_PATH variable and then use that to set the STATICFILES_DIRS variable:

import os
CURRENT_PATH = os.path.abspath(os.path.dirname(__file__).decode(‘utf-8’)).replace(‘\’, ‘/’)

STATICFILES_DIRS = (
    os.path.join(CURRENT_PATH, ‘static’),
)

Next, in urls.py we’ll import staticfiles_urlpatterns from django.contrib.staticfiles.urls and call that function to add the static file URL patterns to the application’s URL patterns:

from django.contrib.staticfiles.urls import staticfiles_urlpatterns

urlpatterns = patterns(
    # your app’s url patterns here
)

urlpatterns += staticfiles_urlpatterns()

The final line there is what adds the static file URL patterns into the mix. If you output staticfiles_urlpatterns() you’ll see it’s something like so:

[<RegexURLPattern None ^static/(?P<path>.*)$>]

And finally, at the very top of your templates you load the static template tags and then simply use the {% get_static_prefix %} tag to render the static URL:

{% load static %}
<html>
<head>
    <script src=”{% get_static_prefix %}scripts/jquery/jquery-1.8.1.min.js”></script>
</head>
<body>
    <img src=”{% get_static_prefix %}images/header.jpg” />
</body>
</html>

Conclusion

So there you have it, three approaches that more or less accomplish the same thing, but depending on the specific needs of your application or environment one approach may work better for you than another.

For our purposes on our current application we’re using the first approach outlined above since it’s simple and meets our needs, but it’s great to know there’s so much flexibility around static file handling in Django when you need it. As always read the docs for more information and yet more options for managing static files in your Django apps.

Installing python-ldap in virtualenv on Ubuntu

We’re authenticating against Active Directory in our current Python/Django project and though we’ve had excellent luck with python-ldap in general, I ran into issues when trying to install python-ldap in a virtualenv this afternoon. As always a lot of DuckDuckGoing and a minimal amount of headbanging led to a solution.

The error I was getting after activating the virtualenv and running pip install python-ldap was related to gcc missing, which was misleading since that wasn’t actually the issue:

error: Setup script exited with error: command ‘gcc’ failed with exit status 1

To add to the weirdness, when I installed python-ldap outside the context of a virtualenv, everything worked fine.
I’ll save you the blow-by-blow and just tell you that on my machine at least, other than the required OpenLDAP installation and some other libraries, I also had to install libsasl2-dev:

sudo apt-get install libsasl2-dev

Once I had that installed, I could activate my virtualenv, run pip install python-ldap and install without errors.
If you still run into issues make sure (in addition to OpenLDAP) to have these packages installed:
  • python-dev
  • libldap2-dev
  • libssl-dev
Hope that saves someone else some time!

A Week at a Conference with the Dell Sputnik

I attended DjangoCon a couple of weeks ago (which was awesome!) so that was my first opportunity to spend some concentrated quality time with the Dell Sputnik and put it through its paces.

As you’ve probably guessed from my previous posts I’m pretty enamored with the Sputnik, and using it full-time for a week straight didn’t change my opinion. It’s a particularly great machine to travel with since it’s the ideal combination of light but still powerful enough to use for “real” work. In years past I’ve traveled with only a 10.1″ netbook and that was a miserable experience, but the 13″ screen and fantastic battery life of the Sputnik make it the ideal conference companion and I could still catch up on work in the evenings without feeling like I had one hand tied behind my back.

Battery Life

Day one of DjangoCon was a full day of extended tutorials so I used the Sputnik all day to take notes in Zim. There was no wi-fi available so I ran the Sputnik with wi-fi disabled (had there been a decent wi-fi signal I probably would have taken notes in Google Docs), and I did disable Bluetooth as well since I wasn’t using it. I also dimmed the screen to its lowest level since the room was dimly lit anyway and that brightness setting was more than enough to be usable.

In the second tutorial of the day I did enable wi-fi and used my Verizon hotspot since there were some materials to download for the second tutorial. I also close the lid putting the Sputnik into suspend mode when there were breaks, which accounted for a total of an hour of suspend time, and I powered it down completely during the lunch break.

Note that this is a bit of a contrast in my previous battery test when I ran the Sputnik with all the default settings and didn’t even try to conserve battery life, and in that case I still got more than 8 hours out of the battery.

All told the clock time for the day spanned about 7.5 hours including lunch and breaks, and in using the Sputnik with wi-fi on for about 3 hours of that, suspending during breaks, and powering off during lunch, I ended the day with nearly 50% of the battery remaining. Nice.

On the other days of the conference I followed a similar pattern but only powered down during lunch and didn’t ever shut the lid otherwise, and I was using wi-fi most of the day. The schedule on these days ran from 9 am – 6 pm, and even with this longer time span and wi-fi enabled all day, I’d still end each day with about 25% of my battery remaining.

In short, the battery life is darn impressive and beyond up to the task of living through a conference without having access to power during the day.

General Usability

Since I got the Sputnik right before I left for DjangoCon, this also gave me my first opportunity to use the Sputnik as my only machine for a week straight.

I won’t repeat what I said in my previous “initial impressions” post but I will say all those initial impressions held up during the week. The Sputnik is an exceptionally well built machine and is a real pleasure to use. I continued to be impressed with the vividness of the screen and the keyboard is absolutely fantastic.

The only complaint I have from using the Sputnik more heavily for a week is the trackpad. I’m sure this can be resolved through a bit of additional driver work and some setting tweaking, but I did find on occasion that even though I had “tap to click” disabled, and “disable trackpad while typing” turned on, the cursor would occasionally jump around the screen while I was typing, or inadvertently fire click events while I was using the trackpad.

Note there are some setting changes recommended in the Sputnik forums that I enabled which helped quite a bit, but it was still a bit of an issue. On an annoyance scale of 1 – 10 I’d put it at maybe a 3, so not a huge thing but was noticeable. There are also some active bug reports on this issue so I suspect with some driver updates (Sputnik is technically in beta, remember!) they’ll get this resolved.

Summary

After living with the Sputnik as my only machine for a week I continue to be extremely impressed. Particularly in a developer conference situation where power isn’t available at every seat and you have to fight for the few outlets that are available, the Sputnik’s fantastic battery life let me focus on the conference instead of worrying about whether or not my laptop was going to conk out.

Other than the occasional issues with the trackpad I thoroughly enjoyed using the Sputnik at DjangoCon — it makes a great conference companion!

Dell Sputnik: Battery Life Test

After I got the Sputnik set up the way I wanted I figured it was time to test the battery life, so yesterday afternoon I unplugged the power cord and did normal “work stuff” to see how much time I’d get out of the battery under normal working conditions.

Note that I’m not a professional hardware tester or benchmarker, so this was not a scientific experiment resulting in reams of data that can be sliced and diced 100 different ways. To me those charts and graphs always seem artificial anyway (though I suspect it’s quite efficient to do things that way if you test hardware for a living), so my methodology was, quite simply, to use the laptop unplugged until the battery ran out.

Applications Running During Testing

As far as specifically what I was doing and what applications I had running, I was doing Python/Django development yesterday, so here’s what I had running:
  • Emacs
  • Sublime Text 2
  • MySQL
  • Chrome with 6-10 tabs open
  • Terminal with 3-4 tabs open
  • Pidgin IM client
  • WiFi
  • Bluetooth (I didn’t use this, but it’s on by default so it was running)
I intentionally did not dim the screen to its lowest usable setting since I didn’t want to skew the results too much. I just left the screen brightness at the default setting and let it automatically increase and decrease the brightness based on the default power settings.
I also did not (and I don’t even know if you can do this now that I think about it) disable the backlighting on the keyboard, so it would turn on and off depending on the ambient lighting at any given time.

Testing Process

As I alluded to above you won’t find a lot of scientific methodology here, but I figured I’d say a couple of words about what I did specifically during the battery drain period.
If you’re a developer you’ll already know the gist of this, but basically I was hopping back and forth between Emacs and Sublime Text 2 (I was working on converting a CFML application to Python and it was easier to have the CFML app up in Sublime and do the Python work in Emacs), the browser, and the terminal to run scripts or do something in MySQL. I also would start/stop the Django application, which is basically a little web server. (I did not install Apache on this machine yet since I’m pondering switching over to Nginx anyway.)
Also I’d occasional get up to get a cup of coffee, etc. so there were brief periods here and there when I wasn’t using the machine at all, but not so long that it went into hibernation mode.
That’s about it for the process discussion — I just used the thing as I normally would any computer on a normal work day.

Observations

With the default power settings the Sputnik did a great job of being efficient without being annoying. The screen would dim after a couple of minutes if I was reading something and not typing or using the touchpad and it would raise the brightness again once I moved the mouse or typed. I mention this because some machines I’ve had in the past were extremely aggressive with power-saving functionality (constantly “strobing” or adjusting the screen brightness, for example) to the point of me having to turn off the power management altogether which kind of defeats the purpose. So the default settings on this machine with Ubuntu work quite well.
Taken on balance the time remaining in the battery monitor was quite accurate, though I did notice it would jump up when I was doing something like reading and not interacting with the machine. For example I’d bring up a tab in Chrome with something I needed to read, and immediately after doing that I’d check the time remaining and let’s say it said I had 3 hours left. After reading something (i.e. not typing or moving the mouse) I’d check the time remaining again and it’d say I had 6 hours left. Then after using the machine for a minute I’d check again, and it’d be back to the original number of about 3 hours.
I have no engineering rationale to back this up, but as I was seeing this behavior I likened it in my head to the real-time gas mileage gauge in a car. If I’m braking down a hill in my Prius, it’ll tell me I’m getting “infinite” miles per gallon, which is basically a projection based on what’s going on at that point in time. The battery meter seemed to be behaving similarly, projecting battery life based on the activity at a given moment.
Taking the high estimation outliers out of the equation, however, it was quite accurate, even down to the warnings at the end when the battery meter (and the light on the front of the computer, which is a nice touch) turned red. On some machines I’ve had in the past when it says you have 20 minutes left, that may turn to a “plug it in right now” message in about 5 minutes, but that didn’t happen here. When it said I had 17 minutes left, it meant I had 17 minutes left.

Hibernation

If you know anything about running GNU/Linux on laptops you’ll know that hibernation working correctly can be a bit of a problem if the hardware and drivers don’t play nicely together.
I’m happy to report that the hibernation function on the Sputnik works exactly as you’d expect — you simply close the lid, and a few seconds later the light on the front of the machine will start flashing slowly, indicating it’s in hibernation mode. When you re-open the screen things come back quite quickly including reconnecting to WiFi, so there are no problems at all with hibernation.

Results

One more proclamation of how unscientific this was — I didn’t run a stopwatch while doing this. I just checked the time when I started and checked it again when the battery was more or less dead.
The results are quite impressive, with a run time of about 8 hours 20 minutes in my usage. The marketing materials for this machine claim between 8 and 9 hours so these numbers are quite legitimate in comparison to my usage, and if you disabled WiFi and Bluetooth and dimmed the screen, I can see pushing 9 hours of run time pretty easily.
This is all the more impressive when I compare it to my old Asus Eee netbook, which was a vastly less capable machine and would get 10 hours of battery life only with the larger 9-cell battery that not only stuck out the back of the machine like a tumor but also greatly added to the weight of the machine. So for the Sputnik to get this kind of battery life in normal usage in such a small, light package is really great.
Another big plus for the Sputnik!
I still haven’t covered installation and some settings tweaks so I’ll do that in another post soon.

Dell Sputnik Ultrabook: Initial Impressions

I got word a couple of days ago that I got into the Dell “Sputnik” beta program, which is Dell’s incubation project to bring a GNU/Linux-based (specifically Ubuntu-based) XPS 13 Ultrabook to market as a product, specifically as a machine aimed at developers. (We developers, lone wolves though we tend to be, do like a little attention now and then!)

As a huge free software and Linux proponent I have been watching this project closely so I was really excited to get an invite to the beta, and I received the machine today. General Disclaimer: The machine was not free. Dell did offer a discount for beta program participants but is heavily encouraging everyone to be very transparent and honest with their impressions of the machine.

In another blog post I’ll cover how I went about installing Ubuntu and the Sputnik-specific PPAs, but in this post I’ll share my initial impressions of the machine. (tl;dr version: With one minor exception, this is an absolutely beautiful, exceedingly well-built ultrabook that’s an absolute pleasure to use.)

This is a DELL?

As I said above I’ll cover Ubuntu installation in another post, but as a very quick aside — the machine does ship with Windows 7, the explanation being they didn’t want to delay shipping the beta machines to get the logistics of pre-installing Ubuntu worked out. On first boot I booted to an Ubuntu USB stick so I never let Windows see the light of day.
As the heading above indicates, to be perfectly honest I was shocked that this is a Dell. Admittedly I haven’t owned a Dell in many years though I’ve had several Dell laptops and desktops in the past, but everything about this machine screams high-end quality. (You can see my quick, too dark unboxing photos on Google+) From the packaging to the machine itself, they clearly put a lot of thought into the design from top to bottom.
The first thing I noticed is how unbelievably solid it feels for such a small, light machine. The machined aluminum lid opens smoothly and firmly to an extremely nice backlit keyboard with a really nice, solid feel. The 13.3″ HD screen is extremely sharp, running at a native 1366×768 pixels. Note that some 13″ ultrabooks have a slightly higher screen resolution, but for me this resolution is perfect for this size screen, striking a good balance between screen real estate and readability.
The bottom of the machine is carbon fiber which is quite cool to the touch for the most part (particularly given that there’s an Intel i7 chip inside), save for one warm spot towards the back of the machine (presumably right where the processor is). There is a bit of faint fan noise as the machine is running but it’s very, very quiet compared to a couple other ultrabooks I’ve had so they’ve put some nice thought into cooling such a small little package without copping out and constantly running a loud fan full speed.
I feel like I should say a bit more about the keyboard since I’m typing this blog post on it — it’s incredibly nice to type on. The keys have a really great, firm action and as you depress the keys the machine body doesn’t “give” as with some ultrabooks, so it doesn’t feel flimsy in any way or like you’ll break it if you get to typing incredibly quickly (which I have a tendency to do).
Even just picking up the machine and carrying it around everything feels amazingly tight and solid. This is a far cry from the Dells I’ve had in the past, and I would think even the most ardent Mac proponent would have to admit the quality of the hardware rivals, if not exceeds, that of Apple.

Tech Specs

You can see and read more about the XPS 13 in general on Dell’s web site, but the Sputnik developer machine is the high-end model, specifically with the following specs:
  • 13.3″ HD (720p) Corning Gorilla Glass WLED screen, 1366×768 resolution
  • Intel i7 processor running at 1.7GHz (with TurboBoost up to 2.8GHz)
  • Intel HD Graphics 3000
  • 256GB SSD
  • 4GB of dual-channel DDR3 RAM (1333MHz)
  • Backlit keyboard
  • Two USB ports, 1 USB 2, 1 USB 3
  • Mini display port
  • Headphone jack
  • Marketing claim on battery life: 8 – 9 hours (I’ll test this to see what real-world battery life is)
In my estimation they did a good job of keeping things very minimalistic while focusing on what’s important. Adding things like more USB ports, multiple (and potentially large in size) display output options, and even wired ethernet would have cause the machine to lose its singular focus which is a small, light, high quality ultrabook.
What this machine doesn’t have, that you should be aware of if these things matter to you, are an optical drive and the aforementioned wired ethernet, and if you want to output to VGA or HDMI you need to pick up a dongle. None of these things matter to me personally; in fact on my other new, larger laptop I got a couple of weeks ago I opted for a second hard drive in place of an optical drive because I can’t even remember the last time I used an optical drive.
The lack of wired ethernet also doesn’t bother me because I personally never used a wired connection, and since this machine is so incredibly thin (and I mean incredibly thin) there isn’t space for an ethernet jack anyway. If you absolutely have to have wired ethernet in an office setting or somewhere without wi-fi you can always get a USB ethernet adapter.
Overall, this is an impressively equipped little powerhouse.

One Minor Potential Downside: Memory

On balance Dell made very smart decisions about what to put into the machine, with the one question mark in my mind being why they chose not to put more than 4GB of RAM in the machine. (It’s worth noting at this point that the machine is not user-upgradeable.) I can’t imagine it was for cost or size reasons, so I have to guess that it was a necessary design/engineering decision for some other reason.
Granted, for many users 4GB of RAM is plenty, and if nothing else putting Ubuntu on the machine will make it run a lot more efficiently than Windows 7. That said, for a machine targeted at developers 4GB seems a tad on the low side these days, so although I’d certainly stop well short of calling it a show-stopper for me at least, it’s something to be aware of and given your situation that may or may not work for you. (Note that Dell makes a slightly larger XPS 14 in which you can get 8GB of RAM.)

Sputnik as a Developer Machine

I’ll find out a lot more about what the Sputnik can and can’t handle next week when I take it to DjangoCon, but the i7 chip is extremely zippy, and my guess is that I’ll find 4GB to be adequate for most of my needs. Ubuntu, Chrome with my usual 12-15 tabs open, and Emacs or Sublime Text are running quite happily and responsively. (Note that I have not yet installed and tried a heavier IDE like PyCharm or Eclipse.)
If you’re a big user of VMs, given my experience on another ultrabook with 4GB of RAM I can say Windows 7 runs decently in 2GB of RAM in VirtualBox on Ubuntu. I wouldn’t want to spend a lot of time in it with only 2GB, but if you need to test something in IE or jump into Windows to run a specific application (e.g. I have to get into Windows on rare occasions to run the VMWare console app to manage our VMs since their web-based console tool is abysmal), it should be a decent experience. Not as nice as on my laptop with 16GB of RAM certainly, and you won’t be able to get away with running multiple VMs at once, but running a VM is certainly possible.
If you’re a Java or CFML developer, I fired up some Java-based applications on Tomcat on this machine, giving Tomcat 1GB and then 2GB of heap space. Things ran fine, though if like me you abuse the heck out of memory if you have it available, I suspect you’re not going to be able to get away running dozens of webapps simultaneously under Tomcat on this machine.
That said I’m moving away from the Java/CFML world into the Python/Django world which is much more memory-friendly, so for Python development this machine should be really excellent. Most of the time when I’m doing Python development I don’t see my memory or CPU monitors even act like much is going on, which is quite a different experience than I was used to with Java and CFML.
The CPU is certainly up to practically any development task, just be aware that depending on the type of development you’re doing and your development style, you might need to think a little bit about the RAM limitation.
Bottom line on Sputnik as a developer machine is I see this as being an absolutely fantastic developer machine, with a minor caveat about the RAM.

Conclusions

The Dell XPS 13 is a huge winner in my book. It’s exceedingly well built, light, quiet, and has all the bells and whistles you need in an ultrabook — particularly one aimed at developers — and Dell made intelligent omissions across the board with the possible exception of the amount of RAM pre-installed.
If like me you’ve had Dells in the past and hadn’t thought about Dell in a while, this machine may well change your mind about Dell. After only a few hours of using it it’s certainly starting to change mine, and I can already see myself gravitating to the Sputnik as my go-to machine.

Installing MySQL Python Module on Ubuntu

After tearing through several other Django books and tutorials using sqlite3 as the database, I’m starting to go through the book Beginning Django E-Commerce and it uses MySQL as the database. I use MySQL quite a lot so that side of things isn’t an issue, but I did run into a couple of wrinkles getting MySQL and Django playing nice so I thought I’d share.

Basically if after configuring your database in settings.py and running python manage.py dbshell you get a bunch of errors, you have a minor amount of installation work to do to get things rolling.

First thing I did was install pip, which is a better version of/replacement for easy_install:
sudo easy_install pip
Next I ran pip upgrade for good measure (probably not necessary but can’t hurt, and worth running if you already had pip installed):
sudo pip install pip –upgrade
On my machine (Ubuntu 12.04 64-bit) I also had to build the dependencies for the python-mysqldb libraries:
sudo apt-get build-dep python-mysqldb
And finally with that in place you can use pip to install the Python MySQL libraries:
sudo pip install MySQL-python
If everything worked you should now be able to run python manage.py dbshell from your Django project and have it load up the MySQL console.

Manually Installing the Django Plugin for Eric

If you install Eric (specifically Eric 4) from the Ubuntu software repos, the Eric plugin repository points to a location that’s unavailable:
http://die-offenbachs.homelinux.org/eric/plugins/repository.xml

I’m sure there’s a way to change it but I don’t see how to do it in the app itself (haven’t started poking around to see if there are config files somewhere yet), but luckily Eric plugins are just zip files so you can download them from a repository URL that works, and then add them to Eric.

The working plugin repository is here:
http://eric-ide.python-projects.org/plugins4/repository.xml

From there just do a ctrl-F to find the plugin you’re looking for, then copy/paste the URL for the plugin’s zip file into your browser (or use wget or whatever floats your boat) to download the plugin.

With the zip file downloaded, in Eric go to Plugins -> Install Plugins, click Add, and then point to the zip file you downloaded.

If someone knows how to change the plugin repository URL in Eric I’d love to learn.

Connecting to SQL Server with pyodbc

At long last after my previous posts on this topic we get to the easy part: using pyodbc to run queries against SQL Server.

If you need to get caught up, the posts you’ll want to go through before proceeding with this one are:

With a datasource created in unixODBC this last part is rather anti-climactic, but this is of course also where you can get some real work done in Python.
First, in a Python console (‘python’ from a terminal or whatever your favorite Python console tool is), import pyodbc:
>>> import pyodbc
Next, create a connection to the datasource using the datasource name, which in this case we’ll assume is foo:
>>> conn = pyodbc.connect(‘DSN=foo;UID=username;PWD=password’)
If you don’t get any errors, the connection was successful. Next, create a cursor on that connection:
>>> cursor = conn.cursor()
Now we can execute a query against SQL Server using that cursor:
>>> cursor.execute(“SELECT * FROM bar”)
If the query ran successfully, you’ll see that a pyodbc.Cursor object is returned:
<pyodbc.Cursor object at 0x15a5a50>
Next, and this is is not the most efficient way to do things (see below) but is good for demonstration purposes, let’s call fetachall() on the cursor and set that to a variable called rows:
>>> rows = cursor.fetchall()
This returns a list of pyodbc Row objects, which are basically tuples.
Finally, we can iterate on the Cursor and output the results of the query:
for row in rows:
    print row

This will output each row so you can see the results of your handiwork.

That’s all there is to it!

Of course there are a lot of ways to get at the individual column values returned from the query, so be sure and check out the pyodbc Getting Started wiki for details.

Note that there are numerous considerations depending on the volume and nature of data with which you’re dealing. For example if you have a large amount of data, using fetchall() isn’t advisable since that will load all the results into memory, so you’ll probably much more commonly want to use fetchone() in a loop instead since that’s much more memory efficient.

You can learn all about the objects, methods, etc. involved with pyodbc in the pyodbc wiki.

Next up is pymssql!