Archive for September, 2010

The everpresent “util” module.

Sunday, September 12th, 2010

Every major library or application I write seems to have a module named “util” these days.  I think it represents a kind of “impedance mismatch” between the platform I’m using (C runtime, C++ runtime, python standard libraries) and the platform I *wish* I were using.

Recently, I’ve been writing python code that runs lots of little UNIX utilities.  You know, like: find, ls, chmod, etc, etc.  It’s the kind of code that might also be written as a shell script, but python is much nicer when the program gets larger than about a page.  If you’re running lots of utilities, you want a variety of ways to interact with them.

Sometimes, you don’t want to send it any input, sometimes you do, sometimes you are expecting one line of output.  Sometimes you’re expecting a list of lines.  Sometimes you’re going to check the return code, sometimes you’re not.  These functions are all just small wrappers around calls to the python subprocess module in python.  But if you’re writing a lot of code that uses them, it’s important to make that code readable, so you want to streamline away most of the goop for dealing with the subprocess module.

I have utility routines for creating temporary files and removing them all when the program exits. There are routines to keep me from adding a lot of obscure import statements to the top of most of my modules.

Here’s some examples of what I’m using for now:

def gethostname():
   from socket import gethostname
   return gethostname()

def timestamp():
   import datetime
   return str(

Here’s a recipe that I got from  I wanted the equivalent of “mkdir -p”, and you need a few lines to do that in python.

def mkdir_p(dir):
  import errno
  except OSError, exc:
    if exc.errno == errno.EEXIST:

There’s also code to do things that I’m convinced must have a better answer in python, but I haven’t found it yet.  So I isolate the hack to the until module.

def is_executable(file):
  S_IEXEC = 00100
  mode = os.stat(file).st_mode
  return mode & S_IEXEC

Moving code in and out of my util module also prevents me from worrying so much about obscure modularity issues. Any code I don’t want to worry about today goes into the util module. When I know where it belongs, I can easily move it later. Of course, that’s much easier to do with python than in a language that uses header files like C or C++.