The everpresent “util” module.

Every major library or application I write seems to have a module named “util” these days.  I think it represents a kind of “impedance mismatch” between the platform I’m using (C runtime, C++ runtime, python standard libraries) and the platform I *wish* I were using.

Recently, I’ve been writing python code that runs lots of little UNIX utilities.  You know, like: find, ls, chmod, etc, etc.  It’s the kind of code that might also be written as a shell script, but python is much nicer when the program gets larger than about a page.  If you’re running lots of utilities, you want a variety of ways to interact with them.

Sometimes, you don’t want to send it any input, sometimes you do, sometimes you are expecting one line of output.  Sometimes you’re expecting a list of lines.  Sometimes you’re going to check the return code, sometimes you’re not.  These functions are all just small wrappers around calls to the python subprocess module in python.  But if you’re writing a lot of code that uses them, it’s important to make that code readable, so you want to streamline away most of the goop for dealing with the subprocess module.

I have utility routines for creating temporary files and removing them all when the program exits. There are routines to keep me from adding a lot of obscure import statements to the top of most of my modules.

Here’s some examples of what I’m using for now:

def gethostname():
   from socket import gethostname
   return gethostname()

def timestamp():
   import datetime
   return str(

Here’s a recipe that I got from  I wanted the equivalent of “mkdir -p”, and you need a few lines to do that in python.

def mkdir_p(dir):
  import errno
  except OSError, exc:
    if exc.errno == errno.EEXIST:

There’s also code to do things that I’m convinced must have a better answer in python, but I haven’t found it yet.  So I isolate the hack to the until module.

def is_executable(file):
  S_IEXEC = 00100
  mode = os.stat(file).st_mode
  return mode & S_IEXEC

Moving code in and out of my util module also prevents me from worrying so much about obscure modularity issues. Any code I don’t want to worry about today goes into the util module. When I know where it belongs, I can easily move it later. Of course, that’s much easier to do with python than in a language that uses header files like C or C++.

4 Responses to “The everpresent “util” module.”

  1. Micki St James says:

    Wow, a post today Sept 12 and previous post June 23.
    It’s like, wow, there is still a pulse over there in Oracle Solaris Studio
    land but it’s very weak. Maybe you have customers who only ever
    check in with you once a year? Maybe I’m behind a Great Firewall where
    none of the good stuff gets through? Really, I’m not trying to be
    sarcastic, what’s the deal here? Why does this blog exist?
    My working theory is that this post was a secret message intended for spy Anna Chapman that needs to be obscured from onlookers but not lost in too
    high a noise::signal ratio. Am I close?

  2. Chris Quenelle says:

    This reply reminds me, I should write a blog entry about trolls. My twitter feed gets updated more often than this blog.

  3. Micki St James says:

    You missed the point of my well-intentioned but obscure chiding. This seems to be an Oracle Solaris Studio blog but if I were Chief Blog Officer I would encourage you to shut it down completely, and protect it from further indexing by Google. The level of interactivity is just not high enough to benefit your customers. This is not a comment on the quality of your blog entries.

    Sorry it took so long to get back to you.

  4. Chris Quenelle says:

    My intention is to make each of these blog posts useful as a reference, even if the site doesn’t get updated very often. I’m certainly willing to retire any posts which are no longer possibly useful in any way. Feel free to point those out, if you run across some. This isn’t really a Solaris Studio blog. It has tech bits of all kinds. Also, since it’s on my own domain, it should be clear I have editorial discretion over the content. Hopefully that’s enough that it doesn’t make a bad impression on Solaris Studio users.