Decorators, Closures, and Partial Application

Wed, 28 Jan 2015 06:50:04 GMT

I really like Python decorators. They provide a useful way to manipulate functions in a language that treats functions as first-class objects (i.e., a language that can pass around a function in the same way that you can pass around a string or an integer). They also provide a useful road into talking about some topics in computer science, namely closures and partial application!

But let’s start with the concrete, shall we?

Say that I’m working on an application that uses a database connection, and I find myself writing code that looks like this, using a global db variable:

def do_a_query(query):
    db.connect()
    result = db.query(query)
    db.close()
    return result

def execute_an_insert(obj):
    db.connect()
    db.insert(obj)
    db.close()
    return True

If you’ve got an eye for patterns, you’ll likely notice that I’ve re-used the same code in two places and will probably re-use it in more: opening and closing a database connection acts like a set of parentheses around my other code, and it seems tedious to write it again and again.

But here come decorators to the rescue! I can, instead, write:

def with_db(f):
    def _f_with_db(*args, **kwargs):
        db.connect()
        result = f(*args, **kwargs)
        db.disconnect()
        return result
    return _f_with_db

@with_db
def do_a_query(query):
    return db.query(query)

@with_db
def execute_an_insert(obj):
    db.insert(obj)
    return True

And it clears up a lot of the boilerplate. Fantastic! (Ignore for a moment those *args and **kwargs; we can come back to what they mean later. Suffice it to say, for now, that they pass along the arguments untouched.) I could even do something fancy, like extend my with_db function to re-use existing open connections:

def with_db(f):
    def _f_with_db(*args, **kwargs):
        try:
            result = _f_with_db(*args, **kwargs)
        except DBConnectionError:
            db.connect()
            result = f(*args, **kwargs)
    return _f_with_db

And the functions decorated with it shouldn’t care: it provides them with an active db connection that they can rely on, and that’s all they need to know.

Closures

So, let’s talk for a moment about closures. A closure, simply put, is a function – or a reference to a function – with some additional state attached. In Python, this generally looks like:

def make_a_closure(seed_variable):
    state = something_generates_state_with(seed_variable)
    def _closure(other_arg):
        return do_some_things_with(state, other_arg)
    return _closure

We would then call make_a_closure(some_seed) to get a closure function seeded with some_seed. For example, imagine that we wanted to make a function that, given a Unix Epoch timestamp, would return some predetermined format of date:

def make_a_strftime_closure(format_string):
    def _strftime_closure(epoch_timestamp):
        return time.strftime(format_string, time.gmtime(epoch_timestamp))
    return _strftime_closure

pretty_date = make_a_strftime_closure("%A, %B %e %Y")
print "The unix epoch began on %s" % pretty_date(0)
# The unix epoch began on Thursday, January  1 1970
print "Beyoncé was born on %s" % pretty_date(368434800)
# Beyoncé was born on Friday, September  4 1981

Suppose, though, we want a standardized format of time for our logs:

iso_date = make_a_strftime_closure("%Y-%m-%dT%H:%M:%S%z")
print "The unix epoch began on %s" % iso_date(0)
# The unix epoch began on 1970-01-01T00:00:S+0000

To tie this back to decorators: if you notice, a decorator function is just a closure with the decorated function as its bound state:

def decorate_a_function(f):
    def _decorated(*args, **kwargs):
        return f(*args, **kwargs)
    return _decorated

Partially Applied Functions

Partially applied functions sounds kind of gnarly and complicated but describes a pretty straightforward concept: applying some arguments to a function ahead of time.

Imagine, if you will, our original example with do_a_query and execute_an_insert, but this time without a global DB object:

def do_a_query_with_db(db, query):
    db.connect()
    result = db.query(query)
    db.close()
    return result

If you find yourself writing a lot of functions with a shared argument (e.g. db), you could use a partial application to supply the db argument ahead of time, using functools.partial:

do_a_query = partial(do_a_query_with_db, global_db)
do_a_query(some_query) # equivalent to do_a_query_with_db(global_db, some_query)

Python’s own documentation calls out that partial() is a pretty simple decorator:

Roughly equivalent to:

def partial(func, *args, **keywords):
    def newfunc(*fargs, **fkeywords):
        newkeywords = keywords.copy()
        newkeywords.update(fkeywords)
        return func(*(args + fargs), **newkeywords)
    newfunc.func = func
    newfunc.args = args
    newfunc.keywords = keywords
    return newfunc

So there you have it: Python’s decorators not only provide a neat way to manipulate and use functions, they also provide us a foot in the door for some pretty neat CS concepts!

Tune in next time for a choice of:

Functional Python II: Decorators and Wacky Hidden Functions
Functional Programming II: Folding and Currying

comments

joshu’s blog

Decorators, Closures, and Partial Application

Closures

Partially Applied Functions