Python Decorators Overview

Decorators in Python seem complicated, but they’re very simple. You’ve probably seen them; they’re the odd bits before a function definition that begins with ‘@‘, e.g.:

def decorator(fn):
    def inner(n):
        return fn(n) + 1
    return inner
 
@decorator
def f(n):
    return n + 1

Note the function called decorator; it takes a function as an argument and defines and returns a new function that uses the one it was passed. That pattern is common to almost all decorators. The @decorator notation is simply a special syntax to call an existing function, passing the new function as an argument, and use the return value to replace the new function.

In the example above, then, decorator is called with f as an argument, and it returns a new function that replaces f. The same effect could be produced less concisely by writing this:

def decorator(fn):
    def inner(n):
        return fn(n) + 1
    return inner
 
def f(n):
    return n + 1
 
f = decorator(f)

All the @ notation does is to make the syntax more concise.

See Python Decorators At Work

Here’s a simple example whose output illustrates how a decorator works.

def wrap_with_prints(fn):
    # This will only happen when a function decorated
    # with @wrap_with_prints is defined
    print('wrap_with_prints runs only once')
 
    def wrapped():
        # This will happen each time just before
        # the decorated function is called
        print('About to run %s' % fn.__name__)
        # Here is where the wrapper calls the decorated function
        fn()
        # This will happen each time just after
        # the decorated function is called
        print('Done running %s' % fn.__name__)
 
    return wrapped
 
@wrap_with_prints
def func_to_decorate():
    print('Running the function that was decorated.')
 
func_to_decorate()

When you run the example, the output will look like this:

wrap_with_prints runs only once
About to run func_to_decorate
Running the function that was decorated.
Done running func_to_decorate

Note that the decorator (wrap_with_prints) runs only once, when the decorated function is created, but the inner function (wrapped) runs each time you run func_to_decorate.

An Almost Practical Example

Python’s functions are like any other Python object; you can assign them to variables, pass them as arguments in function calls, return them from other functions, put them in lists and dictionaries, and so forth. (That’s what it means when people say Python has first class functions.) Decorators are a concise way of taking advantage of that fact to provide useful functionality.

For example, say we have the following code (you’ll recognize it as fizzbuzz):

def fizz_buzz_or_number(i):
    ''' Return "fizz" if i is divisible by 3, "buzz" if by
        5, and "fizzbuzz" if both; otherwise, return i. '''
    if i % 15 == 0:
        return 'fizzbuzz'
    elif i % 3 == 0:
        return 'fizz'
    elif i % 5 == 0:
        return 'buzz'
    else:
        return i
 
for i in range(1, 31):
    print(fizz_buzz_or_number(i))

Then, say we wanted to log the arguments and return values of our functions for debugging. (There are better ways of debugging, and you should use those instead, but this is a useful example for us.) We could add logging statements in the function, but that would be cumbersome if we had more functions, and changing the functions being debugged is likely to introduce other errors, which will also have to be debugged. We need a generalized method that leaves the functions intact.

Enter decorators to save the day! We can write a decorator function to do our logging for us:

def log_calls(fn):
    ''' Wraps fn in a function named "inner" that writes
        the arguments and return value to logfile.log '''
    def inner(*args, **kwargs):
        # Call the function with the received arguments and
        # keyword arguments, storing the return value
        out = apply(fn, args, kwargs)
 
        # Write a line with the function name, its
        # arguments, and its return value to the log file
        with open('logfile.log', 'a') as logfile:
            logfile.write(
                '%s called with args %s and kwargs %s, returning %s\n' %
                (fn.__name__,  args, kwargs, out))
 
        # Return the return value
        return out
    return inner

Then, all we need to do is add our decorator to the definition of fizz_buzz_or_number:

@log_calls
def fizz_buzz_or_number(i):
    # Do something

And running the program will produce a log file that looks like this:

fizz_buzz_or_number called with args (1,) and kwargs {}, returning 1
fizz_buzz_or_number called with args (2,) and kwargs {}, returning 2
fizz_buzz_or_number called with args (3,) and kwargs {}, returning fizz
fizz_buzz_or_number called with args (4,) and kwargs {}, returning 4
fizz_buzz_or_number called with args (5,) and kwargs {}, returning buzz
# Do something
fizz_buzz_or_number called with args (28,) and kwargs {}, returning 28
fizz_buzz_or_number called with args (29,) and kwargs {}, returning 29
fizz_buzz_or_number called with args (30,) and kwargs {}, returning fizzbuzz

Again, there are better ways of debugging, but it’s a nice illustration of a possible use for a decorator.

More Advanced Uses

The decorators above are simple examples; there are more advanced possibilities available.

Multiple Decorators

You can chain decorators. For example, you could use decorators to wrap the text returned from a function in HTML tags — though I wouldn’t recommend it; use a template engine instead. In any case:

def b(fn):
    return lambda s: '<b>%s</b>' % fn(s)
 
def em(fn):
    return lambda s: '<em>%s</em>' % fn(s)
 
@b
@em
def greet(name):
    return 'Hello, %s!' % name

Then, calling greet('world') will return:

<b><em>Hello, world!</em></b>

Note that the decorators are applied in order, bottom to top; the function is first wrapped by em, and then the result is wrapped in b. Failing to put decorators in the correct order can cause confusing, difficult-to-trace errors.

Decorators with Arguments

Sometimes, you might want to pass arguments to the decorator function along with the new function to decorate. For example, you might want to abstract the HTML tag-wrapping decorators b and em in the previous example into a generic tag_wrap decorator that takes the tag as an extra argument, so we don’t have to have separate decorators for every HTML tag. Unfortunately, you can’t do that. However, you can do something that looks like you’re passing arguments to a decorator, and has a similar effect: you can write a function that takes arguments and returns a generator function:

def tag_wrap(tag):
    def decorator(fn):
        def inner(s):
            return '<%s>%s' % (fn(s), tag)
        return inner
    return decorator
 
@tag_wrap('b')
@tag_wrap('em')
def greet(name):
    return 'Hello, %s!' % name
 
print(greet('world'))

The result of this example is the same as that of the previous one.

Let’s recap, outside to inside, what tag_wrap does. tag_wrap is a function that accepts a tag and returns a function decorator which accepts a function. When passed a function, decorator returns a function called inner that accepts a string, passes the string to the function that was passed to decorator, and wraps the result in whatever tag was passed to tag_wrap.

That is a mind-stretching series of definitions, but consider it from the point of view of a library writer: it’s complicated for the person who has to writetag_wrap, but for the user of the library that contains it, it’s dead simple.

Decorators in the Real World

Many writers of libraries make use of decorators to provide simple ways for library users to add complex functionality to their code. For example, the web framework Django uses a decorator, login_required, to make a view require user authentication — surely a convenient way to extend a function in such a complex way.

Another Python web framework, CherryPy, goes further with its Tools, which handle authentication, static directories, error handling, caching, and much more. Every tool CherryPy provides can be configured with a config file, by calling the tool directly, or by using decorators, some of which “take arguments”, as in:

@tools.staticdir(root='/path/to/app', dir='static')
def page(self):
    # Do something

Which defines a static directory called “static”, located at “/path/to/app”.

Though these decorators are designed to simplify their use even for those who do not fully understand how they work, a complete comprehension of decorators will allow you to use them more flexibly and intelligently, and to create your own when appropriate, saving you development time and effort as well as easing maintenance.

Leave a Reply

Your email address will not be published. Required fields are marked *