Ming's Blog
0%

Partial Function and its Benefits

Posted on Edited on In Disqus:

The Concept

Partial function is a function partially complete. As a transformer, it is a way to create new functions from existing functions, meanwhile adding your own things. It can be named whatever you want, but I call it partial to keep consistency with the concept.

Here is a typical implementation of partial function.

1
2
3
4
5
6
7
8
def partial(method, name, **specified):
    @functools.wraps(method)
    def wrapper(self, *args, **kwargs):
        final = dict(specified)
        final.update(kwargs)
        return getattr(self, method.__name__)(*args, **final)
    wrapper.__name__ = name
    return wrapper

It seems to be pretty complex. This article will help you understand this function.


Return Value

The function partial mainly does two things: (1) defines a function and (2) returns that. The function returned is called "wrapper". Before it is returned, the function's name is changed in this line of code

1
wrapper.__name__ = name

As a result, the parameter name controls the name of returned function. If you pass "a_fancy_function" to name then the function returned will be a_funcy_function.


Decorator

The first line of function body is a decorator

1
@functools.wraps(method)

To those who are not familiar to decorator, you can either spend 15 minutes to read the famous article, or you can just believe the following words:

Decorator is a function that consumes a function and produces a new one to replace the original function.

If we have the following function

1
2
3
@decorator
def func():
    do_something()

When function func is invoked, instead of execute func(), the Python interpreter will actually calls decorator(func)().

Decorator is achieved via wrapping the original function by another function (inside the decorator). The original function is called "the wrapped function". The function inside decorator is called "wrapper".

As you can imagine, usually a decorator is defined like this.

1
2
3
4
5
6
7
8
def decorator(func):
    def wrapper():
        do_something_with_func()
    return wrapper

@decorator
def func():
    do_something()

In our case there is a parameter in the decorator. This can be translated to functools.wraps(method)(wrapper). So functools.wraps takes the parameter method and returns a function. That function serves as the decorator: modifies and replaces the function wrapper.

Before moving forward let's summarize the progress so far.

  1. There is a function called wrapper which does something we haven't known yet.
  2. The function's name is changed from "wrapper" to the argument passed to name.
  3. A decorator is constructed by passing method to a function called functools.wraps.
  4. That decorator is used to update the wrapper function.
  5. A function called partial provides an interface for all those activities.

Obviously, we need to know more about wrapper and functools.wraps to reveal the meaning of partial.


functools.wraps

We have already known how decorator works. This time we want to implement the wrapper. We need to know some information of the wrapped function. Could we achieve that with a decorator? Initially it seems to be a crazy idea, because the wrapped function will then become the wrapper of wrapper. However, functools.wraps is a special decorator. It is designed to pass the wrapped function to the wrapper.


The Function wrapper

The function body of wrapper only contains three lines of code.

1
2
3
4
def wrapper(self, *args, **kwargs):
    final = dict(specified)
    final.update(kwargs)
    return getattr(self, method.__name__)(*args, **final)

The first line is to construct a dictionary for specified, which is a group of named arguments passed to partial function.

Then the update method is called, those named arguments in kwargs are integrated into final. New key/value pairs are added and existing keys are overwritten. You will soon know that it is the main purpose of using partial.

Finally a value is returned. Here is the hard part.

You may be confused by the first argument self in wrapper. It seems that wrapper is not inside a class. So why does it contain self as the first parameter? While, I hope you still remember that wrapper is the function returned by partial. If partial is used inside a class to generate a group of methods, then definitely we need self as the first parameter for those methods to work.

Then what does getattr(self, method.__name__) mean? getattr is a built-in function to retrieve a named attribute inside an object. getattr(foo, "bar") is equivalent to foo.bar. We know self is an instance of a class, method is a function, so getattr(...) returns a function with the same name as method and invoked on self instance. Let's call this function "middle". The whole wrapper returns the execution result of middle(*args, **final). The key point is that the return of wrapper is not a function any more! This is the place that does the real work.

Put Things Together

Why we spend so much time looking at the crazy partial function? Because it is really useful! Here is an illustration.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
class OSWarehouse:
    ...
    def create_OS_img():
        do_a_lot_of_things()

    create_windows = partial(create_OS_img,
                             "create_windows", type="Windows")
    create_macos   = partial(create_OS_img,
                             "create_macos", type="Unix", pm="appstore")
    create_ubuntu  = partial(create_OS_img,
                             "create_ubuntu", type="Linux", pm="deb")
    create_rhel    = partial(create_OS_img,
                             "create_rhel", type="Linux", pm="rpm")
    create_suse    = partial(create_OS_img,
                             "create_suse", type="Linux", pm="rpm")

factory = OSRegistry()
ubuntu_img = factory.create_ubuntu(version="16.04")

When using a factory to generate instances, it is common to write many methods to produce different things. Among the required parameters, some of them are already known (such as type), others need to be provided by the clients at runtime. By using partial function, we hide a group of named parameters with default values from the interface, and generate a large amount of class methods in a very compact and efficient way. That's the reason why partial functions are commonly used in real projects.