If you use Python often, you probably came across the namedtuple function, from the collections module, for those who didn't: namedtuple is a function that builds a container type from a tuple of strings. Each of those strings became attributes of the newly created "tuple subclass".

You can then build an instance of it from a simple tuple, the whole point is to access tuple elements using explicit attribute names, rather than obscure indexes (integers).

Here is an example:

#!/usr/bin/env python3
from collections import namedtuple

FooType = namedtuple("FooType", ["foo", "bar"])
foo_instance = FooType("hello", "world")

assert hasattr(foo_instance, "foo")
assert hasattr(foo_instance, "bar")

print("%s %s !" % (foo_instance.foo, foo_instance.bar))

print(type(foo_instance))

As you might expect, this script only prints:

hello world!
<class '__main__.FooType'>

What's cool about namedtuple ?

nametuples are usefull for code clarity, but the cool stuff in the above code is the dynamic type creation !

Nowhere in the code a FooType is declared, so this type must has been dynamically created, by the namedtuple function. But how ?

Let's dive into the collection module source code

Because Python is awesome, its whole souce code is available here, after a bit of grep-ing, you can easily spot the namedtuple implementation (Python-3.4.4/Lib/collections/init.py).

How does "namedtuple" work ?

Here is the "core" code of namedtuple:

 _class_template = """\
from builtins import property as _property, tuple as _tuple
from operator import itemgetter as _itemgetter
from collections import OrderedDict

class {typename}(tuple):
  '{typename}({arg_list})'

  __slots__ = ()

  _fields = {field_names!r}

  def __new__(_cls, {arg_list}):
     'Create new instance of {typename}({arg_list})'
     return _tuple.__new__(_cls, ({arg_list}))

  # [ Skipping some function templates ]

  {field_defs}
"""

followed by :

def namedtuple(typename, field_names, verbose=False, rename=False):
    """Returns a new subclass of tuple with named fields.
       [ Skipping some great docstring ]
    """
    # [ skipping some argument processing ]

    # Fill-in the class template
    class_definition = _class_template.format(
        typename = typename,
        field_names = tuple(field_names),
        # [ Skipping lot of argurment ]
    )


    namespace = dict(__name__='namedtuple_%s' % typename)
    exec(class_definition, namespace)
    result = namespace[typename]
    result._source = class_definition
    
    # [ ... ]

    return result

So what does it do ? Basically, the namedtuple function fills a python code template, and then execute it, thanks to the exec() function.

This templates holds a class definition, which defines our beloved container. I had no idea a dynamic type could be done so easilly !


Conclusion

In Python, Types can be created at running time, the combination of a template + the exec() function is one way to do that.

I was always told to avoid the use of exec() , (or eval() in other language), so I did not suspect widely used modules to use it, I still have a lot to learn ! :)

  • Sep. 2 2016, 09:56 am