python 元类——metaclass
from stack overflow:http://stackoverflow.com/questions/100003/what-is-a-metaclass-in-python
Classes as objects
Before understanding metaclasses, you need to master classes in Python. And Python has a very peculiar idea of what classes are, borrowed from the Smalltalk language.
In most languages, classes are just pieces of code that describe how to produce an object. That's kinda true in Python too:
>>> class ObjectCreator(object):
... pass
... >>> my_object = ObjectCreator()
>>> print(my_object)
<__main__.ObjectCreator object at 0x8974f2c>
But classes are more than that in Python. Classes are objects too.
Yes, objects.
As soon as you use the keyword class
, Python executes it and creates an OBJECT. The instruction
>>> class ObjectCreator(object):
... pass
...
creates in memory an object with the name "ObjectCreator".
This object (the class) is itself capable of creating objects (the instances), and this is why it's a class.
But still, it's an object, and therefore:
- you can assign it to a variable
- you can copy it
- you can add attributes to it
- you can pass it as a function parameter
e.g.:
>>> print(ObjectCreator) # you can print a class because it's an object
<class '__main__.ObjectCreator'>
>>> def echo(o):
... print(o)
...
>>> echo(ObjectCreator) # you can pass a class as a parameter
<class '__main__.ObjectCreator'>
>>> print(hasattr(ObjectCreator, 'new_attribute'))
False
>>> ObjectCreator.new_attribute = 'foo' # you can add attributes to a class
>>> print(hasattr(ObjectCreator, 'new_attribute'))
True
>>> print(ObjectCreator.new_attribute)
foo
>>> ObjectCreatorMirror = ObjectCreator # you can assign a class to a variable
>>> print(ObjectCreatorMirror.new_attribute)
foo
>>> print(ObjectCreatorMirror())
<__main__.ObjectCreator object at 0x8997b4c>
Creating classes dynamically
Since classes are objects, you can create them on the fly, like any object.
First, you can create a class in a function using class
:
>>> def choose_class(name):
... if name == 'foo':
... class Foo(object):
... pass
... return Foo # return the class, not an instance
... else:
... class Bar(object):
... pass
... return Bar
...
>>> MyClass = choose_class('foo')
>>> print(MyClass) # the function returns a class, not an instance
<class '__main__.Foo'>
>>> print(MyClass()) # you can create an object from this class
<__main__.Foo object at 0x89c6d4c>
But it's not so dynamic, since you still have to write the whole class yourself.
Since classes are objects, they must be generated by something.
When you use the class
keyword, Python creates this object automatically. But as with most things in Python, it gives you a way to do it manually.
Remember the function type
? The good old function that lets you know what type an object is:
>>> print(type(1))
<type 'int'>
>>> print(type(""))
<type 'str'>
>>> print(type(ObjectCreator))
<type 'type'>
>>> print(type(ObjectCreator()))
<class '__main__.ObjectCreator'>
Well, type
has a completely different ability, it can also create classes on the fly. type
can take the description of a class as parameters, and return a class.
(I know, it's silly that the same function can have two completely different uses according to the parameters you pass to it. It's an issue due to backwards compatibility in Python)
type
works this way:
type(name of the class,
tuple of the parent class (for inheritance, can be empty),
dictionary containing attributes names and values)
e.g.:
>>> class MyShinyClass(object):
... pass
can be created manually this way:
>>> MyShinyClass = type('MyShinyClass', (), {}) # returns a class object
>>> print(MyShinyClass)
<class '__main__.MyShinyClass'>
>>> print(MyShinyClass()) # create an instance with the class
<__main__.MyShinyClass object at 0x8997cec>
You'll notice that we use "MyShinyClass" as the name of the class and as the variable to hold the class reference.
They can be different, but there is no reason to complicate things.
type
accepts a dictionary to define the attributes of the class. So:
>>> class Foo(object):
... bar = True
Can be translated to:
>>> Foo = type('Foo', (), {'bar':True})
And used as a normal class:
>>> print(Foo)
<class '__main__.Foo'>
>>> print(Foo.bar)
True
>>> f = Foo()
>>> print(f)
<__main__.Foo object at 0x8a9b84c>
>>> print(f.bar)
True
And of course, you can inherit from it, so:
>>> class FooChild(Foo):
... pass
would be:
>>> FooChild = type('FooChild', (Foo,), {})
>>> print(FooChild)
<class '__main__.FooChild'>
>>> print(FooChild.bar) # bar is inherited from Foo
True
Eventually you'll want to add methods to your class. Just define a function with the proper signature and assign it as an attribute.
>>> def echo_bar(self):
... print(self.bar)
...
>>> FooChild = type('FooChild', (Foo,), {'echo_bar': echo_bar})
>>> hasattr(Foo, 'echo_bar')
False
>>> hasattr(FooChild, 'echo_bar')
True
>>> my_foo = FooChild()
>>> my_foo.echo_bar()
True
And you can add even more methods after you dynamically create the class, just like adding methods to a normally created class object.
>>> def echo_bar_more(self):
... print('yet another method')
...
>>> FooChild.echo_bar_more = echo_bar_more
>>> hasattr(FooChild, 'echo_bar_more')
True
You see where we are going: in Python, classes are objects, and you can create a class on the fly, dynamically.
This is what Python does when you use the keyword class
, and it does so by using a metaclass.
What are metaclasses (finally)
Metaclasses are the 'stuff' that creates classes.
You define classes in order to create objects, right?
But we learned that Python classes are objects.
Well, metaclasses are what create these objects. They are the classes' classes, you can picture them this way:
MyClass = MetaClass()
MyObject = MyClass()
You've seen that type
lets you do something like this:
MyClass = type('MyClass', (), {})
It's because the function type
is in fact a metaclass. type
is the metaclass Python uses to create all classes behind the scenes.
Now you wonder why the heck is it written in lowercase, and not Type
?
Well, I guess it's a matter of consistency with str
, the class that creates strings objects, and int
the class that creates integer objects. type
is just the class that creates class objects.
You see that by checking the __class__
attribute.
Everything, and I mean everything, is an object in Python. That includes ints, strings, functions and classes. All of them are objects. And all of them have been created from a class:
>>> age = 35
>>> age.__class__
<type 'int'>
>>> name = 'bob'
>>> name.__class__
<type 'str'>
>>> def foo(): pass
>>> foo.__class__
<type 'function'>
>>> class Bar(object): pass
>>> b = Bar()
>>> b.__class__
<class '__main__.Bar'>
Now, what is the __class__
of any __class__
?
>>> age.__class__.__class__
<type 'type'>
>>> name.__class__.__class__
<type 'type'>
>>> foo.__class__.__class__
<type 'type'>
>>> b.__class__.__class__
<type 'type'>
So, a metaclass is just the stuff that creates class objects.
You can call it a 'class factory' if you wish.
type
is the built-in metaclass Python uses, but of course, you can create your own metaclass.
The __metaclass__
attribute
You can add a __metaclass__
attribute when you write a class:
class Foo(object):
__metaclass__ = something...
[...]
If you do so, Python will use the metaclass to create the class Foo
.
Careful, it's tricky.
You write class Foo(object)
first, but the class object Foo
is not created in memory yet. Python will look for __metaclass__
in the class definition. If it finds it, it will use it to create the object class Foo
. If it doesn't, it will use type
to create the class.
Read that several times.
When you do:
class Foo(Bar):
pass
Python does the following:
Is there a __metaclass__
attribute in Foo
?
If yes, create in memory a class object (I said a class object, stay with me here), with the name Foo
by using what is in __metaclass__
.
If Python can't find __metaclass__
, it will look for a __metaclass__
at the MODULE level, and try to do the same (but only for classes that don't inherit anything, basically old-style classes).
Then if it can't find any __metaclass__
at all, it will use the Bar
's (the first parent) own metaclass (which might be the default type
) to create the class object.
Be careful here that the __metaclass__
attribute will not be inherited, the metaclass of the parent (Bar.__class__
) will be. If Bar
used a __metaclass__
attribute that created Bar
with type()
(and not type.__new__()
), the subclasses will not inherit that behavior.
Now the big question is, what can you put in __metaclass__
? The answer is: something that can create a class.
And what can create a class? type
, or anything that subclasses or uses it.
Custom metaclasses
The main purpose of a metaclass is to change the class automatically, when it's created.
You usually do this for APIs, where you want to create classes matching the current context.
Imagine a stupid example, where you decide that all classes in your module should have their attributes written in uppercase. There are several ways to do this, but one way is to set __metaclass__
at the module level.
This way, all classes of this module will be created using this metaclass, and we just have to tell the metaclass to turn all attributes to uppercase.
Luckily, __metaclass__
can actually be any callable, it doesn't need to be a formal class (I know, something with 'class' in its name doesn't need to be a class, go figure... but it's helpful).
So we will start with a simple example, by using a function.
# the metaclass will automatically get passed the same argument
# that you usually pass to `type`
def upper_attr(future_class_name, future_class_parents, future_class_attr):
"""
Return a class object, with the list of its attribute turned
into uppercase.
""" # pick up any attribute that doesn't start with '__' and uppercase it
uppercase_attr = {}
for name, val in future_class_attr.items():
if not name.startswith('__'):
uppercase_attr[name.upper()] = val
else:
uppercase_attr[name] = val # let `type` do the class creation
return type(future_class_name, future_class_parents, uppercase_attr) __metaclass__ = upper_attr # this will affect all classes in the module class Foo(): # global __metaclass__ won't work with "object" though
# but we can define __metaclass__ here instead to affect only this class
# and this will work with "object" children
bar = 'bip' print(hasattr(Foo, 'bar'))
# Out: False
print(hasattr(Foo, 'BAR'))
# Out: True f = Foo()
print(f.BAR)
# Out: 'bip'
Now, let's do exactly the same, but using a real class for a metaclass:
# remember that `type` is actually a class like `str` and `int`
# so you can inherit from it
class UpperAttrMetaclass(type):
# __new__ is the method called before __init__
# it's the method that creates the object and returns it
# while __init__ just initializes the object passed as parameter
# you rarely use __new__, except when you want to control how the object
# is created.
# here the created object is the class, and we want to customize it
# so we override __new__
# you can do some stuff in __init__ too if you wish
# some advanced use involves overriding __call__ as well, but we won't
# see this
def __new__(upperattr_metaclass, future_class_name,
future_class_parents, future_class_attr): uppercase_attr = {}
for name, val in future_class_attr.items():
if not name.startswith('__'):
uppercase_attr[name.upper()] = val
else:
uppercase_attr[name] = val return type(future_class_name, future_class_parents, uppercase_attr)
But this is not really OOP. We call type
directly and we don't override or call the parent __new__
. Let's do it:
class UpperAttrMetaclass(type): def __new__(upperattr_metaclass, future_class_name,
future_class_parents, future_class_attr): uppercase_attr = {}
for name, val in future_class_attr.items():
if not name.startswith('__'):
uppercase_attr[name.upper()] = val
else:
uppercase_attr[name] = val # reuse the type.__new__ method
# this is basic OOP, nothing magic in there
return type.__new__(upperattr_metaclass, future_class_name,
future_class_parents, uppercase_attr)
You may have noticed the extra argument upperattr_metaclass
. There is nothing special about it: __new__
always receives the class it's defined in, as first parameter. Just like you have self
for ordinary methods which receive the instance as first parameter, or the defining class for class methods.
Of course, the names I used here are long for the sake of clarity, but like for self
, all the arguments have conventional names. So a real production metaclass would look like this:
class UpperAttrMetaclass(type): def __new__(cls, clsname, bases, dct): uppercase_attr = {}
for name, val in dct.items():
if not name.startswith('__'):
uppercase_attr[name.upper()] = val
else:
uppercase_attr[name] = val return type.__new__(cls, clsname, bases, uppercase_attr)
We can make it even cleaner by using super
, which will ease inheritance (because yes, you can have metaclasses, inheriting from metaclasses, inheriting from type):
class UpperAttrMetaclass(type): def __new__(cls, clsname, bases, dct): uppercase_attr = {}
for name, val in dct.items():
if not name.startswith('__'):
uppercase_attr[name.upper()] = val
else:
uppercase_attr[name] = val return super(UpperAttrMetaclass, cls).__new__(cls, clsname, bases, uppercase_attr)
That's it. There is really nothing more about metaclasses.
The reason behind the complexity of the code using metaclasses is not because of metaclasses, it's because you usually use metaclasses to do twisted stuff relying on introspection, manipulating inheritance, vars such as __dict__
, etc.
Indeed, metaclasses are especially useful to do black magic, and therefore complicated stuff. But by themselves, they are simple:
- intercept a class creation
- modify the class
- return the modified class
Why would you use metaclasses classes instead of functions?
Since __metaclass__
can accept any callable, why would you use a class since it's obviously more complicated?
There are several reasons to do so:
- The intention is clear. When you read
UpperAttrMetaclass(type)
, you know what's going to follow - You can use OOP. Metaclass can inherit from metaclass, override parent methods. Metaclasses can even use metaclasses.
- You can structure your code better. You never use metaclasses for something as trivial as the above example. It's usually for something complicated. Having the ability to make several methods and group them in one class is very useful to make the code easier to read.
- You can hook on
__new__
,__init__
and__call__
. Which will allow you to do different stuff. Even if usually you can do it all in__new__
, some people are just more comfortable using__init__
. - These are called metaclasses, damn it! It must mean something!
Why would you use metaclasses?
Now the big question. Why would you use some obscure error prone feature?
Well, usually you don't:
Metaclasses are deeper magic that 99% of users should never worry about. If you wonder whether you need them, you don't (the people who actually need them know with certainty that they need them, and don't need an explanation about why).
Python Guru Tim Peters
The main use case for a metaclass is creating an API. A typical example of this is the Django ORM.
It allows you to define something like this:
class Person(models.Model):
name = models.CharField(max_length=30)
age = models.IntegerField()
But if you do this:
guy = Person(name='bob', age='')
print(guy.age)
It won't return an IntegerField
object. It will return an int
, and can even take it directly from the database.
This is possible because models.Model
defines __metaclass__
and it uses some magic that will turn the Person
you just defined with simple statements into a complex hook to a database field.
Django makes something complex look simple by exposing a simple API and using metaclasses, recreating code from this API to do the real job behind the scenes.
The last word
First, you know that classes are objects that can create instances.
Well in fact, classes are themselves instances. Of metaclasses.
>>> class Foo(object): pass
>>> id(Foo)
142630324
Everything is an object in Python, and they are all either instances of classes or instances of metaclasses.
Except for type
.
type
is actually its own metaclass. This is not something you could reproduce in pure Python, and is done by cheating a little bit at the implementation level.
Secondly, metaclasses are complicated. You may not want to use them for very simple class alterations. You can change classes by using two different techniques:
- monkey patching(猴子补丁)
- class decorators(类装饰器)
99% of the time you need class alteration, you are better off using these.
But 99% of the time, you don't need class alteration at all.
python 元类——metaclass的更多相关文章
- python 元类metaclass
文章转自:http://www.cnblogs.com/linhaifeng/articles/8029564.html 一 知识储备 exec:三个参数 参数一:字符串形式的命令 参数二:全局作用域 ...
- Python元类(metaclass)以及元类实现单例模式
这里将一篇写的非常好的文章基本照搬过来吧,这是一篇在Stack overflow上很热的帖子,我看http://blog.jobbole.com/21351/这篇博客对其进行了翻译. 一.理解类也是对 ...
- python 元类 MetaClass
type() 动态语言和静态语言最大的不同,就是函数和类的定义,不是编译时定义的,而是运行时动态创建的. 比方说我们要定义一个Hello的class,就写一个hello.py模块: class Hel ...
- 深刻理解Python中的元类metaclass(转)
本文由 伯乐在线 - bigship 翻译 英文出处:stackoverflow 译文:http://blog.jobbole.com/21351/ 译注:这是一篇在Stack overflow上很热 ...
- 深刻理解Python中的元类(metaclass)
译注:这是一篇在Stack overflow上很热的帖子.提问者自称已经掌握了有关Python OOP编程中的各种概念,但始终觉得元类(metaclass)难以理解.他知道这肯定和自省有关,但仍然觉得 ...
- Python中的元类(metaclass)
推荐+收藏:深刻理解Python中的元类(metaclass) 做一些笔记学习学习: 在大多数编程语言中,类就是用来描述如何生成一个对象的代码段,在Python中类也是一个对象,这个(类)对象自身拥有 ...
- [转] 深刻理解Python中的元类(metaclass)
非常详细的一篇深入讲解Python中metaclass的文章,感谢伯乐在线-bigship翻译及作者,转载收藏. 本文由 伯乐在线 - bigship 翻译.未经许可,禁止转载!英文出处:stacko ...
- [转]深刻理解Python中的元类(metaclass)以及元类实现单例模式
使用元类 深刻理解Python中的元类(metaclass)以及元类实现单例模式 在看一些框架源代码的过程中碰到很多元类的实例,看起来很吃力很晦涩:在看python cookbook中关于元类创建单例 ...
- 深刻理解Python中的元类(metaclass)【转】
译注:这是一篇在Stack overflow上很热的帖子.提问者自称已经掌握了有关Python OOP编程中的各种概念,但始终觉得元类(metaclass)难以理解.他知道这肯定和自省有关,但仍然觉得 ...
随机推荐
- How to include JavaScript file in JSF
In JSF 2.0, you can use <h:outputScript /> tag to render a HTML "script" element, an ...
- Java transient、Volatile关键字
Volatile修饰的成员变量在每次被线程访问时,都强迫从主内存中重读该成员变量的值.而且,当成员变量发生变化时,强迫线程将变化值回写到主内存.这样在任何时刻,两个不同的线程总是看到某个成员变量的同一 ...
- Webstorm2016激活码[ 转]
转至:http://blog.csdn.net/tingwode2014_/article/details/51063657 43B4A73YYJ-eyJsaWNlbnNlSWQiOiI0M0I0QT ...
- 多线程和Boost::Asio
线程安全 一般的,高并发使用不同的对象是安全的,在高并发中使用单一的对象是不安全的,io_service类型提供了单对象高并发的强安全保证. 线程池 多线程可能调用io_service::run()来 ...
- GCD求最大公约数
求最大公约数哪个强,果断GCD,非递归版本和递归版本如下: #include<iostream> using namespace std; int gcd(int a, int b){ / ...
- Unix: How to Install BerkeleyDB From Source
http://www.masaokitamura.com/2010/07/23/unix-how-to-install-berkeleydb-from-source/ This documentati ...
- UVa Online Judge 工具網站
UVa Online Judge 工具網站 UVa中译题uHuntAlgorithmist Lucky貓的ACM園地,Lucky貓的 ACM 中譯題目 Mirror UVa Online Judg ...
- PostgreSQL的 initdb 源代码分析之三
继续 其实接前面,整个while循环是这样的: ) { switch (c) { ...... } ...... } 这一句,c = getopt_long(argc, argv, "dD: ...
- [ALGO-3] K好数
算法训练 K好数 时间限制:1.0s 内存限制:256.0MB 问题描写叙述 假设一个自然数N的K进制表示中随意的相邻的两位都不是相邻的数字,那么我们就说这个数是K好数.求L位K进制数中K好数 ...
- redis list 使用
参考:http://redis.cn/commands.html#list BLPOP key [key ...] timeout删除,并获得该列表中的第一元素,或阻塞,直到有一个可用 BRPOP k ...