tensorflow 中 name_scope 及 variable_scope 的异同

Let's begin by a short introduction to variable sharing. It is a mechanism in TensorFlow that allows for sharing variables accessed in different parts of the code without passing references to the variable around. The method tf.get_variable can be used with the name of the variable as argument to either create a new variable with such name or retrieve the one that was created before. This is different from using the tf.Variable constructor which will create a new variable every time it is called (and potentially add a suffix to the variable name if a variable with such name already exists). It is for the purpose of the variable sharing mechanism that a separate type of scope (variable scope) was introduced.

As a result, we end up having two different types of scopes:

• name scope, created using tf.name_scope or tf.op_scope
• variable scope, created using tf.variable_scope or tf.variable_op_scope

Both scopes have the same effect on all operations as well as variables created using tf.Variable, i.e. the scope will be added as a prefix to the operation or variable name.

However, name scope is ignored by tf.get_variable. We can see that in the following example:

with tf.name_scope("my_scope"):
    v1 = tf.get_variable("var1", [1], dtype=tf.float32)
    v2 = tf.Variable(1, name="var2", dtype=tf.float32)
    a = tf.add(v1, v2)

print(v1.name)  # var1:0
print(v2.name)  # my_scope/var2:0
print(a.name)   # my_scope/Add:0

The only way to place a variable accessed using tf.get_variable in a scope is to use variable scope, as in the following example:

with tf.variable_scope("my_scope"):
    v1 = tf.get_variable("var1", [1], dtype=tf.float32)
    v2 = tf.Variable(1, name="var2", dtype=tf.float32)
    a = tf.add(v1, v2)

print(v1.name)  # my_scope/var1:0
print(v2.name)  # my_scope/var2:0
print(a.name)   # my_scope/Add:0

Finally, let's look at the difference between the different methods for creating scopes. We can group them in two categories:

• tf.name_scope(name) (for name scope) and tf.variable_scope(name_or_scope, ...)(for variable scope) create a scope with the name specified as argument
• tf.op_scope(values, name, default_name=None) (for name scope) and tf.variable_op_scope(values, name_or_scope, default_name=None, ...) (for variable scope) create a scope, just like the functions above, but besides the scope name, they accept an argument default_name which is used instead of name when it is set to None. Moreover, they accept a list of tensors (values) in order to check if all the tensors are from the same, default graph. This is useful when creating new operations, for example, see the implementation of tf.histogram_summary.

大意是说 name_scope及variable_scope的作用都是为了不传引用而访问跨代码区域变量的一种方式，其内部功能是在其代码块内显式创建的变量都会带上scope前缀（如上面例子中的a），这一点它们几乎一样。而它们的差别是，在其作用域中获取变量，它们对 tf.get_variable() 函数的作用是一个会自动添加前缀，一个不会添加前缀。