see this

and rewrite your code to

import tensorflow as tf

b = None


@tf.function
def f():
    a = tf.constant([[10, 10], [11., 1.]])
    x = tf.constant([[1., 0.], [0., 1.]])
    global b
    if b is None:
        b = tf.Variable(12.)
    y = tf.matmul(a, x) + b
    return y

print(f())

or

class F(object):
    def __init__(self):
        self.b = None

    @tf.function
    def __call__(self):
        a = tf.constant([[10, 10], [11., 1.]])
        x = tf.constant([[1., 0.], [0., 1.]])
        if self.b is None:
            self.b = tf.Variable(12.)
        y = tf.matmul(a, x) + self.b
        return y


f = F()
print(f())

HI @zakizhou ! I'm aware that defining a function that declares a variable inside creates a status and it has to be handled differently.

As you suggested an alternative I to wrap the function in a class and make the variable a private attribute, or declaring the variable as global and check if it is None.

However, as stated in the RFC:

State (like tf.Variable objects) are only created the first time the function f is called.

Thus, when I invoke f() for the first time, I should have the graph definition with a tf.Variable object created and executed once. And it should work according to what is written in the RFC.

Instead, what happens is that even though I call the function only once, and thus this is the first call, it seems like tf.function is evaluating the function twice (like calling f(), f()), making what is stated in the RFC false.

tf.function may evaluate your python function more than once.

What the RFC states instead is that you are allowed to create variables as long as variable creation only happens the first time your python function is evaluated.

And the reason for this is that if you write python code which unconditionally creates variables then I can't tell whether you mean to create a new variable every time the python function is called (eager behavior) or reuse the existing variables (what happens in graph tf 1.x) so an error felt safer.

Thank you for the explanation @alextp - now everything is clearer. The RFC just describes the first call, but there is no guarantee that tf.function won't call the function more than once while converting it as a graph and for this reason, the developer should take care of handling the variables creation; thus If I need to reuse or not a variable it makes no difference, I have to take care of its status manually.

I have an exact issue like this, but with using different optimizers:

@tf.function
def apply_gradients_once(optimizer, grads, vars):
    optimizer.apply_gradients(zip(
        grads, vars))


def apply_gradients(self, use_fast, grads_per_model):
    for i in range(self.nmodels):
        grads = grads_per_model[i]
        vars = self.model[i].get_trainable_variables()

        if use_fast[i]:
            optimizer = self.fast_optimizer
            apply_gradients_once(optimizer, grads, vars)
        else:
            optimizer = self.slow_optimizer
            apply_gradients_once(optimizer, grads, vars)

On a first epoch, I have use_fast = [True, False], and then on a second one I have
use_fast = [False, True].

With this, I have a huge error trace on the apply_gradients_once call, ending with ValueError: tf.function-decorated function tried to create variables on non-first call..

From my understanding, since the function is always called with different arguments, and there are no external dependencies, this error should not be happening.

@ericpts this is an interesting use case I hadn't thought of.

As a workaround I recommend you use two instances of tf.function here.

Let's open another issue to discuss this?

Hello, Is there a workaround for a situation like this. Here I need to create several w, b variables using add_layer method. But I get the variable reused error.

def add_layer(input, c_kdimension, c_kstrides):
w = tf.get_variable(name="w", shape=c_kdimension, initializer = tf.contrib.layers.xavier_initializer()
b = tf.get_variable(name="b", shape=c_kdimension[-1], initializer=tf.contrib.layers.xavier_initializer())
return tf.nn.conv2d(input, w, strides=c_kstrides, padding="SAME")

@tf.function
def inference(input):
model = Model()
layer_1 = add_layer(input, c_kdimension=[3, 3, 1, 32], c_kstrides=[1, 1, 1, 1])
layer_2 = add_layer(layer_1, c_kdimension=[3, 3, 32, 32], c_kstrides=[1, 1, 1, 1])
return layer2

Hello @galeone ,
As you are using TensorFlow 2.0 you must add
tf.config.experimental_run_functions_eagerly(True)
after import and then run the code.

Hello @galeone ,
As you are using TensorFlow 2.0 you must add
tf.config.experimental_run_functions_eagerly(True)
after import and then run the code.

Sorry, this is wrong, do not use it. This disables the graph construction and runs everything eagerly, like numpy. This can be wanted for debugging or in certain cases, but it is surely not the workaround.

@praveen-14 read again above about creating them as class variables or in the global scope.