tf.keras.backend.moving_average_update

TensorFlow 1 version

View source on GitHub

Compute the exponential moving average of a value.

tf.keras.backend.moving_average_update(
    x, value, momentum
)

The moving average 'x' is updated with 'value' following:

x = x * momentum + value * (1 - momentum)

For example:

x = tf.Variable(0.0)
momentum=0.9
moving_average_update(x, value = 2.0, momentum=momentum).numpy()
x.numpy()
0.2

The result will be biased towards the initial value of the variable.

If the variable was initialized to zero, you can divide by 1 - momentum ** num_updates to debias it (Section 3 of Kingma et al., 2015):

num_updates = 1.0
x_zdb = x/(1 - momentum**num_updates)
x_zdb.numpy()
2.0