tf_agents.agents.ddpg.critic_network.CriticNetwork

Creates a critic network.

Inherits From: Network

tf_agents.agents.ddpg.critic_network.CriticNetwork(
    input_tensor_spec,
    observation_conv_layer_params=None,
    observation_fc_layer_params=None,
    observation_dropout_layer_params=None,
    action_fc_layer_params=None,
    action_dropout_layer_params=None,
    joint_fc_layer_params=None,
    joint_dropout_layer_params=None,
    activation_fn=tf.nn.relu,
    output_activation_fn=None,
    kernel_initializer=None,
    last_kernel_initializer=None,
    last_layer=None,
    name='CriticNetwork'
)

Used in the notebooks

Used in the tutorials

input_tensor_spec A tuple of (observation, action) each a nest of tensor_spec.TensorSpec representing the inputs.
observation_conv_layer_params Optional list of convolution layer parameters for observations, where each item is a length-three tuple indicating (num_units, kernel_size, stride).
observation_fc_layer_params Optional list of fully connected parameters for observations, where each item is the number of units in the layer.
observation_dropout_layer_params Optional list of dropout layer parameters, each item is the fraction of input units to drop or a dictionary of parameters according to the keras.Dropout documentation. The additional parameter permanent, if set to True, allows to apply dropout at inference for approximated Bayesian inference. The dropout layers are interleaved with the fully connected layers; there is a dropout layer after each fully connected layer, except if the entry in the list is None. This list must have the same length of observation_fc_layer_params, or be None.
action_fc_layer_params Optional list of fully connected parameters for actions, where each item is the number of units in the layer.
action_dropout_layer_params Optional list of dropout layer parameters, each item is the fraction of input units to drop or a dictionary of parameters according to the keras.Dropout documentation. The additional parameter permanent, if set to True, allows to apply dropout at inference for approximated Bayesian inference. The dropout layers are interleaved with the fully connected layers; there is a dropout layer after each fully connected layer, except if the entry in the list is None. This list must have the same length of action_fc_layer_params, or be None.
joint_fc_layer_params Optional list of fully connected parameters after merging observations and actions, where each item is the number of units in the layer.
joint_dropout_layer_params Optional list of dropout layer parameters, each item is the fraction of input units to drop or a dictionary of parameters according to the keras.Dropout documentation. The additional parameter permanent, if set to True, allows to apply dropout at inference for approximated Bayesian inference. The dropout layers are interleaved with the fully connected layers; there is a dropout layer after each fully connected layer, except if the entry in the list is None. This list must have the same length of joint_fc_layer_params, or be None.
activation_fn Activation function, e.g. tf.nn.relu, slim.leaky_relu, ...
output_activation_fn Activation function for the last layer. This can be used to restrict the range of the output. For example, one can pass tf.keras.activations.sigmoid here to restrict the output to be bounded between 0 and 1.
kernel_initializer kernel initializer for all layers except for the value regression layer. If None, a VarianceScaling initializer will be used.
last_kernel_initializer kernel initializer for the value regression layer. If None, a RandomUniform initializer will be used.
last_layer An optional custom last layer.
name A string representing name of the network.

ValueError If observation_spec or action_spec contains more than one observation.

input_tensor_spec Returns the spec of the input to the network of type InputSpec.
layers Get the list of all (nested) sub-layers used in this Network.
state_spec

Methods

copy

View source

copy(
    **kwargs
)

Create a shallow copy of this network.

Args
**kwargs Args to override when recreating this network. Commonly overridden args include 'name'.

Returns
A shallow copy of this network.

create_variables

View source

create_variables(
    input_tensor_spec=None, **kwargs
)

Force creation of the network's variables.

Return output specs.

Args
input_tensor_spec (Optional). Override or provide an input tensor spec when creating variables.
**kwargs Other arguments to network.call(), e.g. training=True.

Returns
Output specs - a nested spec calculated from the outputs (excluding any batch dimensions). If any of the output elements is a tfp Distribution, the associated spec entry returned is a DistributionSpec.

Raises
ValueError If no input_tensor_spec is provided, and the network did not provide one during construction.

get_initial_state

View source

get_initial_state(
    batch_size=None
)

Returns an initial state usable by the network.

Args
batch_size Tensor or constant: size of the batch dimension. Can be None in which case not dimensions gets added.

Returns
A nested object of type self.state_spec containing properly initialized Tensors.

get_layer

View source

get_layer(
    name=None, index=None
)

Retrieves a layer based on either its name (unique) or index.

If name and index are both provided, index will take precedence. Indices are based on order of horizontal graph traversal (bottom-up).

Args
name String, name of layer.
index Integer, index of layer.

Returns
A layer instance.

Raises
ValueError In case of invalid layer name or index.

summary

View source

summary(
    line_length=None, positions=None, print_fn=None
)

Prints a string summary of the network.

Args
line_length Total length of printed lines (e.g. set this to adapt the display to different terminal window sizes).
positions Relative or absolute positions of log elements in each line. If not provided, defaults to [.33, .55, .67, 1.].
print_fn Print function to use. Defaults to print. It will be called on each line of the summary. You can set it to a custom function in order to capture the string summary.

Raises
ValueError if summary() is called before the model is built.