View source on GitHub |
Scaled Exponential Linear Unit (SELU).
tf.keras.activations.selu(
x
)
The Scaled Exponential Linear Unit (SELU) activation function is defined as:
scale * x
ifx > 0
scale * alpha * (exp(x) - 1)
ifx < 0
where alpha
and scale
are pre-defined constants
(alpha=1.67326324
and scale=1.05070098
).
Basically, the SELU activation function multiplies scale
(> 1) with the
output of the keras.activations.elu
function to ensure a slope larger
than one for positive inputs.
The values of alpha
and scale
are
chosen so that the mean and variance of the inputs are preserved
between two consecutive layers as long as the weights are initialized
correctly (see keras.initializers.LecunNormal
initializer)
and the number of input units is "large enough"
(see reference paper for more information).
Args | |
---|---|
x
|
Input tensor. |
Notes:
- To be used together with the
keras.initializers.LecunNormal
initializer. - To be used together with the dropout variant
keras.layers.AlphaDropout
(rather than regular dropout).