tfm.nlp.layers.TNTransformerExpandCondense
Stay organized with collections
Save and categorize content based on your preferences.
Transformer layer using tensor network Expand-Condense layer.
tfm.nlp.layers.TNTransformerExpandCondense(
num_attention_heads,
intermediate_size,
intermediate_activation,
dropout_rate=0.0,
attention_dropout_rate=0.0,
output_range=None,
kernel_initializer='glorot_uniform',
bias_initializer='zeros',
kernel_regularizer=None,
bias_regularizer=None,
activity_regularizer=None,
kernel_constraint=None,
bias_constraint=None,
use_bias=True,
norm_first=False,
norm_epsilon=1e-12,
intermediate_dropout=0.0,
attention_initializer=None,
**kwargs
)
This layer implements the Transformer from transformer.py, with a single
tensor network layer replacing the usual intermediate and output Dense
layers.
Args |
num_attention_heads
|
Number of attention heads.
|
intermediate_size
|
Size of the intermediate layer.
|
intermediate_activation
|
Activation for the intermediate layer.
|
dropout_rate
|
Dropout probability for the post-attention and output dropout.
|
attention_dropout_rate
|
Dropout probability for within the attention layer.
|
output_range
|
the sequence output range, [0, output_range) by slicing the
target sequence. None means the target sequence is not sliced.
|
kernel_initializer
|
Initializer for dense layer kernels.
|
bias_initializer
|
Initializer for dense layer biases.
|
kernel_regularizer
|
Regularizer for dense layer kernels.
|
bias_regularizer
|
Regularizer for dense layer biases.
|
activity_regularizer
|
Regularizer for dense layer activity.
|
kernel_constraint
|
Constraint for dense layer kernels.
|
bias_constraint
|
Constraint for dense layer kernels.
|
use_bias
|
Whether to enable use_bias in attention layer. If set to False,
use_bias in attention layer is disabled.
|
norm_first
|
Whether to normalize inputs to attention and intermediate dense
layers. If set False, output of attention and intermediate dense layers is
normalized.
|
norm_epsilon
|
Epsilon value to initialize normalization layers.
|
intermediate_dropout
|
Dropout probability for intermediate_dropout_layer.
|
attention_initializer
|
Initializer for kernels of attention layers. If set
None, attention layers use kernel_initializer as initializer for kernel.
|
Methods
call
View source
call(
inputs
)
This is where the layer's logic lives.
The call() method may not create state (except in its first
invocation, wrapping the creation of variables or other resources in
tf.init_scope()). It is recommended to create state, including
tf.Variable instances and nested Layer instances,
in __init__(), or in the build() method that is
called automatically before call() executes for the first time.
| Args |
inputs
|
Input tensor, or dict/list/tuple of input tensors.
The first positional inputs argument is subject to special rules:
inputs must be explicitly passed. A layer cannot have zero
arguments, and inputs cannot be provided via the default value
of a keyword argument.
- NumPy array or Python scalar values in
inputs get cast as
tensors.
- TF-Keras mask metadata is only collected from
inputs.
- Layers are built (
build(input_shape) method)
using shape info from inputs only.
input_spec compatibility is only checked against inputs.
- Mixed precision input casting is only applied to
inputs.
If a layer has tensor arguments in *args or **kwargs, their
casting behavior in mixed precision should be handled manually.
- The SavedModel input specification is generated using
inputs
only.
- Integration with various ecosystem packages like TFMOT, TFLite,
TF.js, etc is only supported for
inputs and not for tensors in
positional and keyword arguments.
|
*args
|
Additional positional arguments. May contain tensors, although
this is not recommended, for the reasons above.
|
**kwargs
|
Additional keyword arguments. May contain tensors, although
this is not recommended, for the reasons above.
The following optional keyword arguments are reserved:
training: Boolean scalar tensor of Python boolean indicating
whether the call is meant for training or inference.
mask: Boolean input mask. If the layer's call() method takes a
mask argument, its default value will be set to the mask
generated for inputs by the previous layer (if input did come
from a layer that generated a corresponding mask, i.e. if it came
from a TF-Keras layer with masking support).
|
| Returns |
|
A tensor or list/tuple of tensors.
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates. Some content is licensed under the numpy license.
Last updated 2026-11-06 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2026-11-06 UTC."],[],[]]