tf.compat.v1.keras.layers.experimental.preprocessing.StringLookup

Maps strings from a vocabulary to integer indices.

Inherits From: StringLookup, PreprocessingLayer, Layer, Module

tf.compat.v1.keras.layers.experimental.preprocessing.StringLookup(
    max_tokens=None, num_oov_indices=1, mask_token='',
    oov_token='[UNK]', vocabulary=None, encoding=None, invert=False,
    **kwargs
)

Methods

adapt

View source

adapt(
    data, reset_state=True
)

Fits the state of the preprocessing layer to the dataset.

Overrides the default adapt method to apply relevant preprocessing to the inputs before passing to the combiner.

Arguments
data The data to train on. It can be passed either as a tf.data Dataset, or as a numpy array.
reset_state Optional argument specifying whether to clear the state of the layer at the start of the call to adapt. This must be True for this layer, which does not support repeated calls to adapt.

get_vocabulary

View source

get_vocabulary()

set_vocabulary

View source

set_vocabulary(
    vocab
)

Sets vocabulary data for this layer with inverse=False.

This method sets the vocabulary for this layer directly, instead of analyzing a dataset through 'adapt'. It should be used whenever the vocab information is already known. If vocabulary data is already present in the layer, this method will either replace it

Arguments
vocab An array of string tokens.

Raises
ValueError If there are too many inputs, the inputs do not match, or input data is missing.

vocab_size

View source

vocab_size()