tf.contrib.lookup.index_table_from_tensor
Stay organized with collections
Save and categorize content based on your preferences.
Returns a lookup table that converts a string tensor into int64 IDs.
tf.contrib.lookup.index_table_from_tensor(
mapping, num_oov_buckets=0, default_value=-1,
hasher_spec=tf.contrib.lookup.FastHashSpec, dtype=tf.dtypes.string, name=None
)
This operation constructs a lookup table to convert tensor of strings into
int64 IDs. The mapping can be initialized from a string mapping 1-D tensor
where each element is a key and corresponding index within the tensor is the
value.
Any lookup of an out-of-vocabulary token will return a bucket ID based on its
hash if num_oov_buckets is greater than zero. Otherwise it is assigned the
default_value.
The bucket ID range is [mapping size, mapping size + num_oov_buckets - 1].
The underlying table must be initialized by calling
session.run(tf.compat.v1.tables_initializer) or session.run(table.init)
once.
Elements in mapping cannot have duplicates, otherwise when executing the
table initializer op, it will throw a FailedPreconditionError.
Sample Usages:
mapping_strings = tf.constant(["emerson", "lake", "palmer"])
table = tf.contrib.lookup.index_table_from_tensor(
mapping=mapping_strings, num_oov_buckets=1, default_value=-1)
features = tf.constant(["emerson", "lake", "and", "palmer"])
ids = table.lookup(features)
...
tf.compat.v1.tables_initializer().run()
ids.eval() ==> [0, 1, 3, 2]
Args |
|---|
mapping
|
A 1-D Tensor that specifies the mapping of keys to indices. The
type of this object must be castable to dtype.
|
num_oov_buckets
|
The number of out-of-vocabulary buckets.
|
default_value
|
The value to use for out-of-vocabulary feature values.
Defaults to -1.
|
hasher_spec
|
A HasherSpec to specify the hash function to use for
assignment of out-of-vocabulary buckets.
|
dtype
|
The type of values passed to lookup. Only string and integers are
supported.
|
name
|
A name for this op (optional).
|
Returns |
|---|
The lookup table to map an input Tensor to index int64 Tensor.
|
Raises |
|---|
ValueError
|
If mapping is invalid.
|
ValueError
|
If num_oov_buckets is negative.
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2020-10-01 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2020-10-01 UTC."],[],[],null,["# tf.contrib.lookup.index_table_from_tensor\n\n\u003cbr /\u003e\n\n|---------------------------------------------------------------------------------------------------------------------------------|\n| [View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/contrib/lookup/lookup_ops.py#L77-L143) |\n\nReturns a lookup table that converts a string tensor into int64 IDs. \n\n tf.contrib.lookup.index_table_from_tensor(\n mapping, num_oov_buckets=0, default_value=-1,\n hasher_spec=tf.contrib.lookup.FastHashSpec, dtype=tf.dtypes.string, name=None\n )\n\nThis operation constructs a lookup table to convert tensor of strings into\nint64 IDs. The mapping can be initialized from a string `mapping` 1-D tensor\nwhere each element is a key and corresponding index within the tensor is the\nvalue.\n\nAny lookup of an out-of-vocabulary token will return a bucket ID based on its\nhash if `num_oov_buckets` is greater than zero. Otherwise it is assigned the\n`default_value`.\nThe bucket ID range is `[mapping size, mapping size + num_oov_buckets - 1]`.\n\nThe underlying table must be initialized by calling\n`session.run(tf.compat.v1.tables_initializer)` or `session.run(table.init)`\nonce.\n\nElements in `mapping` cannot have duplicates, otherwise when executing the\ntable initializer op, it will throw a `FailedPreconditionError`.\n\n#### Sample Usages:\n\n mapping_strings = tf.constant([\"emerson\", \"lake\", \"palmer\"])\n table = tf.contrib.lookup.index_table_from_tensor(\n mapping=mapping_strings, num_oov_buckets=1, default_value=-1)\n features = tf.constant([\"emerson\", \"lake\", \"and\", \"palmer\"])\n ids = table.lookup(features)\n ...\n tf.compat.v1.tables_initializer().run()\n\n ids.eval() ==\u003e [0, 1, 3, 2]\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ---- ||\n|-------------------|--------------------------------------------------------------------------------------------------------------------|\n| `mapping` | A 1-D `Tensor` that specifies the mapping of keys to indices. The type of this object must be castable to `dtype`. |\n| `num_oov_buckets` | The number of out-of-vocabulary buckets. |\n| `default_value` | The value to use for out-of-vocabulary feature values. Defaults to -1. |\n| `hasher_spec` | A `HasherSpec` to specify the hash function to use for assignment of out-of-vocabulary buckets. |\n| `dtype` | The type of values passed to `lookup`. Only string and integers are supported. |\n| `name` | A name for this op (optional). |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ------- ||\n|---|---|\n| The lookup table to map an input `Tensor` to index `int64` `Tensor`. ||\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Raises ------ ||\n|--------------|-----------------------------------|\n| `ValueError` | If `mapping` is invalid. |\n| `ValueError` | If `num_oov_buckets` is negative. |\n\n\u003cbr /\u003e"]]
View source on GitHub