tf.compat.v1.lookup.StaticVocabularyTable
Stay organized with collections
Save and categorize content based on your preferences.
String to Id table wrapper that assigns out-of-vocabulary keys to buckets.
Inherits From: StaticVocabularyTable
tf.compat.v1.lookup.StaticVocabularyTable(
initializer, num_oov_buckets, lookup_key_dtype=None, name=None
)
For example, if an instance of StaticVocabularyTable is initialized with a
string-to-id initializer that maps:
emerson -> 0lake -> 1palmer -> 2
The Vocabulary object will performs the following mapping:
emerson -> 0lake -> 1palmer -> 2<other term> -> bucket_id, where bucket_id will be between 3 and
3 + num_oov_buckets - 1, calculated by:
hash(<term>) % num_oov_buckets + vocab_size
If input_tensor is ["emerson", "lake", "palmer", "king", "crimson"],
the lookup result is [0, 1, 2, 4, 7].
If initializer is None, only out-of-vocabulary buckets are used.
Example usage:
num_oov_buckets = 3
input_tensor = tf.constant(["emerson", "lake", "palmer", "king", "crimnson"])
table = tf.lookup.StaticVocabularyTable(
tf.TextFileIdTableInitializer(filename), num_oov_buckets)
out = table.lookup(input_tensor).
table.init.run()
print(out.eval())
The hash function used for generating out-of-vocabulary buckets ID is
Fingerprint64.
Args |
|---|
initializer
|
A TableInitializerBase object that contains the data used to
initialize the table. If None, then we only use out-of-vocab buckets.
|
num_oov_buckets
|
Number of buckets to use for out-of-vocabulary keys. Must
be greater than zero.
|
lookup_key_dtype
|
Data type of keys passed to lookup. Defaults to
initializer.key_dtype if initializer is specified, otherwise
tf.string. Must be string or integer, and must be castable to
initializer.key_dtype.
|
name
|
A name for the operation (optional).
|
Raises |
|---|
ValueError
|
when num_oov_buckets is not positive.
|
TypeError
|
when lookup_key_dtype or initializer.key_dtype are not
integer or string. Also when initializer.value_dtype != int64.
|
Attributes |
|---|
initializer
|
|
key_dtype
|
The table key dtype.
|
name
|
The name of the table.
|
resource_handle
|
Returns the resource handle associated with this Resource.
|
value_dtype
|
The table value dtype.
|
Methods
lookup
View source
lookup(
keys, name=None
)
Looks up keys in the table, outputs the corresponding values.
It assigns out-of-vocabulary keys to buckets based in their hashes.
| Args |
|---|
keys
|
Keys to look up. May be either a SparseTensor or dense Tensor.
|
name
|
Optional name for the op.
|
| Returns |
|---|
A SparseTensor if keys are sparse, otherwise a dense Tensor.
|
| Raises |
|---|
TypeError
|
when keys doesn't match the table key data type.
|
size
View source
size(
name=None
)
Compute the number of elements in this table.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2020-10-01 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2020-10-01 UTC."],[],[],null,["# tf.compat.v1.lookup.StaticVocabularyTable\n\n\u003cbr /\u003e\n\n|-------------------------------------------------------------------------------------------------------------------------------|\n| [View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v2.0.0/tensorflow/python/ops/lookup_ops.py#L1211-L1218) |\n\nString to Id table wrapper that assigns out-of-vocabulary keys to buckets.\n\nInherits From: [`StaticVocabularyTable`](../../../../tf/lookup/StaticVocabularyTable) \n\n tf.compat.v1.lookup.StaticVocabularyTable(\n initializer, num_oov_buckets, lookup_key_dtype=None, name=None\n )\n\nFor example, if an instance of `StaticVocabularyTable` is initialized with a\nstring-to-id initializer that maps:\n\n- `emerson -\u003e 0`\n- `lake -\u003e 1`\n- `palmer -\u003e 2`\n\nThe `Vocabulary` object will performs the following mapping:\n\n- `emerson -\u003e 0`\n- `lake -\u003e 1`\n- `palmer -\u003e 2`\n- `\u003cother term\u003e -\u003e bucket_id`, where bucket_id will be between `3` and `3 + num_oov_buckets - 1`, calculated by: `hash(\u003cterm\u003e) % num_oov_buckets + vocab_size`\n\nIf input_tensor is `[\"emerson\", \"lake\", \"palmer\", \"king\", \"crimson\"]`,\nthe lookup result is `[0, 1, 2, 4, 7]`.\n\nIf `initializer` is None, only out-of-vocabulary buckets are used.\n\n#### Example usage:\n\n num_oov_buckets = 3\n input_tensor = tf.constant([\"emerson\", \"lake\", \"palmer\", \"king\", \"crimnson\"])\n table = tf.lookup.StaticVocabularyTable(\n tf.TextFileIdTableInitializer(filename), num_oov_buckets)\n out = table.lookup(input_tensor).\n table.init.run()\n print(out.eval())\n\nThe hash function used for generating out-of-vocabulary buckets ID is\nFingerprint64.\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ---- ||\n|--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `initializer` | A TableInitializerBase object that contains the data used to initialize the table. If None, then we only use out-of-vocab buckets. |\n| `num_oov_buckets` | Number of buckets to use for out-of-vocabulary keys. Must be greater than zero. |\n| `lookup_key_dtype` | Data type of keys passed to `lookup`. Defaults to `initializer.key_dtype` if `initializer` is specified, otherwise [`tf.string`](../../../../tf#string). Must be string or integer, and must be castable to `initializer.key_dtype`. |\n| `name` | A name for the operation (optional). |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Raises ------ ||\n|--------------|-----------------------------------------------------------------------------------------------------------------------|\n| `ValueError` | when `num_oov_buckets` is not positive. |\n| `TypeError` | when lookup_key_dtype or initializer.key_dtype are not integer or string. Also when initializer.value_dtype != int64. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Attributes ---------- ||\n|-------------------|------------------------------------------------------------|\n| `initializer` | \u003cbr /\u003e \u003cbr /\u003e |\n| `key_dtype` | The table key dtype. |\n| `name` | The name of the table. |\n| `resource_handle` | Returns the resource handle associated with this Resource. |\n| `value_dtype` | The table value dtype. |\n\n\u003cbr /\u003e\n\nMethods\n-------\n\n### `lookup`\n\n[View source](https://github.com/tensorflow/tensorflow/blob/v2.0.0/tensorflow/python/ops/lookup_ops.py#L1168-L1207) \n\n lookup(\n keys, name=None\n )\n\nLooks up `keys` in the table, outputs the corresponding values.\n\nIt assigns out-of-vocabulary keys to buckets based in their hashes.\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ||\n|--------|--------------------------------------------------------------------|\n| `keys` | Keys to look up. May be either a `SparseTensor` or dense `Tensor`. |\n| `name` | Optional name for the op. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ||\n|---|---|\n| A `SparseTensor` if keys are sparse, otherwise a dense `Tensor`. ||\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Raises ||\n|-------------|----------------------------------------------------|\n| `TypeError` | when `keys` doesn't match the table key data type. |\n\n\u003cbr /\u003e\n\n### `size`\n\n[View source](https://github.com/tensorflow/tensorflow/blob/v2.0.0/tensorflow/python/ops/lookup_ops.py#L1159-L1166) \n\n size(\n name=None\n )\n\nCompute the number of elements in this table."]]
View source on GitHub