tfr.keras.pipeline.AbstractDatasetBuilder
Stay organized with collections
Save and categorize content based on your preferences.
Interface for datasets and signatures.
The AbstractDatasetBuilder
class is an abstract class to serve data in
tfr.keras. A DatasetBuilder
will be passed to an instance of
AbstractPipeline
and called to serve the training and validation datasets
and to define the serving signatures for saved models to treat the
corresponding format of data.
To be implemented by subclasses:
build_train_dataset()
: Contains the logic to build a tf.data.Dataset
for training.
build_valid_dataset()
: Contains the logic to build a tf.data.Dataset
for validation.
build_signatures()
: Contains the logic to build a dict of signatures
that formulate the model in functions that render the input data with given
format.
Example subclass implementation:
class NullDatasetBuilder(AbstractDatasetBuilder):
def __init__(self, train_dataset, valid_dataset, signatures=None):
self._train_dataset = train_dataset
self._valid_dataset = valid_dataset
self._signatures = signatures
def build_train_dataset(self, *arg, **kwargs) -> tf.data.Dataset:
return self._train_dataset
def build_valid_dataset(self, *arg, **kwargs) -> tf.data.Dataset:
return self._valid_dataset
def build_signatures(self, *arg, **kwargs) -> Any:
return self._signatures
Methods
build_signatures
View source
@abc.abstractmethod
build_signatures(
*arg, **kwargs
) -> Any
Returns the signatures to export a SavedModel.
Example usage:
dataset_builder = NullDatasetBuilder(train_data, valid_data)
signatures = dataset_builder.build_signatures()
Args |
*arg
|
arguments that might be used to build signatures.
|
**kwargs
|
keyword arguments that might be used to build signatures.
|
Returns |
None or a dict of concrete functions.
|
build_train_dataset
View source
@abc.abstractmethod
build_train_dataset(
*arg, **kwargs
) -> tf.data.Dataset
Returns the training dataset.
Example usage:
dataset_builder = NullDatasetBuilder(train_data, valid_data)
train_dataset = dataset_builder.build_train_dataset()
Args |
*arg
|
arguments that might be used to build training dataset.
|
**kwargs
|
keyword arguments that might be used to build training dataset.
|
build_valid_dataset
View source
@abc.abstractmethod
build_valid_dataset(
*arg, **kwargs
) -> tf.data.Dataset
Returns the validation dataset.
Example usage:
dataset_builder = NullDatasetBuilder(train_data, valid_data)
valid_dataset = dataset_builder.build_valid_dataset()
Args |
*arg
|
arguments that might be used to build validation dataset.
|
**kwargs
|
keyword arguments that might be used to build validation
dataset.
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2023-08-18 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2023-08-18 UTC."],[],[],null,["# tfr.keras.pipeline.AbstractDatasetBuilder\n\n\u003cbr /\u003e\n\n|----------------------------------------------------------------------------------------------------------------------------------|\n| [View source on GitHub](https://github.com/tensorflow/ranking/blob/v0.5.3/tensorflow_ranking/python/keras/pipeline.py#L159-L258) |\n\nInterface for datasets and signatures.\n\nThe `AbstractDatasetBuilder` class is an abstract class to serve data in\ntfr.keras. A `DatasetBuilder` will be passed to an instance of\n`AbstractPipeline` and called to serve the training and validation datasets\nand to define the serving signatures for saved models to treat the\ncorresponding format of data.\n\nTo be implemented by subclasses:\n\n- `build_train_dataset()`: Contains the logic to build a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) for training.\n- `build_valid_dataset()`: Contains the logic to build a [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) for validation.\n- `build_signatures()`: Contains the logic to build a dict of signatures that formulate the model in functions that render the input data with given format.\n\nExample subclass implementation: \n\n class NullDatasetBuilder(AbstractDatasetBuilder):\n\n def __init__(self, train_dataset, valid_dataset, signatures=None):\n self._train_dataset = train_dataset\n self._valid_dataset = valid_dataset\n self._signatures = signatures\n\n def build_train_dataset(self, *arg, **kwargs) -\u003e tf.data.Dataset:\n return self._train_dataset\n\n def build_valid_dataset(self, *arg, **kwargs) -\u003e tf.data.Dataset:\n return self._valid_dataset\n\n def build_signatures(self, *arg, **kwargs) -\u003e Any:\n return self._signatures\n\nMethods\n-------\n\n### `build_signatures`\n\n[View source](https://github.com/tensorflow/ranking/blob/v0.5.3/tensorflow_ranking/python/keras/pipeline.py#L240-L258) \n\n @abc.abstractmethod\n build_signatures(\n *arg, **kwargs\n ) -\u003e Any\n\nReturns the signatures to export a SavedModel.\n\n#### Example usage:\n\n dataset_builder = NullDatasetBuilder(train_data, valid_data)\n signatures = dataset_builder.build_signatures()\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ||\n|------------|-----------------------------------------------------------|\n| `*arg` | arguments that might be used to build signatures. |\n| `**kwargs` | keyword arguments that might be used to build signatures. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ||\n|---|---|\n| None or a dict of concrete functions. ||\n\n\u003cbr /\u003e\n\n### `build_train_dataset`\n\n[View source](https://github.com/tensorflow/ranking/blob/v0.5.3/tensorflow_ranking/python/keras/pipeline.py#L199-L217) \n\n @abc.abstractmethod\n build_train_dataset(\n *arg, **kwargs\n ) -\u003e tf.data.Dataset\n\nReturns the training dataset.\n\n#### Example usage:\n\n dataset_builder = NullDatasetBuilder(train_data, valid_data)\n train_dataset = dataset_builder.build_train_dataset()\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ||\n|------------|-----------------------------------------------------------------|\n| `*arg` | arguments that might be used to build training dataset. |\n| `**kwargs` | keyword arguments that might be used to build training dataset. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ||\n|---|---|\n| A [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset). ||\n\n\u003cbr /\u003e\n\n### `build_valid_dataset`\n\n[View source](https://github.com/tensorflow/ranking/blob/v0.5.3/tensorflow_ranking/python/keras/pipeline.py#L219-L238) \n\n @abc.abstractmethod\n build_valid_dataset(\n *arg, **kwargs\n ) -\u003e tf.data.Dataset\n\nReturns the validation dataset.\n\n#### Example usage:\n\n dataset_builder = NullDatasetBuilder(train_data, valid_data)\n valid_dataset = dataset_builder.build_valid_dataset()\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ||\n|------------|-------------------------------------------------------------------|\n| `*arg` | arguments that might be used to build validation dataset. |\n| `**kwargs` | keyword arguments that might be used to build validation dataset. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ||\n|---|---|\n| A [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset). ||\n\n\u003cbr /\u003e"]]