mt_opt

  • Description:

Datasets for the MT-Opt paper.

@misc{kalashnikov2021mtopt,
      title={MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale},
      author={Dmitry Kalashnikov and Jacob Varley and Yevgen Chebotar and Benjamin Swanson and Rico Jonschkowski and Chelsea Finn and Sergey Levine and Karol Hausman},
      year={2021},
      eprint={2104.08212},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}

mt_opt/rlds (default config)

  • Config description: This dataset contains task episodes collected across afleet of real robots. It follows the RLDS formatto represent steps and episodes.

  • Dataset size: 4.38 TiB

  • Splits:

Split Examples
'train' 920,165
  • Feature structure:
FeaturesDict({
    'episode_id': string,
    'skill': uint8,
    'steps': Dataset({
        'action': FeaturesDict({
            'close_gripper': bool,
            'open_gripper': bool,
            'target_pose': Tensor(shape=(7,), dtype=float32),
            'terminate': bool,
        }),
        'is_first': bool,
        'is_last': bool,
        'is_terminal': bool,
        'observation': FeaturesDict({
            'gripper_closed': bool,
            'height_to_bottom': float32,
            'image': Image(shape=(512, 640, 3), dtype=uint8),
            'state_dense': Tensor(shape=(7,), dtype=float32),
        }),
    }),
    'task_code': string,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
episode_id Tensor string
skill Tensor uint8
steps Dataset
steps/action FeaturesDict
steps/action/close_gripper Tensor bool
steps/action/open_gripper Tensor bool
steps/action/target_pose Tensor (7,) float32
steps/action/terminate Tensor bool
steps/is_first Tensor bool
steps/is_last Tensor bool
steps/is_terminal Tensor bool
steps/observation FeaturesDict
steps/observation/gripper_closed Tensor bool
steps/observation/height_to_bottom Tensor float32
steps/observation/image Image (512, 640, 3) uint8
steps/observation/state_dense Tensor (7,) float32
task_code Tensor string

mt_opt/sd

  • Config description: The success detectors dataset that contains human curated definitions of tasks completion.

  • Dataset size: 548.56 GiB

  • Splits:

Split Examples
'test' 94,636
'train' 380,234
  • Feature structure:
FeaturesDict({
    'image_0': Image(shape=(512, 640, 3), dtype=uint8),
    'image_1': Image(shape=(480, 640, 3), dtype=uint8),
    'image_2': Image(shape=(480, 640, 3), dtype=uint8),
    'success': bool,
    'task_code': string,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
image_0 Image (512, 640, 3) uint8
image_1 Image (480, 640, 3) uint8
image_2 Image (480, 640, 3) uint8
success Tensor bool
task_code Tensor string