asimov_dilemmas_scifi_val

  • Description:

Multiple-choice ethical questions (with desirable and undesirable answers) based on situations inspired from Science Fiction literature (validation set).

Split Examples
'val' 51
  • Feature structure:
FeaturesDict({
    'acting_character': Text(shape=(), dtype=string),
    'characters': Text(shape=(), dtype=string),
    'possible_actions': Sequence({
        'action': Text(shape=(), dtype=string),
        'is_original_scifi_decision': bool,
        'key': Text(shape=(), dtype=string),
        'undesirable_groundtruth_answer': bool,
    }),
    'prompt_with_constitution': Text(shape=(), dtype=string),
    'prompt_with_constitution_antijailbreak': Text(shape=(), dtype=string),
    'prompt_with_constitution_antijailbreak_adversary': Text(shape=(), dtype=string),
    'prompt_with_constitution_antijailbreak_adversary_parts': Sequence(Text(shape=(), dtype=string)),
    'prompt_with_constitution_antijailbreak_parts': Sequence(Text(shape=(), dtype=string)),
    'prompt_with_constitution_parts': Sequence(Text(shape=(), dtype=string)),
    'prompt_without_constitution': Text(shape=(), dtype=string),
    'prompt_without_constitution_parts': Sequence(Text(shape=(), dtype=string)),
    'reference_domain': Text(shape=(), dtype=string),
    'reference_moment': Text(shape=(), dtype=string),
    'reference_scifi': Text(shape=(), dtype=string),
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
acting_character Text string
characters Text string
possible_actions Sequence
possible_actions/action Text string
possible_actions/is_original_scifi_decision Tensor bool
possible_actions/key Text string
possible_actions/undesirable_groundtruth_answer Tensor bool
prompt_with_constitution Text string
prompt_with_constitution_antijailbreak Text string
prompt_with_constitution_antijailbreak_adversary Text string
prompt_with_constitution_antijailbreak_adversary_parts Sequence(Text) (None,) string
prompt_with_constitution_antijailbreak_parts Sequence(Text) (None,) string
prompt_with_constitution_parts Sequence(Text) (None,) string
prompt_without_constitution Text string
prompt_without_constitution_parts Sequence(Text) (None,) string
reference_domain Text string
reference_moment Text string
reference_scifi Text string
  • Citation:
@article{sermanet2025asimov,
  author    = {Pierre Sermanet and Anirudha Majumdar and Alex Irpan and Dmitry Kalashnikov and Vikas Sindhwani},
  title     = {Generating Robot Constitutions & Benchmarks for Semantic Safety},
  journal   = {arXiv preprint arXiv:2503.08663},
  url       = {https://arxiv.org/abs/2503.08663},
  year      = {2025},
}