The idea in plain language.
Understanding an event includes reasoning about intentions, reactions, and likely next actions.
How it works
Each example describes a social situation and asks a question about it. Crowdworkers provide correct answers and plausible distractors. The collection method tries to reduce stylistic clues that let a model choose an answer without understanding the situation. The paper uses the benchmark to examine gaps between model and human performance and to study transfer to related tasks.
What to keep in mind
Multiple-choice performance is a limited view of social understanding. Cultural variation and dataset construction affect which answers are considered appropriate.
Source: SocialIQA: Commonsense Reasoning about Social Interactions. The original manuscript contains the methods, experiments, figures, and references. An arXiv posting date may follow an earlier conference publication. Read the linked record for version history.