CoRL 2026

HelpMeOut: A Neuropsychology-Grounded Benchmark for Error-Aware Monitoring of Household Task Execution

Household benchmarks ask what a person wants to do. HelpMeOut asks whether their ongoing execution is going wrong, when, and how.

Haoran Wang1Jingwei Huang2Anqing Duan2Ho Seok Ahn1Bruce Macdonald1
1University of Auckland, New Zealand2MBZUAI, United Arab Emirates
arXiv Code soon

Monitoring, one step at a time

A monitor sees the task, the initial scene, and the actions so far, never the future. After every step it must say whether an error has occurred, whether it happens at this step, and which kinds of error have appeared. Step through an instant-coffee episode.

task: make instant coffee  ·  objects: mug, coffee, water, sugar, spoon, fork
Correct reference (hidden from the monitor)
Observed execution
Whether
Where
What kind

Four questions, four metrics

Each metric isolates one part of the monitoring problem, so a single accuracy number cannot hide where a method fails.

M1

Whether

Trajectory-level discrimination: does this episode contain an error?

M2

Where

Step-level detection: is the error happening at this step?

M3

When

First-alarm timing: is the first alarm early, on target, late, or missing?

M4

What kind

Error-type diagnosis over the nine categories, accumulated over the episode.

Nine kinds of everyday-action error

The categories follow clinical studies of everyday action (NAT / MLAT). Every errorful trajectory stays physically executable: the person can carry out each action, but the execution is wrong for the task.

Progression-level

Break the structure or order of the task.

OMISSIONA step the task requires is never done; confirmed when the person declares the task finished.
SEQUENCETwo steps that must happen in order are done in the wrong order, e.g. buttering bread before toasting it.
ANTICIPATIONAn item is stored, served, or packed before it receives the processing the task requires.
PERSEVERATIONAfter doing something else, the person repeats an action that should be done only once.

Current-action

Corrupt the choice of object, tool, location, or action.

OBJECT_SUBSTITUTIONA similar object of the same kind is used instead of the one the task needs.
TOOL_SUBSTITUTIONA similar tool is used instead of the appropriate one, e.g. stirring with a fork.
ACTION_ADDITIONAn unrelated item is put into or onto the task's target container, plate, or tray.
CAPTURE_ERRORAn action the task needs is applied to an unrelated object of a different kind.
SPATIAL_MISREACHThe person goes to an unrelated storage location, opens it, and leaves without taking anything.

The benchmark

A symbolic household domain turns ADL / IADL routines into paired correct and errorful episodes, with error rates calibrated to clinical populations.

20,000episodes: 12,000 errorful, 8,000 correct
50ADL / IADL task templates
9error categories
38symbolic action operators
Error-generation pipeline: a clinical profile drives a goal-control model that samples an error type and onset, edits a valid reference trajectory, and validates the errorful trajectory by symbolic replay.
Error generation. A clinical profile drives a goal-control model that samples an error type and onset, edits a valid reference trajectory, and validates the result by symbolic replay.

Citation

@inproceedings{wang2026helpmeout,
  title     = {HelpMeOut: A Neuropsychology-Grounded Benchmark for Error-Aware Monitoring of Household Task Execution},
  author    = {Wang, Haoran and Huang, Jingwei and Duan, Anqing and Ahn, Ho Seok and Macdonald, Bruce},
  booktitle = {Conference on Robot Learning (CoRL)},
  year      = {2026}
}