Class: 685 (Odometer), Feature: 1204

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Main Object	5	The red region in the highlighted visual attributes focuses on the part of an odometer.
Separate Object	4	It seems that the red object it is trying to hit is the speedometer which is the needle and not the odometer which is the list of numbers tracking mileage.
Main Object	5	Because focus in shown inside the odometer only, its a part of the main object, not other object or background of the main object.
Separate Object	5	The focus of the red region in the highlighted visual attributes is on the meter that shows the mileage traversed.
Main Object	4	Above images focusing on some part of the main object.


Votes for core: 3, Votes for spurious: 2. Feature is thus annotated as core for the class.