Class: 929 (Popsicle), Feature: 80

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Separate Object	3	Red focus is on the children mouth.
Separate Object	5	The main object is not present on images 1 and 5. on images 2, 3 and 4 there is some overlap in between a red region and the main object, but my guess is that the focus of the red regions is on the cheeks of the children (part of a different object than the main one), and not on the main object itself (popsicle).
Main Object	4	The main object is ice lolly whose some part got focus.
Separate Object	5	The main object is an ice lolly, lolly, lollipop, popsicle. the focus of the red region on the highlighted visual attributes is on the kid's, showing something that's not relevant for the visual attributes since the main object is an ice lolly or a lollipop. because of that, i choose to focus is on some other object (not a part of the main object).
Separate Object	5	The focus is not on the main object.


Votes for core: 1, Votes for spurious: 4. Feature is thus annotated as spurious for the class.