Class: 631 (Lotion), Feature: 799

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Main Object	4	The red region mainly focuses on the predicted object but also spreads to the background.
Separate Object	4	The red focuses on the labels on the lotion.
Main Object	5	There is some overlap in between a red region and the background on images 1 and 5, but the focus is clearly on the main object. there are no other objects other than the main one, and the background, in any of the five images for a red region to overlap with.
Main Object	5	The main focus is on the lotion bottle which is the main object
Main Object	4	Focus is on the label.


Votes for core: 4, Votes for spurious: 1. Feature is thus annotated as core for the class.