Class: 837 (Sunglasses), Feature: 1651

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Background	2	The red region in the highlighted visual attributes focuses on the background of the sunglasses.
Background	3	Red focus is on the background of the main object.
Main Object	5	The main focus is on the sunglasses, and the focus is on some of the parts of the sunglasses.
Main Object	4	Red area focusing on the bridge of the sunglasses, but sometimes it's too broad and focusing on the objects nose as well.
Separate Object	3	The red region mainly focuses on the region below the bridge of the predicted object but it sometimes focuses on persons wearing it and others on the background.


Votes for core: 2, Votes for spurious: 3. Feature is thus annotated as spurious for the class.