Class: 124 (Crayfish), Feature: 110

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Main Object	4	Focus is on features of the object.
Main Object	5	All 5 images shown are close shot pictures of the main object. there is no background, or other object to be shown, and the red regions overlap fully with parts of the main object in all five images.
Main Object	4	The red region mainly focuses on the predicted object but overlaps slightly with the background.
Main Object	5	The main focus is on the cray fish's body as a whole which is the main object
Main Object	4	From the heatmaps of the product it is clear that the intensity of red color is high on some part of the main object.


Votes for core: 5, Votes for spurious: 0. Feature is thus annotated as core for the class.