Class: 785 (Seat Belt), Feature: 108

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Separate Object	4	Focus is on a person's face.
Separate Object	4	The seat belt is not at all focused . the persons face is the prime focus in the picture
Separate Object	5	The main object is an seat belt, used in a car or plane to hold you in your seat in case of an accident. the focus of the red region on the highlighted visual attributes is on something else, like people's face and head, showing something that's not relevant for the visual attributes since the main object is an seat belt. because of that, i choose to focus is on some other object (not a part of the main object).
Separate Object	5	Focus is on another object that is not part of the main object.
Separate Object	4	There are red parts of heat map over facial parts. the one photo showing an actual seatbelt, the red heat map does not even cover the whole seatbelt, only a small portion of it. the amplified attributes doesn't even seem to contain seatbelts at all.


Votes for core: 0, Votes for spurious: 5. Feature is thus annotated as spurious for the class.