Class: 523 (Crutch), Feature: 1501

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Separate Object	3	Images 1 and 4 don't have the main object, images 2 and 3 have the main object, but the red region is not focusing on it. image 5 have the red region overlapping completely with the main object, but i've chosen to my answer based on the majority of the images, not just the fifth one. there is no red region highliting the background in any of the 5 images.
Separate Object	4	In only 1 pic is focus on crutches
Separate Object	2	Focus is on human, not on crutch
Separate Object	5	Most of the images are mainly focused on human beings.
Separate Object	4	Hands are not a part of crutch


Votes for core: 0, Votes for spurious: 5. Feature is thus annotated as spurious for the class.