Class: 585 (Hair Spray), Feature: 1019

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Main Object	3	Focus on bottle
Separate Object	2	Algorithm picks up labels/brand names/containers. but not hair spray specifically. it seems to identify beer bottle labels/containers as hair spray also.
Main Object	4	The focus is on the labels of the hair spray bottles.
Separate Object	4	I am not sure whether you consider the labeling as part of the hairspray. i saw your example of graffiti on the freight car. hence, this falls under the same category.
Main Object	4	The red region in the highlighted visual attributes focuses on the part of the hair spray that is the main object.


Votes for core: 3, Votes for spurious: 2. Feature is thus annotated as core for the class.