Class: 909 (Wok), Feature: 895

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Separate Object	5	All images but image four have red regions focusing on the objects which are being fried into the wok (main object). on image four there is significant overlap in between a red region and the surface of the main object, but it's not clear to me if the focus is truly on the main object. none of the images have red regions overlapping with the background.
Separate Object	4	The main object is wok but the focus is on its food ingredients.
Separate Object	5	Red region is on the food, not on the visible edges of wok.
Separate Object	5	The focus is on to the wok 's inside dish and not on the wok
Main Object	5	The main object is a wok, pan with a convex bottom. the focus of the red region on the highlighted visual attributes is on the wok, showing its main parts. because of that, i choose that focus is on some part of the main object.


Votes for core: 1, Votes for spurious: 4. Feature is thus annotated as spurious for the class.