Class: 626 (Lighter), Feature: 1009

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Separate Object	4	Flames from the lighter are highlighted in the picture.
Separate Object	4	The red region in the highlighted visual attributes focuses on the flame of a lighter.
Background	5	Focus is on the background of the main object.
Separate Object	4	It's not really focusing on the device but rather the flame caused by the device. i doubt it would correctly identify the device if there was no flame coming from it.
Background	3	Amplified visual attributes can be useful in some cases when the red region overlaps over multiple objects such as in these examples: overlap between cougar and wires, overlap between groom and flowers.


Votes for core: 0, Votes for spurious: 5. Feature is thus annotated as spurious for the class.