Class: 1 (Goldfish), Feature: 495

MTurk Worker Annotations

We now show the responses from five crowdworkers used to annotate this class-feature pair as core or spurious. Workers first denote if the focus of the image is on the main object (core), or a separate or background object (spurious). They also mark their confidence on a scale of 1 to 5, and provide written explanations for their annotation.



Main Focus	Confidence	Explanation
Main Object	5	The main object is an goldfish. the focus of the red region in the highlighted visual attributes is on the fish's head and body, showing their main parts. because of that, i choose focus is on some part of the main object.
Main Object	5	The majority of the red regions contain the color orange. the object definition states that goldfish are golden or orange-red. in addition some of the red regions contain fin shapes. these match and look similar to the main object photos provided. comparing all this information, it is highly likely that the red regions contain some part of the main object.
Main Object	5	The red region in the highlighted visual attributes focuses on the goldfish.
Main Object	5	Because the focus of the red color was at least somewhat on the main object in the highlighted visual attributes, in this case the goldfish.
Main Object	3	Yellow fish is highlighted in red colour.


Votes for core: 5, Votes for spurious: 0. Feature is thus annotated as core for the class.