Paper Reading #13: Cosaliency: Where People Look When Comparing Images

Comments: Alyssa Nabors, Miguel Alex Cardenas.
Reference Information:
Title: Cosaliency: Where People Look When Comparing Images
Authors: David E. Jacobs, Dan B. Goldman, Eli Shechtman.
Venue: UIST’10, October 3–6, 2010, New York, New York, USA.

Summary: In Cosaliency, the authors discuss the algorithms involved in detecting the significant changes between two different, but similar images. The purpose of their algorithm is to be able to quantifiably show the differences in posture, orientation, and appearance. The use case they gave for such an algorithm is so called triage situations on a digital camera, where a photographer needs to decide which images are worth keeping, and which ones they can delete in order to quickly free up space on their card.

Using Amazon's Mechanical Turk service, the authors developed a baseline set of image, in which humans had pointed out what they considered the most important portions of an image. These test subjects generally focused on the human subjects of the photos, giving a bit of buffer area around the image to maintain context when making the triage decisions.

Using the information gathered form the human tests subjects, the developers designed an algorithm which performed the same context aware cropping procedure as the Mechanical Turks had done. The developers used a "leaning machine" approach, which used the human-cropped photos in order to gain a sense of aesthetic. In the end, the machine cropped photos were found more useful than the ones cropped by the humans.

Discussion: This article was fairly interesting, although highly technical. I found their use of Amazon's Mechanical Turk service highly innovative, and an incredibly useful way of gaining a baseline to use for a machine learning technique. One aspect I have to call into question about the entire premise however is that the price of storage seems to be rapidly decreasing, while the size of storage mechanisms conversely is increasing. By the time they perfect their software, people could very easily be no longer worried about the size of their camera's storage.