The app also allows the user to get additional information about the bird species from Wikipedia and Google. Users can also see additional pictures on Flickr.
Under the hood
Computer vision searches involve two levels of processing. Entry-level categories identify the kind of image the user is looking for, such as a bird, a car, a flower and so on. For Visipedia, that category is already preset, since the user is exclusively searching for birds.
The second level of processing is known as fine-grained categories. Each category is defined by its parts and attributes. For birds that would be the head, wings, tail, and so on, and their color and shape, for example. The app generates a heat map that helps the system determine which areas of the picture are likely bird parts. Each part and attribute are stored in the app's code as a long list of numbers, called vectors. As more users work with the app, they train the app's detectors to become more accurate. The process should yield some additional benefits, Belongie said. For example, code designed to recognize butterflies could also be helpful for moths.
The app's user interface was designed by computer science undergraduate student Grant Van Horn. Its search system is the work of computer science graduate students Catherine Wah and Steven Branson in the Department of Computer Science and Engineering at the Jacobs School. Belongie has applied for a grant that would allow him to hire a developer, who could package the app's code for other tablets and smart phones and maintain it. That's what it will take to get it in the App Store, the researcher explained. Meanwhile, a demo version of the app is available upon request via h
|Contact: Ioana Patringenaru|
University of California - San Diego