Navigation Links
Computer scientists develop an interactive field guide app for birders

A team of researchers led by computer scientist Serge Belongie at the University of California, San Diego, has good news for birders: they have developed an iPad app that will identify most North American birds, with a little help from a human user.

The app is essentially an interactive field guide, where computer vision algorithms analyze the picture the user submitted, ask questions and call up pages with pictures and information about a bird species that is a likely match. The researchers' ultimate goal is to fill a gap in the world of online search. Text-driven search, such as Google and Wikipedia, has been wildly successful. But identifying images has so far proven much more difficult.

That process involves uploading a picture to a search tool, such as Google Goggles, and asking what it is. This works well with well-known landmarks and pictures. But it mostly fails with lesser-known visuals, such as images of animals and plants.

Computer scientists at the Jacobs School of Engineering at UC San Diego, UC Berkeley and Caltech have designed a search system that can interact with users to provide more accurate results. For example, when the system is stumped, users can label parts of an image, such as a bird's head, chest and tail and provide information about other characteristics, such as the color of the bird's coat.

"We chose birds for several reasons: there is an abundance of excellent photos of birds available on the Internet, the diversity in appearance across different bird species presents a deep technical challenge and, perhaps most importantly, there is a large community of passionate birders who can put our system to the test," explained Belongie, the Jacobs School of Engineering computer scientist at the heart of the project.

The Cornell Lab of Ornithology is contributing its database of images showing the male, female and juveniles of more than 500 North American species. Citizen scientists working with the laboratory will label the images and expand the database further. The ultimate goal is to have 100 images identified by experts for each species.

"We are helping Visipedia reach an audience of birders who are hungry for this kind of technology," said Jessie Barry, a researcher at the Cornell Lab of Ornithology.

The hope is that the app will strengthen the users' connection to the natural world and their desire to preserve natural habitats. Barry and her team are recruiting citizen scientists through the lab's website,, which has more than one million unique visitors a month. When researchers asked birders to help label birds' colors for Visipedia earlier this year, they received one answer a minute, on average. Birders are eager to help train the Visipedia system, Barry said, because they know it will ultimately become a tool that will help them in the field.

Belongie hopes the app will ignite a movement among other enthusiasts, who will partner with computer scientists to create similar apps for everything from flowers, to butterflies, to lichens and mushrooms.

"We can't solve all the challenges by ourselves, but we believe our framework can," Belongie said.

How the app works: the user experience

Users upload their pictures to an iPad equipped with the Visipedia app. The app's algorithms process the picture and display a series of results deemed a good match. Sometimes the app is spot on. But sometimes, it requires some additional help from the user.

To find a more accurate result, the app will ask the user to identify a specific part of the bird in the original picture, such as the head, the tail or the wing. Sometimes, the app also will ask what colors can be found on that part of the bird. Based on the user's answers, the app generates more results, until it finds the correct answer.

This method is similar to a game of 20 questions, played between the system and the user. But the system analyzes the image first to minimize the number of questions a user has to answer. Typically, the app takes about five to six questions to get it right, Belongie said. The goal is to fine-tune it until the right answer pops up after only two to three questions.

The app also allows the user to get additional information about the bird species from Wikipedia and Google. Users can also see additional pictures on Flickr.

Under the hood

Computer vision searches involve two levels of processing. Entry-level categories identify the kind of image the user is looking for, such as a bird, a car, a flower and so on. For Visipedia, that category is already preset, since the user is exclusively searching for birds.

The second level of processing is known as fine-grained categories. Each category is defined by its parts and attributes. For birds that would be the head, wings, tail, and so on, and their color and shape, for example. The app generates a heat map that helps the system determine which areas of the picture are likely bird parts. Each part and attribute are stored in the app's code as a long list of numbers, called vectors. As more users work with the app, they train the app's detectors to become more accurate. The process should yield some additional benefits, Belongie said. For example, code designed to recognize butterflies could also be helpful for moths.

The app's user interface was designed by computer science undergraduate student Grant Van Horn. Its search system is the work of computer science graduate students Catherine Wah and Steven Branson in the Department of Computer Science and Engineering at the Jacobs School. Belongie has applied for a grant that would allow him to hire a developer, who could package the app's code for other tablets and smart phones and maintain it. That's what it will take to get it in the App Store, the researcher explained. Meanwhile, a demo version of the app is available upon request via

Contact: Ioana Patringenaru
University of California - San Diego

Related biology news :

1. Motorola Introduces Mobile Biometric Identification for Handheld Computers
2. Coming soon: Self-guided, computer-based depression treatment
3. New data resource to advance computer-aided drug design
4. San Diego Supercomputer Center and UCSD announce Triton Resource
5. Could Dr. House be replaced by a computer?
6. Supercomputer provides massive computational boost to biomedical research at TGen
7. Oak Ridge supercomputer is the worlds fastest for science
8. ORNL supercomputer simulation wins prize for fastest-running science application
9. Singapore research team first place in Brain-Computer Interface contest
10. Lifecycles of tropical cyclones predicted in global computer model
11. New open-source software permits faster desktop computer simulations of molecular motion
Post Your Comments:
Related Image:
Computer scientists develop an interactive field guide app for birders
(Date:11/19/2015)... Calif. , Nov. 19, 2015  Based on ... Frost & Sullivan recognizes BIO-key with the 2015 Global ... Each year, Frost & Sullivan presents this award to ... line catering to the needs of the market it ... product line meets and expands on customer base demands, ...
(Date:11/18/2015)... 18, 2015 --> ... new market report titled  Gesture Recognition Market - Global ... - 2021. According to the report, the global gesture recognition market was ... to reach US$29.1 bn by 2021, at a CAGR ... North America dominated the global gesture recognition ...
(Date:11/17/2015)... 2015  Vigilant Solutions announces today that Mr. ... Directors. --> --> ... the partnership at TPG Capital, one of the largest ... Billion in revenue.  He founded and led TPG,s Operating ... companies, from 1997 to 2013.  In his first role, ...
Breaking Biology News(10 mins):
(Date:11/24/2015)... , November 24, 2015 SHPG ) announced ... in the Piper Jaffray 27 th Annual Healthcare Conference in ... 1, 2015, at 8:30 a.m. EST (1:30 p.m. GMT). --> ... Financial Officer, will participate in the Piper Jaffray 27 th ... NY on Tuesday, December 1, 2015, at 8:30 a.m. EST (1:30 ...
(Date:11/24/2015)... , Nov. 24, 2015 /PRNewswire/ - Aeterna ... announced today that the remaining 11,000 post-share consolidation ... Purchase Warrants (the "Series B Warrants") subject to ... exercised on November 23, 2015, which will result ... After giving effect to the issuance of such ...
(Date:11/24/2015)... ... November 24, 2015 , ... InSphero AG, the leading supplier of ... has promoted Melanie Aregger to serve as Chief Operating Officer. , Having ... management team and was promoted to Head of InSphero Diagnostics in 2014. ...
(Date:11/24/2015)... , Nov. 24, 2015 Capricor Therapeutics, ... focused on the discovery, development and commercialization of first-in-class ... Chief Executive Officer, is scheduled to present at the ... at 10:50 a.m. EST, at The Lotte New York ... . . --> ...
Breaking Biology Technology: