One of the most amazing things you can do with Wolfram Alpha is ask it what planes are overhead. If you’re on your phone, it will pull your location, then cross reference that with a database of flights, including their altitude, angle, and even their flight number and aircraft type. But in many ways, Stephen Wolfram’s latest search tool is more impressive. It’s designed to identify anything in a picture. You just upload a photo, and get a computer-generated guess just a few seconds later.
“It won’t always get it right.”
“It won’t always get it right, but most of the time I think it does remarkably well,” Wolfram writes. “And to me what’s particularly fascinating is that when it does get something wrong, the mistakes it makes mostly seem remarkably human.” In some brief testing, that’s a pretty fair assessment. I plugged in things like Yosemite National Park’s Half Dome and was told it was “elevation,” while a photo of a gecko was identified as a “night lizard.” Remarkably though, it identified a picture of a cow as “black angus,” and two cups of ice cream as “frozen yogurt.” Close enough.
How all this ascends beyond assaulting a website with photos of your last vacation or what’s in your kitchen, is tantalizing. Wolfram says he imagines the project could be useful if applied to large collections of photos to attempt to identify and categorize them. The technology can also be used by others to build image identification into their apps. Think about the visual recognition found within Google+’s photos, but in other photo apps and services.
The system was trained with cats, sloths, and Chewbacca
In order to train the system, Wolfram says it was fed “a few tens of millions” of images so that it could learn what was what. That “seemed very comparable to the number of distinct views of objects that humans get in their first couple of years of life,” he added. The system was also given tricky images like cats wearing spacesuits, sloths wearing party hats, and even Chewbacca — all things it failed at identifying correctly, but gracefully so:
Now it’s capable of recognizing about 10,000 common kinds of objects, though Wolfram notes that it still has difficulty recognizing specific people, art, and things that are not “real everyday objects.”
The new image project joins Google’s Goggles and Amazon’s Firefly as rapid identification tools, though is notably designed without the intent to try and sell you anything with what it finds. It also comes just a little while after Flickr’s new Magic View, as well as Microsoft’s research site that determines people’s genders and age based on photos. Unlike Microsoft though, Wolfram says it keeps a thumbnail version of the photo after you’ve uploaded it (so it can be shared with other people), and that it’s collecting the images to keep on training its system, so be mindful of what you send in.
Here’s how it did against a handful of different images:
I can kind of see it…
Which is pretty much just: