The same type of technology that allows Tesla cars to drive themselves is about to transform the way photographers organize and retrieve digital image files. It’s artificial intelligence (AI), and it’s about to become the next big thing.
Whereas traditional computing is based on a set of manmade rules, AI uses artificial neural networks that are modeled after the central nervous system and programmed to learn and adapt on the fly. An artificial neural network can be loosely thought of as a computer-based, simplified simulation of a human brain.
This type of “deep learning” is already at work in many technologies currently in use—from the aforementioned self-driving cars, to the speech recognition in personal digital assistants, to the facial recognition used in everything from social media to criminal investigation. It is facial recognition, in particular, that serves as the jumping-off point for considering the vast changes ahead for photographers and our image libraries.
Facial recognition isn’t exactly new, but only recently has it begun to approach human accuracy. Very soon, computers will become more accurate at identifying faces—and other image content—than real humans are.
“Google’s machine learning has allowed us to recall nearly any moment by typing a word,” says David Lieb, product lead for Google Photos. “We’re working toward building a system that will have a human-level understanding of photos and videos. Imagine how great that will be.”
The AI in Google Photos, Google’s cloud-based image management application, sprang from the company’s efforts to identify image content for the purpose of improving Google Image search results. For instance, Lieb says, you can type in “German Shepherd” and see photos from the Internet of German Shepherds. This content recognition is a real triumph. It will enable photographers to create digital images that are automatically tagged—perhaps even in-camera—with pertinent image information beyond where, when and how an image was made. Soon we’ll also know who and what are in the frame, and as better content identification leads to increasingly accurate contextual connections, perhaps even why.
“The number of photos we’re taking is dramatically increasing with mobile,” Lieb says, “and it’s to the point that we literally can’t manually organize our photos anymore. So we leverage computer vision and machine learning to have the app organize your photos for you. You don’t have to tag or label any of them, and you don’t need to laboriously create albums. When you want to find a particular shot, with a simple search you can instantly find any photo—whether it’s your dog, your daughter’s birthday party or your favorite beach in Santa Barbara.”
Consider, too, that unlike a human tasked with identifying and tagging image content, computers won’t suffer identification fatigue. A human may add a few, even a dozen or more tags to an image file. Now imagine a mechanized approach to identifying and prioritizing content tags to include even the most subtle details: clothing brand, eye color, emotional state…With such an explosion of data, unseen connections are bound to open up.
That’s why it’s not surprising that Apple announced built-in machine learning in the iPhone and iPad’s operating system, iOS 10. Apple’s built-in Photos app creates galleries and videos automatically from related images, and it offers similar object-recognition searching as Google Photos. It’s not long before Photos for MacOS has the same features, giving photographers a powerful tool on the desktop to analyze their vast image libraries.
The State Of The Art Today
As for AI in 2016, everybody’s doing it—and they’ve been at it for a while, from IBM’s original “Deep Blue,” the chess-playing computer from the 1990s, to Facebook’s new DeepFace technology that can identify and tag friends in a user’s uploaded photos. Apple Photos uses advanced AI to detect image content in an effort to improve searching across devices. Even Google Search is seeing the influence of AI; the company recently replaced its head of search with its head of AI. If AI improves Google Search results, imagine what it will do for your image library.
Industry analysts predict that major tech companies will each spend billions of dollars on AI over the next few years. Is Adobe is included? Likely yes, as the software giant is already rolling out its AI future. The 2015 release of Lightroom 6.0 included face recognition tools, and this summer’s Photoshop CC update included content recognition—what Adobe tends to refer to as “content awareness.” Along with Content Aware Cropping, face recognition has made its way into the Liquify filter as well. The software can automatically detect a mouth, for instance, so the user can drag a slider to increase or decrease a smile. Want to make someone’s eyes larger? There’s a slider for that, too, as well as one for shrinking large noses or thinning round faces. Consider a future where editing applications not only identify a face but also automatically retouch it based on personal preferences—of the photographer or the subject.
The Bright Future
Automatic content tagging may sound like something for amateurs, but in fact the consequences for professionals and our libraries are profound.
Let’s say you’ve imported a group of images from a recent travel assignment. Instead of simple “you were here” tags in the metadata, you’ll get “you were here, with these people, doing this stuff, for this reason, and it relates to other photographs in your catalog in ways including X, Y and Z.” Perhaps a new folder of images that include tags for the beach prompts your computer to generate galleries of other shoots you’ve done at other beaches, or other sessions with the same models, or even shots from entirely different sessions that also included active, energetic lifestyle models. With the computer automatically making connections, new creative possibilities unfold.
Content identification could be applied not just going forward with new imports but also looking backward as well, to every image already extant. Now instead of trying to remember which client a particular shot was for, all you need to know is that it contained puppies, or a picnic, or the beach, or what the venue was.
The implications for stock photographers and agencies are huge. That’s partly because many of AI’s benefits compound with scale. This may also aid photographers in data mining and trendspotting, for instance. which might help a photographer identify aesthetic trends from recent shoots, good or bad. As the machine identifies aesthetic trends from recent imports, studios and clients may begin to target particular lighting styles or color schemes as the neural network reveals correlations between certain aesthetic choices and “better” images—be that measured through social likes, client approvals or better sales.
What if Lightroom didn’t simply organize your photos? What if it also automatically and accurately selected which ones were best? There’s already an MIT algorithm for that.
Microsoft founder Bill Gates has called AI the “holy grail” of computing, though he also worries about its ramifications. Aside from a Terminator-style takeover, there are some very practical AI concerns as well. Increased unemployment (one Oxford study predicts as many as 47 percent of jobs could disappear), facilitated fraud, even autonomous weapons may be inevitable. These are not to be taken lightly, surely, and great minds such as Tesla’s Elon Musk and theoretical physicist Stephen Hawking have been vocal about treading carefully.
As it relates specifically to photography, AI brings with it a slew of privacy issues. Facebook, for instance, doesn’t use its “faceprints” for identifying and tagging pictures in Europe because privacy advocates there wanted to require explicit consent. (According to Facebook, its technology will never identify an individual in a picture if the uploader is not already the user’s friend. Strangers, therefore, cannot identify you simply by uploading your photo.) As computers get better at identifying who is doing what, where and when, societies will inevitably have to redefine the meaning of privacy.
Technology is already able to turn a two-dimensional smartphone image into a three-dimensional, immersive rendering. Computers can already produce high-resolution CGI images that sure look a lot like photographs. (IKEA says three quarters of its product photography is actually CGI.) Combine the two and wonder, for instance, if future generations of Google Image Searchers will, instead of receiving existing image results, see Google instantly generate a new, photorealistic CGI image custom-made for the user? It sounds far-fetched. But then again, not long ago so did a camera that could name who it was photographing.
At a practical level, perhaps the biggest issue surrounding AI for photographers is the loss of control—not something most serious shooters tend to give up easily. In a future where computers autonomously sort, label and categorize every photo, no picture could ever go missing. This would surely create opportunities for diligent photographers to monetize even the dustiest corners of their archives. And if it’s never a problem to find an image, do you have to worry about keeping them organized in the first place? Traditional file management best practices might simply disappear.
We are about to find out the answers to many of these questions. One thing is already clear: whether you’re meticulous about tagging your image files, or if you prefer a more spontaneous, disorganized approach, AI is about to overhaul your photo management system—whether you want it to or not.