In the immortal words of Internet culture: "This looks shopped. I can tell from some of the pixels and from seeing quite a few shops in my time." This is the default, cheeky response to any questionable photo posted online. Image detectives zoom, enhance, and scrutinize images to look for out of place pixels or other signs of Photoshop shenanigans. Sometimes that sleuthing pays off and debunks faked images. Other times, people spend years (or decades) looking at a famous photo like the Apollo moon landing, and sometimes "proof" of a 'shop isn't exactly, well, convincing.
Enter a project from the University of California, Berkeley, which aims to bring computer intelligence to bear on questionable photos. First the researchers set up the problem: "Recent advances in computational photography, computer vision, and computer graphics allow for the creation of visually compelling photographic fakes," begins the research paper's introduction. The resulting undermining of trust in photographs impacts law enforcement, national security, the media, advertising, e-commerce, and more."
Then, the solution: "a geometric technique to detect physically inconsistent arrangements of shadows in an image. This technique combines multiple constraints from cast and attached shadows to constrain the projected location of a point light source. The consistency of the shadows is posed as a linear programming problem. A feasible solution indicates that the collection of shadows is physically plausible, while a failure to ﬁnd a solution provides evidence of photo tampering."
In simpler terms, the right math can be used to analyze the shadows in photographs and determine if they spring properly from the scenery. It's difficult to tell how well the system would work for any and all images thrown at it. In the methodology section, the paper states that "we assume a single distant or local light source and place no assumptions on the objects being illuminated or the surfaces onto which shadows are cast. We then frame the problem of determining if all shadows are consistent as a linear programming problem."
Photos with multiple light sources would, understandably, be tougher to analyze, though the paper points out that photos illuminated with a flash, and those taken in natural sunlight, will work just fine. The algorithm was also able to identify a fake in the arrangement of photos below. Can you spot the shop?