[Image: “Two images of the same room, one reconstructed from video footage of a bag of chips within the room (top) and the other photographed directly (bottom),” as described by Scientific American. Images courtesy Jeong Joon Park.]
“Researchers have now found that by filming a brief video clip of a shiny item, they can use the light flashing off it to construct a rough picture of the room around it,” Scientific American reports. “The results are surprisingly accurate, whether the reflections come from a bowl, a cylinder or a crinkly bag of potato chips.”
It comes down to mathematically modeling “what a known object will look like—how light will reflect off it—when it is placed in new surroundings,” such that you can then reconstruct the proper orientation of what it reflects.
There’s a lot more in the original article, but what immediately struck me about this was how this technology could be used for crime or espionage, both.
You send an unsuspecting group of school kids into a target building, carrying highly reflective silver balloons, or you wear a slyly reflective and precisely designed item of clothing into a business meeting: in both cases, a photographer on a roof across the street or hidden in a park nearby snaps away through a telephoto lens. The reflections spilling off in all directions are like a 360º spherical photograph of the building interior—the art on the walls, the position of furniture. The location of a safe.
Think of the Japanese pop star who was tracked by a stalker after he deduced her location by analyzing the reflection in her eye in a selfie. Every mirrored surface becomes a security leak—“Las Meninas” as burglary tool.
[Image: “Las Meninas” (1656) by Diego Velázquez; if my reference to this painting makes absolutely no sense in the present context, it’s because I’m being pretentious and indirectly referring to Michel Foucault’s The Order of Things, where he discusses the painting’s use of internal reflection.]
Of course, you may also recall that sounds can be reconstructed from the vibrations of distant objects: “Researchers at MIT, Microsoft, and Adobe have developed an algorithm that can reconstruct an audio signal by analyzing minute vibrations of objects depicted in video. In one set of experiments, they were able to recover intelligible speech from the vibrations of a potato-chip bag photographed from 15 feet away through soundproof glass… In other experiments, they extracted useful audio signals from videos of aluminum foil, the surface of a glass of water, and even the leaves of a potted plant.”
It’s worth noting here how potato chip bags pop up in each example. Ocean’s 14 will open with a surreptitious potato chip delivery…
In any case, political dissidents, high-value corporate CEOs, and adversarial diplomatic attachés will never be safe again. Just a brief reflection from a cigarette lighter or a piece of silverware, just a tiny ripple of sound across the leaves of an exotic orchid in the center of a dinner table, and someone across the city with a telescope has your bank passcode, the location of your home safe, and a complete 3D map of your building interior, even down to where your security guards are sitting.
[This is only somewhat related, but recall that an engineer at Carnegie Mellon has developed “a long-range iris scanner that can identify someone as they glance at their rear-view mirror” in a moving vehicle, Rob Meyer reported for The Atlantic back in 2015.]