Computational Romanticism and the Dream Life of Driverless Cars

[Image by ScanLAB Projects for The New York Times Magazine].

Understanding how driverless cars see the world also means understanding how they mis-see things: the duplications, glitches, and scanning errors that, precisely because of their deviation from human perception, suggest new ways of interacting with and experiencing the built environment.

Stepping into or through the scanners of autonomous vehicles in order to look back at the world from their perspective is the premise of a short feature I’ve written for this weekend’s edition of The New York Times Magazine.

For a new series of urban images, Matt Shaw and Will Trossell of ScanLAB Projects tuned, tweaked, and augmented a LiDAR unit—one of the many tools used by self-driving vehicles to navigate—and turned it instead into something of an artistic device for experimentally representing urban space.

The resulting shots show the streets, bridges, and landmarks of London transformed through glitches into “a landscape of aging monuments and ornate buildings, but also one haunted by duplications and digital ghosts”:

The city’s double-­decker buses, scanned over and over again, become time-­stretched into featureless mega-­structures blocking whole streets at a time. Other buildings seem to repeat and stutter, a riot of Houses of Parliament jostling shoulder to shoulder with themselves in the distance. Workers setting out for a lunchtime stroll become spectral silhouettes popping up as aberrations on the edge of the image. Glass towers unravel into the sky like smoke. Trossell calls these “mad machine hallucinations,” as if he and Shaw had woken up some sort of Frankenstein’s monster asleep inside the automotive industry’s most advanced imaging technology.

Along the way I had the pleasure of speaking to Illah Nourbakhsh, a professor of robotics at Carnegie Mellon and the author of Robot Futures, a book I previously featured here on the blog back in 2013. Nourbakhsh is impressively adept at generating potential narrative scenarios—speculative accidents, we might call them—in which technology might fail or be compromised, and his take on the various perceptual risks or interpretive short-comings posed by autonomous vehicle technology was fascinating.

[Image by ScanLAB Projects for The New York Times Magazine].

Alas, only one example from our long conversation made it into the final article, but it is worth repeating. Nourbakhsh used “the metaphor of the perfect storm to describe an event so strange that no amount of programming or image-­recognition technology can be expected to understand it”:

Imagine someone wearing a T-­shirt with a STOP sign printed on it, he told me. “If they’re outside walking, and the sun is at just the right glare level, and there’s a mirrored truck stopped next to you, and the sun bounces off that truck and hits the guy so that you can’t see his face anymore—well, now your car just sees a stop sign. The chances of all that happening are diminishingly small—it’s very, very unlikely—but the problem is we will have millions of these cars. The very unlikely will happen all the time.”

The most interesting takeaway from this sort of scenario, however, is not that the technology is inherently flawed or limited, but that these momentary mirages and optical illusions are not, in fact, ephemeral: in a very straightforward, functional sense, they become a physical feature of the urban landscape because they exert spatial influences on the machines that (mis-)perceive them.

Nourbakhsh’s STOP sign might not “actually” be there—but it is actually there if it causes a self-driving car to stop.

Immaterial effects of machine vision become digitally material landmarks in the city, affecting traffic and influencing how machines safely operate. But, crucially, these are landmarks that remain invisible to human beings—and it is ScanLAB’s ultimate representational goal here to explore what it means to visualize them.

While, in the piece, I compare ScanLAB’s work to the heyday of European Romanticism—that ScanLAB are, in effect, documenting an encounter with sublime and inhuman landscapes that, here, are not remote mountain peaks but the engineered products of computation—writer Asher Kohn suggested on Twitter that, rather, it should be considered “Italian futurism made real,” with sweeping scenes of streets and buildings unraveling into space like digital smoke. It’s a great comparison, and worth developing at greater length.

For now, check out the full piece over at The New York Times Magazine: “The Dream Life of Driverless Cars.”

Romanticism of the Scanning Error

[Image: ScanLAB Projects].

(A different version of this post previously appeared on Gizmodo).

Matthew Shaw and William Trossell, the London-based duo known as ScanLAB Projects, continue to push the envelope of laser-scanning technology, producing visually stunning and conceptually intricate work that falls somewhere between art and practical surveying.

Their work also bears an unexpected yet increasingly pronounced political dimension, as they have scanned concentration camp sites, designed insurgent objects for thwarting police laser scanners, and even point-mapped melting ice floes in the Arctic as part of a larger study of climate change. The results are astonishingly, almost hypnotically detailed, as in this cinematic fly-through of an outdoor festival, where we pass through tent walls and very nearly see recognizable expressions on participants’ faces. It’s as if the future of the motion picture might really be narrative holograms.

Last week, Shaw and Trossell premiered a new project at London’s Surface Gallery, exploring where laser scanners glitch, skip, artifact, and scatter. Called Noise: Error in the Void, the show utilizes scanning data taken from two locations in Berlin, but—as the show’s title implies—it actually foregrounds all the errors, where the equipment went wrong: a world of “mistaken measurements, confused surfaces and misplaced three-dimensional reflections.”

The tics and hiccups of a scanner gone off the mark thus result in these oddly beautiful, almost Romantic depictions of the world, like some lunatic, lo-fi cosmology filtered through machines.

Frozen datascapes appear like digital mist settling down over empty fields—or perhaps they’re parking lots—a virtual Antarctica appearing in the middle of the city.

[Image: ScanLAB Projects].

Huge domes of white light burn like spherical flames above a central point that remains both mysterious and unidentified, resembling the halos of nuclear explosions or the birth of stars.

[Image: ScanLAB Projects].

Spectacular bursts of color then suggest the presence of some new stratosphere, where black airplanes roam the edge of space and clouds are nothing but processing errors in a blurred celestial rendering. Perhaps we could call it expressionist scanning.

[Image: ScanLAB Projects].

In Shaw’s and Trossell’s own words, “Using terrestrial LIDAR technology it is now possible to capture the world in three dimensions. This technology can create near perfect digital 3D replicas of buildings, landscapes, objects and events. But these digital replicas are always an illusion of perfection. Noise: Error in the Void explores the inherent mistakes made by modern technologies of vision. Here we see the unedited view of the world as seen through the eyes of the LIDAR machine. Reality is shrouded in a cloud of mistaken measurements, confused surfaces and misplaced three-dimensional reflections.”

A short film—more like a dark ambient music video—shows some of the images in action.

In all honesty, many of the images are colored in a way that looks a bit more like a Pink Floyd laser show than the almost melancholy landscapes I like so much above, and I even made a few of these greyscale to see if, stripped of color, they could still repeat the lonely, wanderer-above-a-sea-of-fog feeling that the other images have, the benthic void of miscalculated data that nonetheless results in new worlds. But then I figured I shouldn’t mess with ScanLAB’s work and I left them as is.

[Images: ScanLAB Projects].

But, even here, blinded by the colors of a rave, throbbing architectural shapes rotate and spin, as if parts of London are stuttering in and out of sync with themselves, a whole city rumbling through a shockwave of digital reverb, blinking gyroscopically out of control.

[Images: ScanLAB Projects].

If you’re lucky enough to be in London in the next few weeks, check out their exhibition at Surface Gallery—and, even better, if you’re an architecture student, you can actually take a class with these guys. Check out their teaching work here.

(Read an earlier version of this post at Gizmodo).