Computational Romanticism and the Dream Life of Driverless Cars

[Image by ScanLAB Projects for The New York Times Magazine].

Understanding how driverless cars see the world also means understanding how they mis-see things: the duplications, glitches, and scanning errors that, precisely because of their deviation from human perception, suggest new ways of interacting with and experiencing the built environment.

Stepping into or through the scanners of autonomous vehicles in order to look back at the world from their perspective is the premise of a short feature I’ve written for this weekend’s edition of The New York Times Magazine.

For a new series of urban images, Matt Shaw and Will Trossell of ScanLAB Projects tuned, tweaked, and augmented a LiDAR unit—one of the many tools used by self-driving vehicles to navigate—and turned it instead into something of an artistic device for experimentally representing urban space.

The resulting shots show the streets, bridges, and landmarks of London transformed through glitches into “a landscape of aging monuments and ornate buildings, but also one haunted by duplications and digital ghosts”:

The city’s double-­decker buses, scanned over and over again, become time-­stretched into featureless mega-­structures blocking whole streets at a time. Other buildings seem to repeat and stutter, a riot of Houses of Parliament jostling shoulder to shoulder with themselves in the distance. Workers setting out for a lunchtime stroll become spectral silhouettes popping up as aberrations on the edge of the image. Glass towers unravel into the sky like smoke. Trossell calls these “mad machine hallucinations,” as if he and Shaw had woken up some sort of Frankenstein’s monster asleep inside the automotive industry’s most advanced imaging technology.

Along the way I had the pleasure of speaking to Illah Nourbakhsh, a professor of robotics at Carnegie Mellon and the author of Robot Futures, a book I previously featured here on the blog back in 2013. Nourbakhsh is impressively adept at generating potential narrative scenarios—speculative accidents, we might call them—in which technology might fail or be compromised, and his take on the various perceptual risks or interpretive short-comings posed by autonomous vehicle technology was fascinating.

[Image by ScanLAB Projects for The New York Times Magazine].

Alas, only one example from our long conversation made it into the final article, but it is worth repeating. Nourbakhsh used “the metaphor of the perfect storm to describe an event so strange that no amount of programming or image-­recognition technology can be expected to understand it”:

Imagine someone wearing a T-­shirt with a STOP sign printed on it, he told me. “If they’re outside walking, and the sun is at just the right glare level, and there’s a mirrored truck stopped next to you, and the sun bounces off that truck and hits the guy so that you can’t see his face anymore—well, now your car just sees a stop sign. The chances of all that happening are diminishingly small—it’s very, very unlikely—but the problem is we will have millions of these cars. The very unlikely will happen all the time.”

The most interesting takeaway from this sort of scenario, however, is not that the technology is inherently flawed or limited, but that these momentary mirages and optical illusions are not, in fact, ephemeral: in a very straightforward, functional sense, they become a physical feature of the urban landscape because they exert spatial influences on the machines that (mis-)perceive them.

Nourbakhsh’s STOP sign might not “actually” be there—but it is actually there if it causes a self-driving car to stop.

Immaterial effects of machine vision become digitally material landmarks in the city, affecting traffic and influencing how machines safely operate. But, crucially, these are landmarks that remain invisible to human beings—and it is ScanLAB’s ultimate representational goal here to explore what it means to visualize them.

While, in the piece, I compare ScanLAB’s work to the heyday of European Romanticism—that ScanLAB are, in effect, documenting an encounter with sublime and inhuman landscapes that, here, are not remote mountain peaks but the engineered products of computation—writer Asher Kohn suggested on Twitter that, rather, it should be considered “Italian futurism made real,” with sweeping scenes of streets and buildings unraveling into space like digital smoke. It’s a great comparison, and worth developing at greater length.

For now, check out the full piece over at The New York Times Magazine: “The Dream Life of Driverless Cars.”

Composite Archaeology

[Image: A laser scan of the Pantheon, courtesy ScanLAB Projects and the BBC; view larger!].

ScanLAB Projects, focus of a long article on Wired last month, are back in the news with a BBC documentary exploring the infrastructure of ancient Rome.

The show “explores Roman infrastructure and ingenuity, all below ground level”:

We journeyed via the icy, crystal clear waters of subterranean aqueducts that feed the Trevi fountain and two thousand year old sewers which still function beneath the Roman Forum today, to decadent, labyrinthine catacombs. Our laser scans map these hidden treasures, revealing for the first time the complex network of tunnels, chambers and passageways without which Rome could not have survived as a city of a million people.

The results, as usual, are both breathtaking and bizarre.

[Image: Courtesy ScanLAB Projects and the BBC].

The surface of the city is scraped away, a kind of archaeological dermabrasion, to reveal sprawling networks of knotted masonry and old corridors spliced together in a translucent labyrinth less below than somehow in the city.

[Image: Courtesy ScanLAB Projects and the BBC].

One of the most interesting points made in Mary-Ann Ray’s excellent Pamphlet Architecture installment—1997’s Seven Partly Underground Rooms and Buildings for Water, Ice, and Midgets—is when she describes her use of composite photography as a way to experiment with new forms of archaeological documentation.

Indeed, the pamphlet itself is as much architecture as it is archaeology—perhaps even suggesting a new series of historical site documents someone should produce called Pamphlet Archaeology—looking at wells, baths, cisterns, and spherical refrigeration chambers, in various states of ruin.

All of these are representationally difficult spaces, Ray explains, either curving away from the viewer in a manner that is nearly impossible to photograph or presenting constrictions of perspective that make even wide-angle photographs inadequate.

[Image: Courtesy ScanLAB Projects and the BBC].

Ray writes that the spatial complexity of the buildings, quarries, basements, and other excavations that she explores are, in a sense, an entirely different kind of space: knotty, interconnected, unstable. “They were also spaces,” she writes, “which seemed to have the ability to ‘flip-flop’ in and out of multiple spatial or constructional readings.”

What appears to be near is revealed to be far; what seems far away is suddenly adjacent.

[Image: Courtesy ScanLAB Projects and the BBC; view larger!].

Ray uses the metaphor of a “hyper-camera” here in order to draw comparisons between her composite photography and what she calls “a kind of cubist multiple view,” one where “the frame might succumb to the taper of perspective into deep space, or it may counter it, or build it into something else altogether.”

“In these composite views,” she adds, “the photograph can record the enactment of space as one maneuvers or roams through it with the eye or body.”

While Ray’s photographic approach is technologically, materially, and even visually very different from the work of ScanLAB, the two projects share a great deal, conceptually and methodologically. In fact, if many of the above quotations were applied, instead, to the images seen in the present post, they would seem to be the appropriate descriptions.

[Image: In the ruined basements of architectural simultaneity; ScanLAB Projects and the BBC].

ScanLAB’s laser work seems to fulfill many of the promises of Ray’s composite photography, offering multiple, overlapping perspectives simultaneously whilst also eliminating the problem of the horizon or ground plane: you can thus look straight-on into the basement of an ancient structure without losing sight of the upper floors or chambers.

The city is split in two, made into an architectural section of itself that is then animated, made volumetric, turned into Ray’s “enactment of space as one maneuvers or roams through it with the eye or body.”

The show airs tonight on the BBC. Check out ScanLAB’s website for more info, and definitely consider picking up a copy of Mary-Ann Ray’s book; it remains one of my favorites and has actually become more, not less, topical since its original publication.

Stealth Objects and Scanning Mist

The London-based architectural group ScanLAB—founded by Matthew Shaw and William Trossell—has been doing some fascinating work with laser scanners.

Here are three of their recent projects.

1) Scanning Mist. Shaw and Trossell “thought it might be interesting to see if the scanner could detect smoke and mist. It did and here are the remarkable results!

[Images: From Scanning the Mist by ScanLAB].

In a way, I’m reminded of photographs by Alexey Titarenko.

2) Scanning an Artificial Weather System. For this project, ScanLAB wanted to “draw attention to the magical properties of weather events.” They thus installed a network of what they call “pressure vessels linked to an array of humidity tanks” in the middle of England’s Kielder Forest.

[Image: From Slow Becoming Delightful by ScanLAB].

These “humidity tanks” then, at certain atmospherically appropriate moments, dispersed a fine mist, deploying an artificial cloud or fog bank into the woods.

[Image: From Slow Becoming Delightful by ScanLAB].

Then, of course, Shaw and Trossell laser-scanned it.

3) Subverting Urban-Scanning Projects through “Stealth Objects.” The architectural potential of this final project blows me away. Basically, Shaw and Trossell have been looking at “the subversion of city scale 3D scanning in London.” As they explain it, “the project uses hypothetical devices which are installed across the city and which edit the way the city is scanned and recorded.”

Tools include the “stealth drill” which dissolves scan data in the surrounding area, creating voids and new openings in the scanned urban landscape, and “boundary miscommunication devices” which offset, relocate and invent spatial data such as paths, boundaries, tunnels and walls.

The spatial and counter-spatial possibilities of this are extraordinary. Imagine whole new classes of architectural ornament (ornament as digital camouflage that scans in precise and strange ways), entirely new kinds of building facades (augmented reality meets LiDAR), and, of course, the creation of a kind of shadow-architecture, invisible to the naked eye, that only pops up on laser scanners at various points around the city.

[Images: From Subverting the LiDAR Landscape by ScanLAB].

ScanLAB refers to this as “the deployment of flash architecture”—flash streets, flash statues, flash doors, instancing gates—like something from a short story by China Miéville. The narrative and/or cinematic possibilities of these “stealth objects” are seemingly limitless, let alone their architectural or ornamental use.

Imagine stealth statuary dotting the streetscape, for instance, or other anomalous spatial entities that become an accepted part of the urban fabric. They exist only as representational effects on the technologies through which we view the landscape—but they eventually become landmarks, nonetheless.

For now, Shaw and Trossell explain that they are experimenting with “speculative LiDAR blooms, blockages, holes and drains. These are the result of strategically deployed devices which offset, copy, paste, erase and tangle LiDAR data around them.”

[Images: From Subverting the LiDAR Landscape by ScanLAB].

Here is one such “stealth object,” pictured below, designed to be “undetected” by laser-scanning equipment.

Of course, it is not hard to imagine the military being interested in this research, creating stealth body armor, stealth ground vehicles, even stealth forward-operating bases, all of which would be geometrically invisible to radar and/or scanning equipment.

In fact, one could easily imagine a kind of weapon with no moving parts, consisting entirely of radar- and LiDAR-jamming geometries; you would thus simply plant this thing, like some sort of medieval totem pole, in the streets of Mogadishu—or ring hundreds of them in a necklace around Washington D.C.—thus precluding enemy attempts to visualize your movements.

[Images: A hypothetical “stealth object,” resistant to laser-scanning, by ScanLAB].

Briefly, ScanLAB’s “stealth object” reminds me of an idea bandied about by the U.S. Department of Energy, suggesting that future nuclear-waste entombment sites should be liberally peppered with misleading “radar reflectors” buried in the surface of the earth.

The D.O.E.’s “trihedral” objects would produce “distinctive anomalous magnetic and radar-reflective signatures” for anyone using ground-scanning equipment above. In other words, they would create deliberate false clues, leading potential future excavators to think that they were digging in the wrong place. They would “subvert” the scanning process.

In any case, read more at ScanLAB’s website.