“We don’t have an algorithm for this”

[Image: Comet 67P, via ESA].

In the story of how European Space Agency researchers are scrambling to locate—and possibly move—the Philae probe, which they successfully landed on Comet 67P two days ago, there’s an interesting comment about computer vision and the perception of exotic landscapes.

[Image: Comet 67P, via New Scientist].

“We’re working our eyes off,” one of the scientists says to New Scientist, describing how they are personally and individually poring over photographs of the comet.

“It’s an entirely manual process,” New Scientist continues, “because the complex and bizarre landscape of comet 67P defies any kind of automated search. ‘We don’t have an algorithm for this,’ he says.”

We don’t have an algorithm for this.

[Image: The irregular terrain of Comet 67P, via ESA].

It would be interesting to develop a taxonomy of landscapes based on their recognizability to algorithms. This would tell you as much about how computers see the world as it would about the aesthetic assumptions—even the geological biases—of the people who programmed those computers.

Think, for example, of Adam Harvey’s work, asking When Is An Apple No Longer An Apple? That project explored the point at which machine-learning algorithms could no longer distinguish the iconic fruit from a jumble of colorful objects.

Or take Harvey’s more recent CV Dazzle experiment, which looked at how to prevent facial recognition software from identifying a face at all through the clever use of cosmetic camouflage.

However, in the case of Comet 67P and other extreme topographic environments, we would be looking at when a landscape is no longer a landscape, so to speak, at least in terms of the computer-vision algorithms programmed to analyze it.

[Image: Comet 67P, via ESA].

What other landscapes fall within this category—of spatial environments unrecognizable to machines—and what do those spaces reveal about the dimensional prejudices of the algorithm? Light and shadow; depth and range; foreground and background; geometry and complexity.

Bump Adam Harvey’s investigations up to the scale of a landscape, and a million potential design projects beckon. Learning from Comet 67P.

(Earlier on BLDGBLOG: The Comet as Landscape Art).

The Comet as Landscape Art

[Image: Photo courtesy ESA].

Intrigued by these images as an example of how the tradition of landscape representation has rapidly progressed—from the Romantics and the Hudson River School to Rosetta—I felt compelled to post a few photos of the craggy and glacial surface of Comet 67P/Churyumov–Gerasimenko, sent back to Earth yesterday from the European Space Agency’s Rosetta spacecraft.

The surface of the comet “is porous, with steep cliffs and house-sized boulders,” making it earth-like yet deeply treacherous, an irregular terrain to photograph and a dangerous place to land.

[Image: Photo courtesy ESA].

It is the notion of “land” here that is most interesting, however, as this is really just the imposition of a terrestrial metaphor onto a deeply alien body. Yet the comet is, in effect, literally a glacier: a malleable yet permanently frozen body of ice hurtling through space, occasionally exploding in comas and tails of vapor.

It is “an ancient landscape,” we read, “and yet one that looks strangely contemporary as the sun vaporizes ice, reworking the terrain like a child molding clay.”

Think Antarctica in a winter storm, not southern Utah—or Glacier National Park, not the Grand Canyon.

[Image: Photo courtesy ESA].

Along those lines, some of the most provocative writing on what it means to visually represent the frozen and hostile landscapes of the Antarctic is by writer William L. Fox, whose work offers some useful resonance here.

Fox has written, for example, about the technical and even neurological difficulties in accurately representing—let alone comprehending or simply navigating—Antarctic space and the vast forms that frame it.

Distant landscapes distorted by thermal discontinuities; white levels pushed to the absolute limit of film chemistry; impossible contours throwing off any attempt at depth perception; even the difficulty of distinguishing complicated mirages from actual landforms: these are all part of the challenge of creating images of an exotic landscape such as the Antarctic.

As Fox writes, it was even specifically the tradition of Dutch landscape painting, combined with the maritime practice of sketching coastal profiles, that first introduced the visual world of the Antarctic to western viewers: it was thus seen as an ominous, ice-clogged horizon of fog and low clouds looming always just slightly out of ship’s reach at the bottom of the world.

He calls this the genre of “representational exploration art.”

[Image: Photo by Stuart Klipper from his fantastic book, The Antarctic: From the Circle to the Pole, with a foreword by William L. Fox].

In one interesting passage in his book Terra Antarctica, he suggests that the south polar landscape is so extreme, it often resists natural analogy. As Fox describes it, the wind-carved boulders and isolated pillars and cliffs of ice are more like “artworks by Salvador Dalí and Henry Moore, evoking the spirit of surrealism with the former and modernist forms with the latter. The Antarctic is so extreme to our visual expectations that, once we attempt to move beyond measurement to describe it, analogies with other parts of nature fall short, and we resort to comparisons with cultural artifacts that push at the boundaries of our perceptions.”

These include “cultural artifacts such as sculpture and architecture, products more of the imagination than of nature.”

Consider, for instance, that comet 67P is widely known today as the “rubber-duck comet” due to its bifurcated structure, implying, as Fox suggests with the Antarctic, that no natural analogy seemed adequate for describing the comet’s geometry.

[Image: The gateway arches of the Antarctic; photo by Stuart Klipper from, The Antarctic: From the Circle to the Pole, foreword by William L. Fox].

But what are we to make of comet 67P now that we can see it as a physical landscape, not just an ephemeral optical phenomenon passing, at great distance, through the sky? When a blur becomes focused as terrain, what is the best way to describe it? What visual or textual traditions are the most useful or evocative—vedas and sutras or laboratory reports?

Put another way, is poetry as appropriate as a scientific survey in such a circumstance—should “we attempt to move beyond measurement to describe it,” in Fox’s words—and, if not, what new genres of exploration art might result from this spatial encounter?

I’m reminded here of poet Christian Bök’s wry remark on Twitter: “I am still amazed that poets insist on writing about their divorces, when robots are taking pictures of orange, ethane lakes on Titan…”

Now that humans are beginning to land semi-autonomous camera-ships on the frozen ice fields of passing comets, sending back the (off)world’s strangest landscape art—as if a direct line runs from, say, the pastoral landscapes of Claude Lorrain or the elemental weirdness of J.M.W. Turner to the literally extraterrestrial boulders and gullies depicted by Rosetta—how should our own descriptive traditions adapt? What, we might ask, is comet 67P’s role in art history?

[Image: Approaching 67P, via the ESA].