The Body Digital

Chapter One: Hand

The Artificially Intelligent Hand

In 2018, artificial intelligence made a loud splash in the art world. A trio of French students, calling themselves Obvious, put a smeared, unfinished portrait up for auction at Christie’s. Titled Portrait of Edmond de Belamy, the murky image pictured a suited gentleman with a plain white collar. Rendered in three-quarter profile against an indistinct background, the image mimicked conventions from Renaissance and Baroque portraiture. At the corner of the canvas, a mathematical equation ostentatiously replaced the traditional artist signature. Global media trumpeted the achievement as a stunning first. Obvious oiled its promotional engine by claiming it to be the first ever work of art created by A.I. to go under the hammer. Neither claim turned out to be true: significant human labor went into its making, and a set of AI-generated images had gone to auction at San Francisco’s Gray Area in 2016. Offered at $10,000, the image went to an anonymous buyer for a jaw-dropping $432,000.

Obvious worked with an artificially intelligent system known as a generative adversarial network, or GAN. An algorithm composed in two parts, a GAN reproduces an agonistic relationship between artist and critic. The Generator creates new images based on a massive data set, and the Discriminator evaluates that image against a human-made image. Obvious trained their GAN on a data set of 15,000 portraits from the 14th to the 20th century. Ensconced in a gilded frame, and gorged on the Western canon, the image convincingly reproduced the aesthetic conventions of portraiture. By signing the image with a segment of the algorithm’s code, Obvious had, in a clever sleight of hand, cast an algorithm as a stand-in for the absent hand of the artist.

But the artist in question was not a hypothetical person. Obvious had appropriated their code, as well as the training set, from Robbie Barrat. Then a 19-year-old artist and programmer, Barratt had shared his Old Masters GAN on Github, the open-source sharing website, in 2014. An outcry from the small community of A.I. artists punctuated the auction. They not only emphasized Barrat’s erased contribution but also expressed disbelief that the amateurish image was representing their years of creative experimentation with GANs to the world at large. In their rush to crown the algorithm as the work’s author, Obvious obscured the central roles played by humans in the conception, coding, and curation that yielded the image. If this was a Turing test, then we willfully failed it. But why?

The equation at the traditional site of the artist’s signature offers some clues.

Signatures boast a special connection to the human hand. Historically, they have imprinted the heft of the human body upon an object, whether artwork or legal contract. They index the identity of the signer: they affirm that someone was really there. A hand that can sign is a hand that can write, and a writing hand is trained in one of the most significant technologies in human history. Writing embodies an ever-shifting partnership between hand, tool and mind as they meet world. As a technology, writing highlights how curious and intelligent our hands are. By displacing a human signature with an algorithm, Obvious culminates centuries of the human hand’s slow erasure by automation technologies.

Writing is most obviously identified by its material legacies: ink scrawled on notepaper, a book’s neatly typed rows, or text shimmying across a screen. It can be easy to forget that writing is an embodied activity. Children learn writing alongside hopscotch and jump rope. For my preschooler, these physical activities nourish the same body-mind connection that makes writing possible. When I first learned how to write English letters and Chinese characters, I repeatedly traced meaningless strokes that would coalesce into words, then sentences, then paragraphs, eventually notes, letters, essays, stories. Writing disciplines your hands as well as your mind; in return you gain access to new galaxies of expression and to the minds of others, for to write is also to read.

While much of writing’s profound impact lies in its massive capacity to store and transmit ideas, its ancestry in handmade marks makes it the twin of drawing. Graffiti artists remind us of this deeply physical connection. As they scale buildings, these renegades turn cities into writing surfaces. They leave tags as tall as they are, still texts that are the residues of human performances. These residues have left an imprint all over human history, evidence of how long our hands have been in conversation with the material world. Prehistoric cave paintings from Indonesia to Argentina, Australia to America, are replete with hand motifs. Some of these handprints simply cohabit with renditions of wild boar, deer, mastodons and other ancient fauna. In others, these handprints form the bodies of prehistoric menageries. Like graffiti, this rock art captures the hand as an organ reaching out to mark the world — and being remade in return.

***

I took cartoonist Lynda Barry’s “Writing the Unthinkable” workshop in the summer of 2016. During the weeklong workshop, Barry led her students deep into their memories through their own physical gestures of writing and drawing. As the week went on, the action of writing and drawing seemed to open a portal into my memory. I remembered wood grain under my feet and curlicues in wallpaper, the heat of a summer day searing my skin and the oily scent of a diner.

With incredible clarity, I recalled details, some moving, some mundane. These details became creative resources for the stories, first autobiographical, then semi-fictional, we told in timed bursts of writing, or drawing. Rehearsing these movements, in spirals, in scribbled words, in rapidly sketched images, as she urged us to be in our bodies, granted access to what Barry calls the “back of the mind.” Before the “front of the mind’s” leap to judgment, its preemptive cry of failure and not-good-enough, these movement practices would unleash a visceral surge of images, sensory impressions ripe for the picking.

Barry’s method, which she eagerly and generously shares in her books and workshops, is an expression of haptic creativity that for many people, digital technologies foreclose. In her telling, something about the experience of computing denies access to that most profound and most prosaic of creative states. She often describes the stalled journey of writing her second novel as a process that, at first, involved a lot of staring at a blank screen. When she decided to ditch the computer and write it with a paintbrush, it poured out of her with a life of its own. Barry’s work, and my experience with it, shows how much our hands mediate the encounters of mind and world. The hand is an organ that extends human consciousness into our environments. In this way, the hand might be understood as the original interface — or as Barry puts it, “the original digital device” — between human and world.

Paleoanthropologists, neuroscientists and philosophers have stressed the evolutionary symbiosis of hand and mind. The hand mediates the most complex interactions of the human brain and the realm of technology. At the same time, gestures have themselves been shaped by the dialogue between our selves, our tools and our environments. As our earliest principal technology for information storage and retrieval, writing embodies this dynamic.

Hands are smart. Hands are curious. Hands learn. Hands know things.

Surgeons, chefs, and musicians alike gain and exercise knowledge through their fingers. They intuit by touch, sense by weight and pressure, apply force and release, and know when to press, push, pull, cut, play. This is the “intelligent hand” at work. Imbued with tactile intelligence, hands accumulate experience. Practice embeds this knowledge deeper into their fingertips. These craftspeople, like all of us, practice a repertoire of learned gestures through which they discover and shape the world. Rather than a forceful imposition of ideas onto the environment, the gestures of craft are themselves conversational encounters with the material world and emblematic of what Richard Sennett calls “the evolutionary dialogue between the hand and the brain." Techniques, Sennett writes in his book about craftsmanship, emerge from deploying the hands as intelligent explorers that probe the world, sending feedback to the brain, and acting upon it, gaining expertise through trial and error. Yet it is, perhaps most surprisingly, that such craft practices are also where automation begins to erase the human hand.

Despite the crucial role hands have played in the development of new technologies — and our bodies with them — there have been numerous attempts to automate the human hand out of the equation. A number of ancestors to the modern computer were mechanical objects that reproduced, and eventually effaced, the labor of the human hand. Automata, proto-robots built to act as if under their own power but actually following a predetermined sequence of operations, have existed for over a millennia. Many of them are dedicated to mimicking the unique human performances of the hand. Although most surviving automata are from no earlier than the sixteenth century, tales and visual depictions of automata go as far back as Ancient Greece, spanning Europe and Asia. In the twelfth century, Muslim polymath and inventor Ismail al-Jazari served as chief engineer at the Artuqid palace in Mesopotamia, now known as Turkey. Under this remit, he built a plethora of marvels, including a robot that serves drinks and a floating boat with a water-operated four-piece band – a flautist, a harpist, and two drummers whose rhythms were programmable.

Many automata in the centuries hence have similarly sought to replicate the learned skills of the human hand. The mid-fifteenth century integration of the steel spring into mechanical design sparked the golden age of clockwork automata, begetting many counterfeits of life. The era witnessed the creation of marvelous clockwork wildlife such as an artificial duck that ate and defecated, as well as some truly remarkable simulations of human artistry and expertise. Artificial writing, as the machine double of an especially human skill, proved a particularly intriguing nut to crack. Writing machines sought to both reproduce and erase the human hand. The Writer, designed and built in the 1770s by Swiss-born watchmaker Pierre Jaquet-Droz, his son Henri Louis, and Jean -Frédéric Leschot, was one of these machines.

On its face, the automaton resembles a doll come to life. A petite, barefoot boy perches at a gleaming mahogany desk, holding a goose quill. A cloud of brown hair frames his cherubic face, in which crystalline blue eyes dart back and forth, following his own hand as he dips his quill in an ink well and draws it across a piece of paper. His artificial hand effortlessly controls the pressure of the quill against the sheet, modulating its stroke to achieve elegant, fluid cursive.
The boy’s red jacket opens at the back to reveal a complex clockwork mechanism comprised of 6,000 gleaming parts. Even more remarkable, unlike most automatons of his day, The Writer is programmable: blocks in his back can be shifted to spell any word and sentence of up to forty characters over four lines. Due to this ability, the Writer is sometimes characterized as an early computer.

The Writer hails from a family of three doll automata — still operational and on view at Musée d'Art et d'Histoire of Neuchâtel, in Switzerland — known as the Jaquet-Droz automata, all built between 1768 and 1774. The other two are The Musician, an organist who plays an actual miniature instrument and who also moves her body to play and to breathe, and The Draughtsman, a young child who can draw four different images, including a portrait of Louis XV, a royal couple, and a dog with “Mon toutou” (“my doggy”) written beside it. These robots are mechanized visions of three quintessentially human crafts that each enlist the intelligent hand: writing, drawing and music. Technological marvels that could recreate human performances, they forecast a future where not only the hand, but also its thinking faculties are outsourced to machines.

A few decades later, Swiss watchmaker Henri Maillardet built another writing robot, his Draughtsman-Writer. Maillardet’s automaton has not survived the intervening two hundred years entirely unscathed. Ruined in a fire, its clothes in tatters, the Draughtsman-Writer arrived in shambles at the Franklin Institute in Philadelphia as a donation in 1928. Once repaired by a staff machinist, rewound, and set in motion, the automaton’s florid penmanship produced embellished poems in cursive and drawings of a Chinese temple, a three-masted ship, and Cupids frolicking. With four drawings and three poems stored in its brass memory, it is believed to have the largest cam-based memory of any such machine of its kind. In fact, it is this very memory that testified to its own origins: after restoration, the automaton signed off at the end of a poem, “Ecrit par L’Automate de Maillardet” — “Written by the Automaton of Maillardet.” Storing writing and drawing performances as information, this automaton anticipates the future of computation as one that tames the intelligent hand.