Monsters and wonders inside our machines
Posted in: UncategorizedNicolas Nova left us a few days ago. I, and many many others, will miss his wit, his generosity, his intelligence. Here’s a review of the last book he sent me. I was planning to publish it on Monday, but it’s only now that i’ve accepted that he’ll never get to reading it….
Persistance du merveilleux : le petit peuple de nos machines, by anthropologist, design researcher and writer Nicolas Nova. Published by Premier Parallèle.
The relationships we have built with computers over the years have never been without ambiguities and tensions. Nicolas Nova further complicates the human-machine perspective by presenting us with some of its most enigmatic characters: trolls, ghosts, Trojan horses, Roko’s basilisk, Shoggoths, sprites, daemons, etc. We thought they belonged in the past, in folklore and in fantasy books. Not in the rational Western societies we live in. These creatures have left Mount Olympus and Scandinavian forests to enter our mobile phones and laptops, showing us that modernity can still coexist with the otherworldly. Nicolas Nova argues that what brings this disparate assembly of monsters together is the idea of the “merveilleux”, something that creates wonder and bewilderment, something that is supernatural and doesn’t pertain to the ordinary. The fact that we -users and programmers- rely on a vocabulary that dates back to distant times is a sign that the 21st-century Western human is not quite ready to let go of the irrational. Hence the persistence mentioned in the title of the book. This persistence lays not only in the terminology (the figures evoked are direct references to myths of the past) but also in the fact that, throughout history, humans have looked at legendary entities to help us make sense of technological innovations.
In the book, the anthropologist looks at the symbolic dimensions of this online menagerie but he also explores how the people who create, use, encounter or endure these creatures relate to them.
Nova classifies these creatures thematically, starting with well-intentioned, discreet entities. Like the daemons, a battalion of little assistants that go about their business without the need for users to intervene. Daemons are intermediaries between human and god, between the user and the device. Usually silent and unobtrusive, daemons can slow down machines and create problems if they have been poorly designed. And even the ones that were created with the best intentions, such as Clippy, Microsoft’s eager to please genie, could quickly become annoying.
Google Street View camera error in Senegal, via
The second category of digital entities is made of ghosts and revenants. They can take the form of a dead relative appearing on Google Street View Images recorded just before their passing. Or a conversation with a much-missed friend on an “AI death bot” app. Some users, however, report phenomena of possession and magic spell in their digital devices. They wonder if devices can be haunted, they talk about cursed computer, spectral presences in video games, “possessed” files, phone screens that scroll when untouched or suppressed folders that reappear on their dropbox. Some of these anecdotes become part of creepypastas, frightening legends and anecdotes that circulate online. Unlike the ghosts of the past, they are not really fearsome, but they give us a thrill and that is what matters about digital ghosts.
World Of Warcraft littered with the skeletons of players who’d succumbed to the corrupted blood disease incident in 2005. Photo: Blizzard, via
Far more alarming are the little nasties that form the third category: Trojan horses, bugs, worms, replicants, trolls and viruses can be discreet like daemons but their behaviour is malevolent. They lurk in the dark and, when they attack, they spreads hard and fast. Nova notes that terms like viruses, worms and bugs add an aura of natural sciences to the evil menagerie. The immunology vocabulary suggests new fears and reveals the vulnerabilities of the systems that these digital “germs” parasite.
Media attention and narratives participate in the construction of their folklore and make them look even more ferocious. The most concerning aspect of this cunning and shifty community is the people and organisations behind them. Lone wolves, military, paramilitary groups or activists who use them as means to influence, destabilise, cause harm and spread propaganda.
M’aiq The Liar, a NPC from The Elder Scrolls
The fourth group in the digital menagerie consists of interaction companions such as virtual assistants, sprites, non-playable characters (NPC.) Some of us develop strong feelings for them. Whereas people tend to develop rich, emotional relationships with NPCs, for example, they often distrust chatbots and other conversational agents. The former contribute to giving a sense of realism to a game and make it more engaging. The latter are supposed to behave like humans. They almost never do. They notoriously do not get nuances, for example. While many of us have cursed at chatbots, we tend to develop a sort of attachment to creatures we meet in video games.
The feelings we develop towards digital partners will continue to strengthen as technology evolves. Nova, however, is concerned because the private companies that develop most of these partners will be able to structure our personal relationships according to their own commercial and not entirely transparent logics: favouring some interactions, downplaying or hiding others or making them all more homogenous.
Loab, a legendary figure from the early days (2002) of Generative AI technique
Newsweek editors asked AI to generate images of itself as a “giant tentacled monster with many eyes destroying New York.”… AI Generated image by Newsweek via Midjourney
The fifth and final category is inhabited by the new monsters: the AI systems. Their presence in this collection of digital freaks reflects the alarm that the growing sophistication of those beings causes in both the programmers’ community and in the public.
They go from the Paperclip Maximiser, a thought experiment that participates in the AI mythology, to the Shoggoths, shape-shifting and rebellious creatures from the H.P.Lovecraft universe, that now symbolise the opacity of the linguistic models used by AI chatbots. In his bestiary of AI creatures, Bruce Sterling also mentions Roko’s Basilisk, a beast that can be traced back to classical antiquity. Its modern counterpart is a super AI that, in the future, would create a virtual reality simulation to torture anyone who knew of its potential existence but did not directly contribute to its advancement or development.
The various terms used to characterise AI and its potential dangers help us comprehend the invisible, the intangible and the atypical. They compensate for the lack of representation of AI and they explain its hallucinations and other unexpected behaviours. The super algorithms are monsters because their performances dwarf human cognitive processes and skills. Nova goes to fascinating lengths to explain the various understandings of the word “monster.” For example, he refers to Cicero who used the world monster to mean “ill omen”. The word is derived from the Latin monere. Monster thus meant a character that we should refrain from bringing about. Another of the many examples the author uses to comment on our understanding of AI is Frankenstein: Or, the Modern Prometheus. In the gothic novel, the real monster is the creator, not the creature. Similarly, with AI systems, it is fair to wonder: who/what is the monster exactly?
One of the feathered serpent heads that decorated the Temple of the Feathered Serpent in Teotihuacan. Photo: Jami Dwyer
For Nova, the persistence in the “merveilleux” fulfills different roles: it uses familiar characters to personify the modus operandi of programs and tools and makes them appear as acting entities; it helps us conceptualise hidden threats. This digital folklore also shows the asymmetrical relationship between the users and the designers/producers of technology: only one side has all the information, and the other is left to using magic and fables to fill in the blanks.
Among the many important points that Nova makes in the book, I’m going to highlight two that i found particularly interesting: all the terms used to describe these creatures -apart from “avatar”- originate in European culture. Where are the Jinns, tanukis, masalai, Quetzalcóatl, Cihuacóatl?, asks Nova.
It would be a pity if the book wasn’t translated into English. Its highly original look at technology, its countless historical anecdotes and engaging analyses should interest both computer experts and normies (like me).
Also by Nicolas Nova: A Bestiary of the Anthropocene. Hybrid plants, animals, minerals, fungi and other specimens.
Post a Comment