Two Thing Tuesday: The Archive and Watts’ Blindsight

I’ve continued to think about AI over the last few months and especially the role of generative AI and large language models. Here are two slightly unhinged thoughts about it as a “Two Thing Tuesday”:

Thing the First

Historians are fascinated by archives and they often observe that historical discoveries (e.g. opportunities for knowledge making) occur not because the well-organized structure of the archive, but despite it. Accidental discoveries — like those described by Arlette Farge, Robert Darnton, and Carlo Ginzburg — shape our encounters with archival collections. These discoveries come about not because the careful organization of the archivists anticipate the questions of historians, but because historians can make distinct connections on their own that often belie the structure of the archive itself. 

As we become more and more familiar with the generative AI, we can’t help but notice the sometimes random associations of texts and ideas. At their worst, they manifest as hallucinations where names, ideas, places, and arguments belch forth as hopelessly garbled references or descriptions. Recently certain kinds of scholars have reveled in showing off historically and geographically inaccurate maps on social media demonstrating that they are smarter than generative AI because they know where Bremen is. What’s more interesting, of course, is that these maps represent wildly speculative geographies not generated from whole cloth, but algorithmically assembled from the massive archive underpinning the large language model. In other words, these maps — like hallucinated citations — are wrong because the algorithm has engaged incorrectly with the archive. 

This misengagement with the archive is familiar to most of us as historians (or archaeologists). It’s not uncommon for us to encounter texts or artifacts that lead us to tell a story, but as our assemblage of texts or material expands, we understand that story to be incomplete or even inaccurate. Archaeological notebooks are fully of what we might call hallucinations that dissipate as we read. Our alarm at the propensity of generative AIs to hallucinate is not so much because these are wrong, but because they are uncanny: they look right. Of course, this is always the risk of any engagement with any archive. 

What is more remarkable to me is not that generative AIs can produce bizarre hallucinations (I’ve worked too much with archaeological notebooks to be surprised when something looks right but, in fact, isn’t), but that it can genuinely find connections across the archive that we might not expect. This is largely because the generative AI can constantly adapts the structure of its archive. On the one hand, this is frustrating because it seems to undermine the very idea of an archive as stable and committed to preservation or at least consistency. On the other hand, this is brilliant because we can compel the archival organization to adapt to our inquiries, at least to a certain extent.

Noah Kaye, a friend and frequent commentator on my blog, noted that AIs are at their most spectacular when they find connections that surprise us even in the speculative geography or the hallucinated citations. These products reveal the contingency of the archive itself and the allusive and elusive nature of its shifting organization. 

Thing the Second

I read Peter Watts’ novel Blindsight(2006) this weekend on the recommendation of a colleague. Without spoiling it, it is a first-contact novel where a ship full of modified humans encounter an alien form that calls into question their notions of sentience and even life itself. The crew includes an individual with a partitioned mind that allows for multiple individuals, an individual whose capacities for vision, hearing, and dexterity have been massively expanded through prosthetic forms, a military officer, an observer whose job is to synthesize the experiences and report them back to Earth, and … a vampire. The ship, the Theseus, itself is controlled by an AI who communicates almost exclusively with the reclusive vampire who is the nominal (?) commander of the mission.

Without getting too far into the weeds (and most of the book has a distinctly weedy quality to it), the encounter with alien forms on their ship, the Rorschach, leads the crew of the Theseus to explore how their own forms of sentience shape their ability to apprehend and ultimately engage with the world. Whether the aliens ever qualify as sentient by human standard remains unclear. They certain learn from their environment and by the end of the novel our perform humans at certain things. The critique offered here is not subtle and remains timely. 

Blindsight anticipated the contemporary discussion of sentience and consciousness in relation to various forms of generative and agential AI. More than that, it reminds us that as we try to define what AI is, we invariably also define ourselves. Part of the genius of Blindsight is that the crew all negotiates various situations that in our contemporary we might consider “disabilities” (and here I’m drawing on my conversations with my colleague). Historically these disabilities — from multiple personalities to the reliance on prosthetics to the vampire’s vulnerability to right angles — have defined individuals as “less than” fully human. Peter Watts’s Blindsight suggests alternately that consciousness is what makes us human (and it was not compromised by disabilities) but also that our reliance on consciousness is a rather less efficient way to live. By the end of the novel, it’s become pure metaphor: in the final scenes, the ships AI devises a plan to dispatch the lobotomized narrator to Earth to tell the tragic story. It reduces the intriguing ambiguity of the narrative to a biological metaphor. Our “blindsight” (that is the ability of the unconscious brainstem to process even complex inputs like vision) protects our consciousness. The ship’s AI protects its more fragile, sentimental, and self-destructive occupants. There are shades of Iain Banks here.