Chatbots and visual art

2023 marked the breakthrough, at least for the general public, of ‘generative artificial intelligence’, which means that artificial intelligence is now used to create new content from scratch. A generative text model (ChatGPT, for example) is able to have a seemingly intelligent conversation with human users, and will provide qualitative text-based services such as translation, synthesis of existing text, rewriting, and so forth. Crucially, the technology is now also able to create new text – in any style – having ‘learnt’ to write from the stupendous amounts of online text on which it has been trained.

Using the same technology, but on a more advanced scale, image generators such as DALL-E have been trained on an endless array of text-tagged visual sources to be able to create new images based on a prompt provided by human users. These platforms too, saw spectacular advances in 2023 alone, which can only inspire awe – mixed in with fear – at where we will be in 2025. Video decidedly is the next frontier, and the omnipresence of deepfake images and videos is but a tip of the iceberg in terms of the disruptive power that the technology can potentially have on society.

Whether you are for or against the technology, it looks like it will transform lower-level creative functions such as translation, marketing, copy-editing and so forth in a manner similar to the way that industrialisation, automation and computerisation transformed the production process in the eighteenth, nineteenth and twentieth centuries. At this moment, I am unsure regarding my own position in the debate. Others are better placed to shed light on issues related to copyright, technological improvement, and ethical implications. I can, however, add the perspective of an academic active in medieval studies. What could be the benefits of engaging with this technology in researching and teaching the Middle Ages?

AI and Medieval Studies

As far as the text-based platforms go, there seem to be more upsides than downsides. For example, for medievalists who are studying late medieval Burgundian manuscripts but need to brush up on their medieval French, ChatGPT will provide relatively accurate translations of Old French and Middle French source text to modern English (or indeed any other common language). The source text does need to be machine-readable, but advances in ‘Handwritten Text Recognition’ are quickly leading to a future in which machines can ‘translate’ straight from (digitised) manuscript sources. Translating Middle Dutch remains more difficult, but this disparity is a perfect reflection of the amount of material that is online of both medieval corpora – the aptness of ChatGPT at French material simply shows that it has seen more of it.

Things become altogether more complex when you experiment with engaging ChatGPT as a research assistant. It will produce relatively accurate and well-written prose on general subjects or well-known authors that feature prominently on Wikipedia and other such openly accessible platforms. But a more specific subject – I tried “manuscript illumination in the Tournai region around 1400” – will wake up another side of the chatbot. It will then provide information that looks very convincing but that turns out to be completely irrelevant. When asked to clarify, it will vehemently insist on its mistakes, or it will apologise while making new ones without blinking an eye. The perceived arrogance of the computerised conversation partner has been pointed to elsewhere, and is a problem that still does not seem to have been solved. And the image generators seem to be going down the same street...

Generating miniatures

A few months ago, I asked Microsoft Bing’s image generator (which runs on DALL-E) to draw images of medieval manuscript miniatures in which a train enters the frame. This yielded a number of relatively credible-looking images, which gained some traction on Twitter/X. The way in which these images appealed to people across the world made me wonder about the potentially disruptive power that AI-generated images such as these might have on common perceptions of the Middle Ages.

Of course, no one will claim that trains were invented in the fourteenth century based on these miniatures, but what if the contrast is less marked? Now that the DALL-E 3 image generator has been integrated into Microsoft’s all-round AI assistant, ‘Copilot’ (which allows for a conversation in which you can tweak an image produced and discuss any references or visual sources of inspiration) I decided to put a case to the test.

Reversing gender roles

On the lookout for out-of-place items and themes that were less marked than ‘trains in medieval settings’, I asked ‘Copilot’ the following question: ‘Draw a medieval manuscript page showing two manuscript miniatures. On the left, a man doing laundry. On the right, a woman looking at the stars through a telescope.’ I aimed to produce an image that not only disrupted what we know of medieval gender roles, but also introduced an ‘un-medieval’ piece of technology, namely the telescope (invented in the seventeenth century).

Interestingly, the chatbot lectured me on medieval manuscripts while sternly pointing out that telescopes were not available in the Middle Ages. This did not keep it from producing images that could pass for medieval to the untrained eye, as you can see as you navigate through the carousel of images below.

What struck me here was the parchment support of the images, showing signs of wear, along with the visible fold between two pages – details that added to its seeming verisimilitude. The text, however, is gibberish and the miniatures have not always been positioned correctly within the lay-out, with text often running into the image.

DALL-E’s dirty laundry

When I told the chatbot I was pleased with the images, it gleamed with pride. So far so good. The atmosphere quickly turned when I started querying its sources of inspiration, especially for the laundry images. Its first reference was to three manuscripts that allegedly showed images of women doing laundry. Upon closer inspection, these manuscripts did not contain any images of laundry scenes, did not contain any images at all, or – far worse – did not even exist. After pointing out the latter, the chatbot kept insisting, churning out references to fictitious manuscript signatures in non-existent libraries (have you ever heard of the ‘Biblioteca dell’ Arsenale’ in Venice? No one has, but it sounds convincing enough…).

The chatbot apparently was not able, or at least reluctant, to refer to any relevant visual comparison for the images it drew of laundry scenes. I decided to steer the conversation to the general topic of its training, asking the bot whether it had access to medieval manuscripts that have been digitised and made available through the IIIF protocol. This question made it close ranks altogether. It dished up some general information on IIIF, and only reluctantly admitted that it used digitised manuscripts as a source of inspiration for its newly created images.

History is a lie!

The images generated by DALL-E 3 and its successors will become ever more convincing. Notably in the few months that have passed between my medieval trains and the laundrymen, the visual appearance of the miniatures and what surrounds them has improved considerably. Combined with the chatbot’s capacity to produce fictitious source references that seem to be convincing at a first glance, there is a danger that AI images which are being circulated online may be used to support or even generate misconceptions about the Middle Ages.

Imagine a fourteenth-century looking image of an operating printing press, with an obscure reference to a library somewhere in Russia. I have been on social media long enough to know that some groups will embrace (or start generating) images such as these in support of a narrative stressing that what ‘they’ tell us about history is a big lie. The increasing capacity of AI to produce highly credible-looking images in a medieval style, including relatively convincing references, calls for a new type of source criticism.

AI lessons for medievalists

First of all, institutions that are holding the medieval sources are confronted with a paradox. In their attempt to provide an ever-better service to their users, they have been digitising their holdings at great speeds, and making them available online. The only reason why I was able to point out the mistakes of the chatbot, is because the manuscripts it referred to were (if they existed at all) available online. I embrace institutional efforts to open up their collections, but image generation models are greedily on the lookout for new training materials. As is happening for text, we need to have a critical conversation about the open availability of visual heritage sources such as manuscripts and early printed books, and how this access may be exploited.

As scholars, we need to provide precise references to images that we circulate online, for example on social media. Better still would be to circulate the entire page of a manuscript along with its reference, as the image generator so far has not been able to produce credible written text in a medieval book hand (in due time, it will; note the header of the image above, which seems to read "printing press"). As teachers, we need be aware of the possibilities and train our students in questioning the information that ChatGPT, DALL-E3 and their successors provide.

Students will need to develop an improved orientation on (online) information in library catalogues, reference works, and other relevant metadata platforms, so as to be on guard against false information. An experiment such as the one described here, provides ample food for thought for a class discussion, and can be a starting point for an exercise in which students judge and verify the information that the model provides.

Scholars and students alike share a mission to educate the general public on false conceptions of the Middle Ages, and to call out any AI-generated image that contributes to misconstrued ideas of the past. In the meantime, let us not hesitate to use this new technology as a force of good, while being on guard for its dark side.

All images accompanying this post were generated by AI. You can try creating your own medieval miniature using Microsoft Copilot. This chatbot combines the DALL-E 3 image generator with the text-based conversation style of ChatGPT. Let us know what you create by tagging #MedievalAI!

© Bram Caers and Leiden Medievalists Blog, 2024. Unauthorised use and/or duplication of this material without express and written permission from this site’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Bram Caers and Leiden Medievalists Blog with appropriate and specific direction to the original content.