Did Shakespeare and Dante Use AI Tools? Let’s See What Detectors Say

Over the past few years, there has been a great deal of discussion about texts written with the help of an LLM. It happens pretty much everywhere, in journalism, publishing, education and, perhaps most of all, on the web. Increasingly, we also see texts being submitted to one of the many AI detectors available online, with the result treated almost like a verdict on where that text came from.

For fun, I decided to try one of the best-known tools: GPTZero.

For anyone unfamiliar with it, I’ll try to explain very briefly how it works. Once a text is pasted into GPTZero, it is divided into tokens, small fragments that can correspond to words, parts of words or punctuation marks. The model processes these elements and produces a numerical representation of the characteristics of the writing, then tries to determine how closely the text resembles the human or artificial patterns it learned during training.

It should be made clear from the start that we are not dealing with a lie detector. GPTZero itself acknowledges that false positives and false negatives can occur. The system currently uses deep-learning models and provides classifications both for complete documents and individual sentences, distinguishing between human-written, AI-generated and mixed texts.

In my tests, some results were very much in line with what I expected to see. Others were rather more curious, not to say ridiculous.

Were Shakespeare and Dante the First to Use AI Tools?

The first experiment immediately produced a major surprise: Shakespeare was writing with artificial intelligence!

Well, yes, at least if we take the detector’s verdict literally. To get a reliable test sample, I used the famous “To be, or not to be” soliloquy from Act III, Scene 1 of Hamlet, in its original language.

The result does not leave much room for interpretation: 90% AI, 10% Mixed and 0% Human. GPTZero even declares itself highly confident that the text was generated by artificial intelligence.

Considering that William Shakespeare died in 1616, perhaps it is fair to wonder whether artificial intelligence is rather older than we thought.

The father of the Italian language, Florentine poet Dante Alighieri, fared rather better. Perhaps AI tools were a little less advanced in the fourteenth century, so he limited himself to using them only when absolutely necessary. Who knows.

I submitted a sample from the first canto of the Inferno to the detector, naturally in its original language, fourteenth-century Italian. Result: 85% Human, 15% Mixed and 0% AI.

So Dante apparently did not use AI to write the Divine Comedy, although that 15% Mixed might suggest he had a little help polishing a couple of tercets he was not entirely happy with.

Of course, Shakespeare and Dante are extreme cases. We are feeding a modern tool texts written centuries ago, using forms of language very far removed from ordinary contemporary prose. It would therefore be rather unfair to use these two experiments alone to declare AI detectors useless. Even so, it may instinctively feel strange that works this important can give the system so much trouble, but there we are.

For a more reliable test, I carried on with the experiment.

An Old Text of Mine Left in a Drawer

Some time ago, before I used any AI assistant to help with writing or speed up research, I wrote a piece. One of those articles that ends up sitting in a drawer because you are not quite sure what to do with it. Perhaps one day you will read it on these pages. Who knows.

But that is not the point.

I submitted that old draft to GPTZero and the verdict was emphatic: 100% Human.

The text was not particularly polished either. It needed another read, contained a few repetitions and probably a typo or two. I wondered whether that slightly “messy” draft quality might actually have contributed to the result, but I do not have an answer.

The really interesting part came with the translation.

Italian is my native language and, unfortunately, English has always been my Achilles’ heel. I use ChatGPT as a support tool when translating Retro-Gamers articles. So I asked it to translate that same article into English, preserving its form, content and meaning as much as possible, changing things only where a literal translation of an Italian expression would make little sense to an English-speaking reader.

Want to know the result? 100% Mixed, AI Polished.

And this time I would say the detector did a pretty good job. The original text was mine, but its English form had indeed been reworked by artificial intelligence. GPTZero itself uses the term “AI Polished” for originally human-written texts that have subsequently been improved or modified using AI.

The dilemma, if anything, is another one.

Does that mean the article is no longer the product of my own writing?

The Tony Warriner Interview

I ran another test using a recent interview published on Retro-Gamers with Tony Warriner, co-founder of Revolution Software.

I had personally written an introduction and conclusion for the article, while Tony’s answers were published in the form in which they had been sent to me.

GPTZero detected signs compatible with AI intervention even within some of his answers.

Naturally, that is nowhere near enough to accuse Tony of using an AI assistant. The idea alone makes me smile. Can you imagine a native English speaker of his calibre needing an AI tool simply to tell part of his own story? Come on.

Of course, I cannot know whether he used some kind of revision tool or not, and that is exactly the point: I cannot establish it on the basis of a detector’s verdict.

And What Happened to This Article?

Now we come to the little experiment I would like to try with you.

The final Italian version of this article was written by me, starting from a first draft that GPTZero classified as 100% Human.

After rereading, correcting and polishing it, I submitted it to the detector again. And here came another small surprise: GPTZero still considers the document entirely human, but this time gives it 99% Human, 1% Mixed and 0% AI.

A very good result, you might say, but what exactly is that 1% gained during the polishing process? Does a text that is a little too clean become suspicious?

The English version, on the other hand, was translated with the help of ChatGPT following the same principle as the previous experiment. The substance of the text has not been changed, but it has been adapted so that expressions and turns of phrase typical of Italian make sense to an English-speaking reader.

This time, however, I am not going to show you an image of the result for the English version, as I did with the Italian one. Try it yourselves, simply for the fun of playing along. After all, that is how this experiment should be approached, with as much light-heartedness as possible.

Copy the complete English text and submit it to GPTZero. Then, if you are a real doubting Thomas, go back to the Italian version, copy that one and repeat the test.

See for yourselves what happens.

Were you surprised? Or perhaps not? If you have been paying attention, I have already given you a few clues.

Can a Tool Determine with Certainty Who the Author Is?

The purpose of this article is not to discredit GPTZero. It would be far too easy to use Shakespeare and Dante for a laugh and dismiss the whole subject there.

AI detectors are certainly useful tools, and GPTZero continues to develop increasingly sophisticated models capable of distinguishing completely generated texts from texts of human origin that have subsequently been polished with AI.

The point I would like you to consider is something else.

Is the verdict of a detector enough to accuse someone of not having written a text and perhaps discredit their work without even continuing to read an article that might have been interesting or sparked a worthwhile discussion?

I do not think so. Let me explain why.

There can be an enormous amount of work behind any editorial project. Some people do that work seriously and others less so, just as in any other profession. Some may use AI tools to cut costs and maximise profits, others simply because they have very little time, and others because they find it difficult to express themselves but still have an idea. None of this can be dismissed simply by looking at the result produced by a tool.

Perhaps the problem is not so much the result the detector gives us, which, let us remember, should not be read as a conviction but as an indication.

It is what we decide to attribute to that result, and how we use it to judge the person on the other side.

So I return to the question I asked during this experiment, which began as a bit of fun but, more importantly, became something to think about:

If an author writes a piece and a tool merely helps translate it into another language, remove typos or reconstruct an awkward sentence to make it clearer, who should be credited as the author of that article?