Ahead of Channel 4 screening the second series of its drama about synthetic people, Humans, I thought it timely to say a few words about writing with artificial intelligence, something that regular visitors here will know I practice.
What I find curious about artificial intelligence is how little momentum there is to its literary application. It’s fair to say that AI is recognised for its ubiquity across many spheres of life. Yet the main inroads into writing have been mimetic and mechanical:
- AI is being used to write large numbers of sports articles.
- One author has written over 200,000 books using AI.
- AI can now imitate handwriting.
The most literary development is NaNoGenMo. Many people are aware of NaNoWriMo, the competition for the best novel written over the month of November. Few people, however, are aware of NaNoGenMo, a version of the same competition for novels generated by code. There are some exemplary examples from this.
There is a helpful post here on what is, IMHO, the biggest problem of using AI at length: structure. It is seriously, seriously difficult to get a computer program to output intelligible text – beyond paragraph length – that has any shape or direction. This isn’t necessarily a problem when you are writing poetry or, even as I’ve done, short stories. But my efforts this year have been with working on a coherent book-length piece.
What helps is that I don’t engineer text in the usual way that engineers do. I edit the output. This means that I’m looking for content to respond to. The NaNoGenMo competition reflects an orthodoxy that’s based on pushing a button and a book popping out. The results often have little to offer. They can work well locally, in places, but tend to jump around arbitrarily. The challenge is in sustaining a theme from chapter to chapter.
It would take a book to describe the challenges within this challenge. Here are some main points:
- Each part of a generated text needs to be both distinct from the other parts while at the same time related enough to them to form a whole. The natural tendency to overcome in any generated text is its homogeneity.
- You need a strategy for proper nouns. The subjects and objects need to be consistent. There are people, places, things, genders, tenses etc. The obvious approach to this is to use tables of words that are grammatically constructed from ground up. That’s not quite what I do, but it leads to the next point …
- You need to bind the minutiae to topics. You can’t do this without attending to vocabularies. These need to stick to overarching topics while at the same time not so much so that those topics don’t develop / evolve.
This year I developed code to tackle the above through the way that meanings relate to one another in a given text. I got the idea after attempting to read some Wittgenstein. I must have got no further than a dozen pages in before I started browsing Wikipedia, as you do. This led me to topic modelling. Topic modelling didn’t quite offer what I needed. But in trying to understand it, I found a way to map text into a neural network so that meaning could be inferred from strongly related words and phrases. Put simply, my code now has a (very mild) understanding of semantics.
This represents a huge leap. The code finds thematic connections that are beyond human understanding. It’s ‘thinking’ is mild, but crucially, it’s mathematical – it comes up with what I can only describe as ‘inhumanly human’ prose. It’s a lot of fun to work with. Here’s a snippet from the draft of Being Italo Calvino: [Note: the book has evolved into Being Donald Trump since this blog post – see here for more details]
For the point of honour is not honour, in ordinary language. I was fully convinced (yes, a sense of reality, in spite of all my romanticism!) that all sorts of things might have happened. Only fancy, this was two years after Calvino’s insult to me, and my challenge would be to unmask his villainy. So I wrote this novel with relish. I did unmask his villainy, even if I exaggerated it; at first I think of what might have happened if I had not written it.
And all at once I revenged myself in the simplest and most frightening dream. I dreamt while slumbering deeply in the bed where I usually went to sleep in the evening. I had been oppressed by memories of my miserable days at school, and I could not shake them off. I was nothing but empty space before him, and never, under any circumstances, turned aside. I gloated over my making way for him. And lo and behold a bright idea dawned upon me! “What,” I thought, “if I push against him just as much as he pushes against me?”
There’s more to the above than just semantics. Earlier this year I coded ‘guiderails’ that help with sustaining topics to book length. And there’s also a system for handling those troublesome proper nouns.
It’s hard to say whether writing from generated text has any legs to it. I love the engineering. And Computer-Generated Short Stories has been a perennial seller. But it doesn’t look like it’ll see me into retirement! To use that phrase that describes bands who don’t sell very well, I’m ‘big in Japan’. It’s true. The Japanese and Italians seem to like this stuff. Cool!
The next project after Being Italo Calvino isn’t Being Yukio Mishima though. The closest to a Japanese connection is the character Rei Toei in William Gibson’s Idoru. Basically, I’ve experimented with creating a biography of a celebrity, working title: Diva. This means programming some code that enables me to write the book as a questions and answers session so that I effectively interview the artificial celebrity. Unfortunately, I’ll need a computer with a lot more memory to do this. So Diva is some way off yet. Watch this space.
Being Italo Calvino will hopefully be finished by late 2016.