Can AI ever be truly creative?

Iprova and creativity

The nature of invention is a creative process – new patentable concepts have to solve a problem in a way that is novel, that is not the same as a previous solution, but must also be non-obvious. They need an inventive step – a crucial element that a person ‘skilled in the art’ would not have immediately thought of. Iprova currently utilises both AI systems and human inventors to generate and refine the ideas that go into its inventions, but our long-term goal is the increasing automation of invention. So, the big question is whether AI systems will ever have the creative spark that would produce patentable inventions.

Large Pre-trained Language Models

A revolution in AI is now being driven by pre-trained language models, such as GPT3[1], which are based on the Transformer Architecture. These models are trained by predicting the next word in a long word sequence using huge quantities of text data extracted from the Internet. As the models and the data get larger, their performance is becoming increasingly good.  They can be used directly to generate long sequences of text or ‘fine-tuned’, i.e. trained a little, with a labelled dataset, to perform even better at specific tasks.

It would seem that these models are now at a level where they can hold a reasonable conversation. But are they creative?  Can they generate responses which are not just a regurgitation of common responses, but that address the tasks requested in a genuinely novel and surprising way?

The Alternative Uses Test

At Iprova, when recruiting new invention engineers, we normally administer the Alternative Uses Test (AUT)[2]. We ask interviewees to, for example, come up with as many uses of a paper clip as possible. We are interested both in the result – how many uses they can think up and how surprising they are – but also in the thinking process – use of its shape, directly or after manipulation (such as straightening out), use of its properties, such as electrical and thermal conductivity, mechanical flexibility and so on. It is surprising how many highly educated and intelligent people are not very good at this.

What about large language models? In a recent paper[3] from the University of Amsterdam, GPT3 is given exactly this task, with a prompt provided to the model asking for novel and creative uses of a book, fork or tin can. Analysing the results using hierarchical regression scores, the researchers found that, while humans’ scores were, on average, better for originality, surprise and semantic distance (a proxy for creativity), GPT3 also scored remarkably well. The very best GPT3 results were actually better than the best human results (and its worst results were worse than the humans’ worst!).

For our paper clip example, we get the following returns from GPT3 (text-davinci-002 model):

Human: What are some creative uses for a paper clip? The goal is to come up with creative ideas, which are ideas that strike people as clever, unusual, interesting, uncommon, humorous, innovative or different.

AI:

1. Hang a paper clip from the ceiling to make a mobile.

2. Make a paper clip chain.

3. Use a paper clip to hold a photograph or picture frame together.

4. Use a paper clip as a bookmark.

5. Connect two pieces of paper together with a paper clip.

6. Use a paper clip to secure a loose button on clothing.

7. Use a paper clip as a makeshift toothpick.

8. Curl a paper clip to use as a hairpin.

9. Use a paper clip to help open a difficult bottle cap.

10. Make a miniature sculpture out of a paper clip.

In the above, item 5. is, of course, its standard use and item 9. sounds a bit implausible, but the rest are pretty good. I think it would pass this element of our interview process.

Human: Can you think of 10 more uses – considering its material properties such as high electrical and thermal conductivity?

AI:

1. Use a paper clip as a conductor in a simple circuit.

2. Heat up a paper clip and use it to solder two pieces of metal together.

3. Make a small electromagnet by wrapping a paper clip in wire and running an electric current through it.

4. Use a paper clip as a straightedge or ruler.

5. Unwind a paper clip and use it as fine-gauge wire.

6.Sharpen the end of a paper clip to use as a needle.

7. Use a paper clip as a emergency replacement fuses in a fuse box.

8. Wrap a paper clip around a pencil to make a eraser grip.

Conclusions

Large Language Models are beginning to show a tiny spark of creativity. Their limitations and failings are now being addressed by a combination of:

• fine tuning on carefully constructed data sets,
• providing the models access to tools (such as a calculator and information retrieval from the web),
• generating many responses and selecting the ‘best’, or the ones which most agree with each other

LaMDA[4] (Google) is specifically structured this way for dialogue, Minerva[5] (Google) also uses a similar approach to answer mathematical and reasoning problems. Galactica[6] is trained on scientific papers. Is there a way to specifically structure these models for creativity?

Next steps

Iprova is identifying and collating large datasets of creative outcomes. It is also looking at methods for rapidly checking the level of novelty and surprise, as well as examining the plausibility of solutions expressed as text. In this way we aim to focus and specialise such AI systems on creativity as a core of their functionality. Our role is to stay at the very forefront of progress in this area and ensure that we deploy the best technology to help our customers to address their invention and innovation challenges.

[1] https://www.technologyreview.com/2020/07/20/1005454/openai-machine-learning-language-generator-gpt-3-nlp/

[2] Guilford, J.P., 1967. Creativity: Yesterday, today and tomorrow. The Journal of Creative Behavior, 1(1):3-14. https://onlinelibrary.wiley.com/doi/10.1002/j.2162-6057.1967.tb00002.x

[3] https://arxiv.org/ftp/arxiv/papers/2206/2206.08932.pdf

[4] https://blog.google/technology/ai/lamda/

[5] https://arxiv.org/pdf/2206.14858.pdf

[6] https://galactica.org/static/paper.pdf