It’s just a next-word predictor

The most important thing to understand about ChatGPT is what it actually is.

At its core, ChatGPT is a large language model (LLM) trained to be excellent at predicting the next word in a sequence of words. That’s it.

It has been trained on something close to ‘all the sequences of words on the internet’, to become really good at predicting the next sequence. As one of the co-founders of OpenAI said: “Machine learning is just statistics. On steroids. Lots and lots of steroids.”

But with the right prompts, the model starts to reveal itself. Take for example this exchange:

ChatGPT listing the five smartest people alive

The natural response to this is to be impressed and a bit annoyed by ChatGPT. But remember, all it is really doing is ‘writing a logical continuation of the conversation’. So, if we alter the prompt, we get a different continuation:

ChatGPT responding to a sarcastic version of the same prompt

Neither of these responses reveal the ‘opinion’ of the AI. For the second request, it wouldn’t be logical for ChatGPT to be a smart-arse — the natural continuation of the request is to populate the list.

For example, when people think they have hacked ChatGPT by telling it to ‘go into admin mode’ or ‘ignore all previous instructions’, have they really? Or have they just created a prompt to which a certain response is the most likely continuation?

ChatGPT being told to act as a toaster

It is really good at writing

There has been a lot of commentary on what ChatGPT says, and remarkably little on how it says it. I think this is because it writes so well — so well that we don’t even notice it, yet at the same time the sense of ‘intelligence’ is unbelievable and magical.

As Arthur C. Clarke famously said, “Any sufficiently advanced technology is indistinguishable from magic.”

I asked ChatGPT for some examples of magic-feeling technology from the past:

ChatGPT describing historic examples of technology that felt like magic

This writing is amazing. Writing well is a non-trivial task. It is very hard to write accurately, coherently, with clear meaning, correct grammar, and in a style and tone suited to the audience. And ChatGPT does all of these things well almost every time.

There is an extra power to this because, culturally and socially, writing is basically a proxy for intelligence. Now, this shouldn’t always be the case — there are many highly intelligent people who haven’t had the training, or need, to write well. But if someone can write well, they are often automatically considered to be well educated, and smart.

Compare ChatGPT to DALL-E. At their core they are the exact same technology, applied to different things. ChatGPT has been trained to predict the next word, so it can write like a human. DALL-E has been trained to predict the next pixel, so it can paint like a human. DALL-E is amazing, but compared to ChatGPT it didn’t trigger anything like the same level of debate about artificial intelligence.

This was one neat and powerful example of the value of good writing: “I mentor a young lad with poor literacy skills who is starting a landscaping business. He struggles to communicate with clients in a professional manner. I created a GPT3-powered Gmail account to which he sends a message. It responds with the text to send to the client.”

It is just the beginning

Even when you understand the plumbing, the potential of these new tools cannot be underestimated. These are the early stages of a new technology — buggy, unpredictable, controversial, poorly understood and at times dangerous.

Some of the core ‘flaws’ or limitations of ChatGPT are very solvable. It can’t currently browse the internet, but it seems obvious that future tools will be able to do that and will have access to live information. Similarly, while ChatGPT can make stuff up, there will be future versions trained to base their responses entirely on explicit source materials.

Language is the most powerful and flexible form of interface. Language AI will leverage this deep-rooted flexibility to do things on our behalf in unimaginable ways. It will not only ‘use a computer like a human does’, but often be able to ‘interact with the world like a human does’.

  • What happens when an AI can phone up every plumber in your area and then compare quotes?
  • What happens when an AI can watch a football match, write you a personalised match report, show you the goals, and suggest a player for each team to sign?
  • What happens when anyone can create bespoke software by describing the tool they need?
  • What happens when an AI can complete a visa application, or job application, for anyone, instantly?

With large-language models, this is all getting closer. As I said, this is just the beginning.