How Do Generative AI Systems Work? - NN/G

Introducing NN/G Membership | Access all self-paced courses for $249 per year | Join by Jan. 26, 2027 |
Find the training option that fits your learning style.
September 13, 2024 2024-09-13
Want to understand how large language models (LLMs) work without getting too deep into the math? Read on for a greatly simplified explanation of a monstrously complex technical topic. I’m using metaphors to prioritize understanding rather than technical accuracy. For an even more in-depth explanation that’s still intended for nontechnical audiences, I recommend this article by Timothy B. Lee and Sean Trott.
Don’t want to read the whole article? Here’s a quick overview.
Large language models are probabilistic systems that attempt to predict word sequences. That’s what generative AI systems (genAI) do — they are making word-by-word predictions in the context of your prompt. LLMs are not fact databases; they statistically model how words tend to appear together based on their training data.
LLMs are based on a neural network architecture. Neural networks are computational systems made up of layers of neurons; a neuron performs a simple math function on its inputs. Each neuron’s output is weighted, meaning that it is multiplied by a number that captures how important it is compared to other neurons’ outputs. Neural networks are trained on large amounts of data in order to “learn” the weights that produce the best outputs.
In LLMs, words (or word fragments) are encoded as word embeddings — a series of coordinates in a multidimensional abstract space, where words with similar meaning or purpose are near each other. This representation allows the AI to “reason” with words by looking at the relative distance between them and, thus, assemble them in a meaningful way based on the context. Word embeddings are also learned by LLMs during the training phase.
Once an LLM system is trained, it is refined by feeding it conversational data (where humans have written both the “human” and “AI” parts) and other refinement training data. Humans then evaluate the outputs of various test scenarios, rewarding the system when it “aligns” with human expectations.
Go in-depth on the topic and learn how it relates to UX work in Understanding LLMs.
Generative-AI (genAI) tools are fundamentally sequence-prediction machines. What does this mean? By and large, these systems complete a sequence with whatever is most likely to appear.
Consider the idiom “It’s raining cats and ____.” If you present this incomplete sentence to a genAI system, the response will likely be the word “dogs.” This is because that pattern (“raining cats and dogs”) often occurs in written English. When you input a prompt into a system like ChatGPT, Gemini, or Claude, it tries to predict the next words that would follow your sentences.
Think of it like a complicated version of the word game Mad Libs — there’s a blank space in a sentence, and the system wants to fill it with a word. Instead of trying to come up with a ridiculous word for a humorous effect, genAI systems aim to provide the most likely response.
The fact that these AIs can perform so well is amazing. Human language is predictable enough for a system to fill in the blanks convincingly, but only after reading nearly the entire internet first.
Even more interestingly, the system’s output isn’t just a guess of the next word — it’s multiple top guesses of the next word, along with their relative probabilities. Engineers can then adjust how “loose” or “creative” the system is allowed to be in selecting from the short list of candidate words (by controlling options such as temperature).
An LLM can predict the next word because it is, essentially, a statistical model of language built with neural networks.
If generative AI systems are just playing a complicated game of Mad Libs, then how do they generate complex answers or perform complete tasks? Predicting the next word in a sequence that already exists is quite different from generating a relevant answer to a question.
LLMs are not just trying to predict a single next word (though that’s the core of the process); they are trying to complete a whole conversation, and that conversation includes your prompt as part of it. The response is the “blank” the AI is supposed to fill.
The AI does not necessarily make this prediction of the response sequentially, one word at a time. A recent breakthrough is the transformer (see below), which is a type of computational architecture that allows each word to be “aware” of all the other words in the passage.
Think of it like social groups in high school. They are self-organizing: no one person assigns people to social groups. Each person finds a group where they “fit in,” and is also aware of their role in the larger community; they know many of the other people around them (even if they aren’t directly friends) and what group they belong to. Also, each person adjusts their behavior a bit to fit in better in their groups and the community.
LLMs are built from neural networks. If we imagine that an LLM is a conductor, the neural network is the orchestra, and the individual neurons are the individual musicians. Each musician may be playing something rather simple on their own but put a bunch of them together in combinations that make sense, and you have something that can be used to voice melodies and harmonies.
Neural networks are computer systems inspired by how brains are organized, but in practice, they’re very different from biological brains. They build statistical models to predict a specific output given an input. They self-modify their statistical model based on their training data -– they “learn” to better predict the correct output after seeing many inputs.
A neural network consists of neurons (also called nodes) — a series of simple math functions that are arranged in interdependent layers:
(The neural networks used in genAI are called “deep” because they have several layers.)
Typically, the outputs of all the