I really hate this reduction of gpt models. Is the model probabilistic? Absolutely. But it isn’t simply learning a comprehensible probability of words–it is generating a massively complex conditional probability sequence for words. Largely, humans might be said to do the same thing. We make a best guess at the sequence of words we decide to use based on conditional probabilities along a myriad number of conditions (including semantics of the thing we want to say).