Man I remember when people downplayed AlphaGo because it didn't teach itself unsupervised. "Nothing to see here". Only took them a few months to do AlphaZero.
AlphaZero works on chess, shogi and go and other perfect information games with discreet moves and board s.
LLMs need pairwise linear input that is composed of independent and identically distributed data.
Feed forward neural networks are effectively DAGs thus semi-decidable.
LLM require a corpus, that data is generated by humans and isn't a perfect information game.
If you dig into how many feedforward neural network can be written as a single pairwise linear function in lower dimensions, you can help build an intuition on how they work in higher dimensions that are beyond our ability to visualize.
AlphaZero being able to build a model without access to opening books or endgame tables in perfect information games was an achievement in implementation, it was not a move past existential quantifiers to a universal quantification.
LLMs still need human produced corpus because the search space is much larger than a simple perfect information game. The game board rules were the source of compression for AlphaZero, while human produced text is the source for LLMs.
Neither have a 'common sense' understanding of the underlying data, their results simply fit a finite subset of the data in the same way that parametric regression does.
As there are no accepted definitions for intelligence, mathematics is the only way to understand this.
VC dimensionally and set shattering is probably the most accessible to programming backgrounds if you are interested.
It's entirely possible that making AlphaGo teach itself with no training examples is much, much easier than doing so for an LLM. Not all problems in AI have the same hardness!
