The difference between traditional software and LLMs is the generality of their intelligence, and there are formal definitions of general intelligence such as https://en.m.wikipedia.org/wiki/AIXI
AIXI is a definition of the optimal agent and is hence uncomputable but LLMs are approximations which are approaching AIXI.
I recommend Fridman's interview with Hutter.
AIXI is a definition of the optimal agent and is hence uncomputable but LLMs are approximations which are approaching AIXI. I recommend Fridman's interview with Hutter.