It seems you didn't mention the Garbage In Garbage Out problem, the problem of AI models training not on raw human-produced text, but on rather previous AI outputs. Could that be one reason the language and compute code models decrease in quality? I'm not sure it would it would be explain decreasing quality of raw math operations like prime number identification, but maybe it would affect that too?