Is AI getting dumber?

[Steviant]

On 20/03/2025 at 12:54, FMH said:

 

Is it just me or the results of Perplexity and ChatGPT seem less smart. Not being able to answer to some basic questions or do analysis where they need to get data from multiple sources before compiling the results. The context window seems to be smaller. 

I do not recall that being the case. I notice a distinct degradation. I wonder if they’re “throttling” their compute to save cost.

AI is getting dumber, and it's not because they're using less compute power. 

In the beginning, LLMs rapidly became smart because the companies took higher quality data sources for training purposes, but as time has passed they have transitioned from using curated and reviewed data and moved to larger more generalized datasets.

Consequently, even though the amount of training data has increased, and the amount of energy used during training has increased, the quality of the training data overall has actually decreased as the companies scrape the bottom of the proverbial data barrel for training input.

This is why LLM improvement was dramatic for a few years but levelled off and now seems to be getting worse. The best of the collective works of humanity have, for the most part, already been ingested.

From here on out, it will be a struggle to improve these models, because they will actually need to improve the neutral networks and not simply feed them increasingly massive amounts of information. The honeymoon is over.

[Nick H. Supervisor]

1- LLMs (I won't call them AI because that is not where we are at) use libraries to source their answers. So with more information inserted from verifiable sources, they can maybe improve. Except that to paraphrase George Carlin, if you take the average IQ of people, that means half of the people are below that average. So half the people spout nonsense, and the LLM takes it and regurgitates it.

2- More people than ever have access to the Internet to provide their opinions. That's a good thing, but your LLM is going to assimilate that. For better or worse.

So yes, it's getting dumber. It was dumb to begin with. But it is working on the average IQ of mankind and what we all shout into the void.

(And yes, I know that the IQ range is not an indicator of people's capacity. It's a system that was created to try and figure people out, but it is equally flawed.)

[Martin27]

I sometimes use ChatGPT to look up information and it feels like nothing’s changed. But I’ve never used it for analyzing data or anything more complex, so maybe I just haven’t noticed.