There is a subgenre developing of excited publications/articles with the high level category of "ChatGPT does X!" where the moment you dig into it it turns out that there's a huge silent "but" that it's performance in "X" is completely unverified by anyone worthwhile.

So we have here "ChatGPT performs better than doctors *"but* we sampled on an unpaid internet forum"

The other one thats doing the rounds where noone is verifying the source is "ChatGPT develops lethal chemical compounds *but*" It would be too unethical to verify with an actual chemist if any of these compounds it dreamt up actually do anything".

ChatGPT is showing how bad the economics of journalism are for understanding a topic outside of taking press releases at face value*