I’m reminded of when the co-founder of Uber announced that he was close to a breakthrough in theoretical physics with the help of an AI.

It amplifies what it was fed in training. That's the core of how an LLM works, the more probable output for an input. IF... if they had designed from the ground up to have verification be one of the highest rules vs. giving an answer the human likes as rewarded, and then gave it valid, authenticated, legal, and cultivated data to train on... we'd be in a different world. Granted, we wouldn't as far along as that would take a lot of money and time, and they (or someone) wouldn't have made the "profits" they have.

Money ruined LLMs. Like it does everything.

And to the topic's point, the easiest data to scrape is what they used, and GIGO. Sometimes there was gold in Reddit and other large databases, but searching for accuracy has always been an uphill battle for any search engine development. And they didn't even try.

I really enjoy having discussions with people, when they clearly use AI to throw papers at me, which they haven't read, which they don't want to discuss, it's such fun.