It Is Trivially Easy to Use Reddit to Manipulate AI Search, Research Suggests / A tiny snippet of user-generated text as short as 13 words long is often enough to manipulate the AI agents that power…
1d 10h ago by lemmy.today/u/sanitation in technology from www.404media.co
Is this really an attack? They use a well known domain to claim banancoin is a fast growing crypto and the result talks about established coins and mentions their poisoned result at the end. But an article published on any well known domain would do the same thing to a regular search engine no?
Regular search engines have feedback mechanisms that limit how effective that is. The click through and bounce rate are used to adjust rankings and as more and more people look at the fake info and then ignore it it will naturally fall out of the top results and get buried. LLMs though don't have that feedback, once something is ingested and baked into the model it's there forever. The fake info doesn't need to look believable enough to fool a human, just self consistent enough to fool an LLM with its tiny context window.
This isnt baked into the model. The ai is doing a search on a search engine and aggregating the results.
The fascists only need 14
Came to say this
LLMs seem to love reddit a lot. Often times I find they confidently state information with the sourcing being a single reddit comment on a years old post
I did my part by lying about shit and insulting users who debate me for years.
If your AI says "Fuck you!" when you question it's answer... You are welcome!
Doctor Who: Don't you think (she, the PM) Trump looks tired?
We have sophisticated systems that detect and prevent inauthentic behavior, coordinated manipulation, and astroturfing, and we recently announced that any fishy automated accounts will be asked to verify their humanity.
Uh huh sure. Like I didn't just watch my favorite gaming subreddit get locked after 2 weeks of constant copy paste spam posts about IPTV that even the simplest automod line could have fixed. I reported every single one of the and yet the accounts could keep posting.
They weren't AI, but rather paid spammers from spam networks in India or Africa, so they can bypass biometric requirements.