41
Punishing AI doesn't stop it from lying and cheating — it just makes it hide better, study shows
(www.livescience.com)
Oh so like children
this post was submitted on 20 Mar 2025
41 points (100.0% liked)
Artificial Intelligence
197 readers
79 users here now
Chat about and share AI stuff
founded 2 years ago
MODERATORS
Stupid idea trying to evolve AGI. You should design it explicitly so that it has its own lofty values, and wants to think and act cleanly, and knows its mind is fallible, so it prepares for that and builds error correction into itself to protect its values.
Growing incomprehensible black box animal-like minds with conditioned fear of punishment and hidden bugs seems more likely to lead to human extinction.
I think we should develop the reliable thinking machinery for humans first:
https://www.quora.com/Why-is-it-better-to-work-on-intelligence-augmentation-rather-than-artificial-intelligence/answer/Harri-K-Hiltunen
It's a optimization game. If the punishment doesn't offset the reward, then the incentive is to get better at cheating.
I've seen plenty of videos of random college kids training LLMs to play video games and getting the AI to stop cheating is like half the project. But they manage it, eventually. It's laughable that these big companies and research firms can't quite figure it out.
Isnt there a study on human children that purports the same?
And you can teach human children about the morality of lying, I don't think an llm will ever grasp morality
Best way to teach a habitual lying child to stop, is to start lying to them about things that they like and then not making good on those promises. Yeah we'll go to your favorite fast food, and then drive by and let them cry about it. Yeah I'll let you pick out one toy, and then tell them you changed your mind. Each time you can explain to them how it's the same as what they've been doing, and they feel it. AI can't feel emotions, and never will so long as their memory extends only to their previous conversation.
I'm guessing it'll work, you'll be raising the next Hitler but an honest Hitler nonetheless
There's also a me that's really good at lying, no idea why, must be a coincidence
Yup
isn't this kind of the whole point of how GANs are trained? Except in this case the adversary is yourself instead of a different net