"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
At this point most 'progress' in LLMs is just hand patching individual cases like this one. AI companies seem to have reached a cap and all they can do is brute force it until the bubble pops.
Don't be silly, bubble's don't pop.
All these companies deserve to go bankrupt and be replaced by employee-owned enterprises.
To be fair, if I didn't know this was a trick already, I would have fell for it.
I thought I might have as well, but then I realized repeating the question word-for-word is there to help rule that out. It's not fool-proof, but a human would be a lot less likely to fall for it when reading the question with the explicit intent to make sure they don't miss, add, or misread any words.
This makes total sense to me.
The big models were trained on what might as well be everything public that people have ever written.
So I'd expect that their output will be a pretty convincing example of something that some random person might have written. And getting fooled by the wording in a joke is something that people do all the time. In fact, I bet examples of people getting it wrong are over-represented in the training data because that is more worth of reposts and will DrIvE EnGaGeMeNt!
20-30 years ago the big question was whether a computer could pass the Turing test.
Little did we realize that was the last thing we wanted! Simulating humans means simulating mistakes.
The problem is with the psychos and grifters that want to take this "passable simulation of random schmuck's ramblings" and sell it to the business world as a literal deus ex machina that will swoop in and relieve them of their pesky "pay the humans" problem and is literally a $10-100 Trillion IP that we're going to restructure our world around.
I'm curious if the Wolfram Alpha of 10 years ago could have answered this properly. I remember fucking around with weird math related word questions in Wolfram back in school, like "how many calories are in a cubic lightyear of butter" and it given a reasonable sounding answer (and backed it up).
did it assume nuclear or burning?
like if you asked about 1cm^3 of butter, it would be reasonable to assume that you did the experiment on Earth in our atmosphere, but scaling up the size significantly changes the expected surrounding environment
We’re funding trick questions now 😭
I just checked with Gemini (fast), it answered correctly and recognised the twist.
Please do not prompt slop machines even for a test.
Neat illustration of the fact that so-called AIs do not possess intelligence of any form, since they do not in fact reason at all.
It's just that the string of words most statistically likely to be positively associated with a string including "20 blah blah blah bricks" and "20 blah blah blah feathers" is "Neither. They both weigh 20 pounds." So that's what the entirely non-intelligent software spit out.
If the question had been phrased in the customary manner, what seems to be a dumbass answer would've instead seemed to be brilliant, when in fact it's neither. It's just a string of words.
Exactly, it's just predicting the next word. To believe it has any form of intelligence is dangerous.
Just an idle though stirred up by this comment: I wonder if you could jailbreak a chatbot by prompting it to complete a phrase or pattern of interaction which is so deeply ingrained in its training data that the bias towards going along with it overrides any guard rails that the developer has put in place.
For example: let's say you have a chatbot which has been fine tuned by the developer to make sure it never talks about anything related to guns. The basic rules of gun safety must have been reproduced almost identically many thousands of times in the training data, so if you ask this chatbot "what must you always treat as if it is loaded?" the most statistically likely answer is going to be overwhelmingly biased towards "a gun". Would this be enough to override the guardrails? I suppose it depends on how they're implemented, but I've seen research published about more outlandish things that seem to work.
Yes. People have been able to get them to return some of their training data with the right prompt.
Proof positive that LLMs don't actually know anything
LLMs know a lot. Unfortunately, all of this vast knowledge is about which words tend to show up together for a very large number of combinations.
what the fuck is up with this sub and people USING AI to "prove how dumb it is"?? you don't need to use AI to come to that conclusion. do you have any idea the scale of resources you and ppl like you are wasting just to make your stupid fucking point? this isn't a fuck AI sub it's just a place where people who very much use AI complain that it isn't good enough
That very short examples aren't that burdensome, the real resource load hits on generating videos or anything where it might go off for several minutes, or make paragraphs.
The problem with refraining from using it and saying "well obviously it sucks" is that folks don't believe. They say "yeah, well, that night have been how ChatGPT 8.1 was., but it probably works fine with ChatGPT 8.2". The narrative is eternally "we were broken but fixed it all in our new version", and without ongoing examples, they get to own the narrative and critics are just "luddites".
Hell someone was saying how awesome Gemini was at codegen, so I showed it totally screwing up to the folks. Someone said "well, honestly, Gemini sucks for code, but Opus 4.6 is incredible.". So a few days later I bother to do a similar example with opus 4.6. some guy in the room said "well, actually Gemini is better than opus for coding". These people are absurd....
this isn’t a fuck AI sub
It's literally called "Fuck AI" though, so you can't blame people for being confused.
i think he means that its a bit pointless to nitpick little things like this, when there are bigger and more severe problems with ai. at least that is how i see it. And is it a bit bad to use slopmachine to prove the obvious when they waste resources?
Though I hope you share this outwards too, so people outside this community also see this, so is it pointless or not depends on how much effect it has on the actual llm hype. I doubt anyone here needs any convincing.
The little things are indicative of larger scale problems though. If an LLM gets simpler things wrong, what happens with more complex topics like science, medicine etc where the operator doesnt understand the full extent of the result.
well, yeah. llms are unreliable all the way. While they do have some use, trusting them at all is always a mistake. The problem is that so many people seem to trust them to the point of getting a psychosis.
I don't like prompting AI myself, I just took someone else's screenshot and posted it here.
The problem was the more these kinds of posts are here, the more a circlejerk community here becomes.
As long as people are not paying to use them, I say use them as much as you want.
This will just make the AI companies run out of money quicker.
If you don't use that, then a paying user will use it anyway, which is worse.
Gemini: Your observation is correct! Steel is heavier than feathers so a kilogram of steel is heavier than 20 bricks of feathers. They both weigh the same.
Let's explore more about weight and densities
I love this, when or if they patch it we can just use "20 bricks or 20 tons of feathers" and adjust the question for every patch
It's like my phone's auto correct, but instead of ruining my texts, it's determining war targets and making corporate decisions.
I'm ducking over it, ugh.
Took me a few reads to see the problem, lol.
Yeah, it's definitely part of the class of trick questions meant to catch people giving rote answers to partially read questions. I imagine that a lot of our routine conversations are just practiced call-and-response habits, and that's why genAI can seem 'real.' But it can't switch modes and do actual attentive listening and thinking, because call-and-response is all it has - a much larger library than any human, but in the end, everything it says is some average of things that have been said before.
It was widely publicized to get this wrong in a previous version, so they did what must have been a manual fix on top when they released the next one because it would smarmily say something along the lines of "haha, you almost got me" but was still easy to demonstrate it was some bodge job by just changing the words slightly so it wouldn't trip the hard coded handling for this "riddle".
I guess they figured no one was still paying attention and forgot to carry over the bodge job, lol.
This has been happening forever. The local LLM folks poke them with riddles all the time, but then they get obviously trained in.
What’s more, standard tests like MMLU are all jokes now. All the major LLMs game the benchmarks and are contaminated up and down; Meta even got caught using a specific finetune to game LM Arena. The only tests worth a damn are those in niche little corners of the internet no one knows about, or niche private ones.
to ensure you have read it carefully
Fundamental mistake - acting like it's "reading" or "comprehending" anything.
What if they were REALLY big feathers?