https://lemmy.ml/post/31490862 pretty interesting article linked in this post, tl;dr researchers tried to get AI agents to run a simulated vending machine (which, let's be clear, is a solved problem and can be done with a normal algorithm better and cheaper) and it didn't go that great. Even if some of the test runs actually managed to earn money, they mostly devolved into the AI becoming convinced that the system doesn't work and desperately trying to email someone about it (even FBI, one memorable time). I think it illustrates quite well just how badly things would go if we left anything to AI agents. What are the odds anyone involved with pushing autoplag into everything actually reads this though...
this post was submitted on 08 Jun 2025
15 points (100.0% liked)
TechTakes
1941 readers
59 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 2 years ago
MODERATORS
Just watched MI: Final Reckoning. Spoiler free comments: I didn’t know that this and the previous film featured an AI based plot. AI doomers feature in a funny way, seemingly inspired by LW doomers, tho definitely not.
Couple months ago I saw a flurry of posts from far-right accounts going 'Jeffrey Epstein Innocent (he didn't do it).' Now it's morphing into 'Jeffrey Epstein Innocent (he DID do it, but ackshually it's ephebaphilia and if ONLY someone would do something about those pesky Age of Consent laws...)'
PS: AT deleted her post, fortunately someone saved it to Internet Archive
Yeah, its the BAP crew. Last few years saw an inrush of far right lolicon fans into that space.
New Blood in the Machine: The weaponization of Waymo, about protesters torching Waymos in a repeat of last year's Waymo Warm-Overs.
My T-shirt: there's 0 good uses for self-driving taxis
Protesters: call self-driving taxis to block streets on the way of the police, then set the damn things on fire
My T-shirt: there's 1 good uses for self-driving taxis
Barricades-as-a-Service
Coupe d'etat
Sans Parking-lottes
New Zitron dropped, and, fuck, I feel this one in my bones.
What does the “better” version of ChatGPT look like, exactly? What’s cool about ChatGPT? [...] Because the actual answer is “a ChatGPT that actually works.” [...] A better ChatGPT would quite literally be a different product.
This is the heart of recognizing so much of the bullshit in the tech field. I also want to make sure that our friends in the Ratsphere get theirs for their role in enabling everyone to pretend there's a coherent path between the current state of LLMs and that hypothetical future where they can actually do things.
But the Ratspace doesn't just expect them to actually do things, but also self improve. Which is another step above just human level intelligence, it also means that self improvement is possible (and on the highest level of nuttyness, unbound), a thing we have not even seen if it is possible. And it certainly doesn't seem to be, as the lengths between a newer better version of chatGPT seems to be increasing (an interface around it doesn't count). So imho due to chatgpt/LLMs and the lack of fast improvements we have seen recently (some even say performance has decreased, so we are not even getting incremental innovations), means that the 'could lead to AGI-foom' possibility space has actually shrunk, as LLMs will not take us there. And everything including the kitchen sink has been thrown at the idea. To use some AI-weirdo lingo: With the decels not in play(*), why are the accels not delivering?
*: And lets face it, on the fronts that matter, we have lost the battle so far.
E: full disclosure I have not read Zitrons article, they are a bit long at times, look at it, you could read 1/4th of a SSC article in the same time.
Can confirm that about Zitron's writing. He even leaves you with a sense of righteous fury instead of smug self-satisfaction.
And I think that the whole bullshit "foom" argument is part of the problem. For the most prominent "thinkers" in related or overlapping spaces with where these LLM products are coming from the narrative was never about whether or not these models were actually capable of what they were being advertised for. Even the stochastic parrot arguments, arguably the strongest and most well-formulated anti-AI argument when the actual data was arguably still coming in, was dismissed basically out of hand. "Something something emergent something." Meanwhile they just keep throwing more money and energy into this goddamn pit and the real material harms keep stacking up.
ran across this, just quickly wanted to scream infinitely
(as an aside, I've also recently (finally) joined the ACM, and clicking around in that has so far been .... quite the experience. I actually want to make a bigger post about it later on, because it is worth more than a single-comment sneer)
- You will understand how to use AI tools for real-time employee engagement analysis
- You will create personalized employee development plans using AI-driven analytics
- You will learn to enhance employee well-being programs with AI-driven insights and recommendations
You will learn to create the torment nexus
- You will prepare your career for your future work in a world with robots and AI
You will learn to live in the torment nexus
- You will gain expertise in ethical considerations when implementing AI in HR practices
I assume it's a single slide that says "LOL who cares"
Bringing over aio's comment from the end of last week's stubsack:
This week the WikiMedia Foundation tried to gather support for adding LLM summaries to the top of every Wikipedia article. The proposal was overwhelmingly rejected by the community, but the WMF hasn't gotten the message, saying that the project has been "paused". It sounds like they plan to push it through regardless.
Way down in the linked wall o' text, there's a comment by "Chaotic Enby" that struck me:
Another summary I just checked, which caused me a lot more worries than simple inaccuracies: Cambrian. The last sentence of that summary is "The Cambrian ended with creatures like myriapods and arachnids starting to live on land, along with early plants.", which already sounds weird: we don't have any fossils of land arthropods in the Cambrian, and, while there has been a hypothesis that myriapods might have emerged in the Late Cambrian, I haven't heard anything similar being proposed about arachnids. But that's not the worrying part.
No, the issue is that nowhere in the entire Cambrian article are myriapods or arachnids mentioned at all. Only one sentence in the entire article relates to that hypothesis: "Molecular clock estimates have also led some authors to suggest that arthropods colonised land during the Cambrian, but again the earliest physical evidence of this is during the following Ordovician". This might indicate that the model is relying on its own internal knowledge, and not just on the contents of the article itself, to generate an "AI overview" of the topic instead.
Further down the thread, there's a comment by "Gnomingstuff" that looks worth saving:
There was an 8-person community feedback study done before this (a UI/UX text using the original Dopamine summary), and the results are depressing as hell. The reason this was being pushed to prod sure seems to be the cheerleading coming from 7 out of those 8 people: "Humans can lie but AI is unbiased," "I trust AI 100%," etc.
Perhaps the most depressing is this quote -- "This also suggests that people who are technically and linguistically hyper-literate like most of our editors, internet pundits, and WMF staff will like the feature the least. The feature isn't really "for" them" -- since it seems very much like an invitation to ignore all of us, and to dismiss any negative media coverage that may ensue (the demeaning "internet pundits").
Sorry for all the bricks of text here, this is just so astonishingly awful on all levels and everything that I find seems to be worse than the last.
Another comment by "CMD" evaluates the summary of the dopamine article mentioned there:
The first sentence is in the article. However, the second sentence mentions "emotion", a word that while in a couple of reference titles isn't in the article at all. The third sentence says "creating a sense of pleasure", but the article says "In popular culture and media, dopamine is often portrayed as the main chemical of pleasure, but the current opinion in pharmacology is that dopamine instead confers motivational salience", a contradiction. "This neurotransmitter also helps us focus and stay motivated by influencing our behavior and thoughts". Where is this even from? Focus isn't mentioned in the article at all, nor is influencing thoughts. As for the final sentence, depression is mentioned a single time in the article in what is almost an extended aside, and any summary would surely have picked some of the examples of disorders prominent enough to be actually in the lead.
So that's one of five sentences supported by the article. Perhaps the AI is hallucinating, or perhaps it's drawing from other sources like any widespread llm. What it definitely doesn't seem to be doing is taking existing article text and simplifying it.
The thing that galls me here even more than other slop is that there isn't even some kind of horrible capitalist logic underneath it. Like, what value is this supposed to create? Replacing the leads written by actual editors, who work for free? You already have free labor doing a better job than this, why would you compromise the product for the opportunity to spend money on compute for these LLM not-even-actually-summaries? Pure brainrot.
Maybe someone has put into their heads that they have to "go with the times", because AI is "inevitable" and "here to stay". And if they don't adapt, AI would obsolete them. That Wikipedia would become irrelevant because their leadership was hostile to "progress" and rejected "emerging technology", just like Wikipedia obsoleted most of the old print encyclopedia vendors. And one day they would be blamed for it, because they were stuck in the past at a crucial moment. But if they adopt AI now, they might imagine, one day they will be praised as the visionaries who carried Wikipedia over to the next golden age of technology.
Of course all of that is complete bullshit. But instilling those fears ("use it now, or you will be left behind!") is a big part of the AI marketing messaging which is blasted everywhere non-stop. So I wouldn't be surprised if those are the brainworms in their heads.
That's probably true, but it also speaks to Ed Zitron's latest piece about the rise of the Business Idiot. You can explain why Wikipedia disrupted previous encyclopedia providers in very specific terms: crowdsourced production to volunteer editors cuts costs massively and allows the product to be delivered free (which also increases the pool of possible editors and improves quality), and the strict* adherence to community standards and sourcing guidelines prevents the worse loss of truth and credibility that you may expect.
But there is no such story that I can find for how Wikipedia gets disrupted by Gen AI. At worst it becomes a tool in the editor's belt, but the fundamental economics and structure just aren't impacted. But if you're a business idiot then you can't actually explain it either way and so of course it seems plausible
Some AI company waving a big donation outside of the spotlight? Dorks trying to burnish their resumes?
Ya gotta think it's going to lead to a rebellion.
but the WMF hasn't gotten the message, saying that the project has been "paused". It sounds like they plan to push it through regardless.
Classic “Yes” / “ask me later”. You hate to see it.
Example #"I've lost count" of LLMs ignoring instructions and operating like the bullshit spewing machines they are.
So, I've been spending too much time on subreddits with heavy promptfondler presence, such as /r/singularity, and the reddit algorithm keeps recommending me subreddit with even more unhinged LLM hype. One annoying trend I've noted is that people constantly conflate LLM-hybrid approaches, such as AlphaGeometry or AlphaEvolve (or even approaches that don't involve LLMs at all, such as AlphaFold) with LLMs themselves. From their they act like of course LLMs can [insert things LLMs can't do: invent drugs, optimize networks, reliably solve geometry exercise, etc.].
Like I saw multiple instances of commenters questioning/mocking/criticizing the recent Apple paper using AlphaGeometry as a counter example. AlphaGeometry can actually solve most of the problems without an LLM at all, the LLM component replaces a set of heuristics that make suggestions on proof approaches, the majority of the proof work is done by a symbolic AI working with a rigid formal proof system.
I don't really have anywhere I'm going with this, just something I noted that I don't want to waste the energy repeatedly re-explaining on reddit, so I'm letting a primal scream out here to get it out of my system.
Yes, thank you, I'm also annoyed about this. Even classic "AI" approaches for simple pattern detection (what used to be called "ML" a few hype waves ago, although it's much older than that even) are now conflated with capabilities of LLMs. People are led to believe that ChatGPT is the latest and best and greatest evolution of "AI" in general, with all capabilities that have ever been in anything. And it's difficult to explain how wrong this is without getting too technical.
Related, this fun article: ChatGPT "Absolutely Wrecked" at Chess by Atari 2600 Console From 1977
Relatedly, the gathering of (useful, actually works in real life, can be used to make products that turn a profit or that people actually want, and sometimes even all of the above at the same time) computer vision and machine learning and LLMs under the umbrella of “AI” is something I find particularly galling.
The eventual collapse of the AI bubble and the subsequent second AI winter is going to take a lot of useful technology with it that had the misfortune to be standing a bit too close to LLMs.