TechTakes

courtesy @self

• preprint: https://arxiv.org/pdf/2309.02926
• blackhat abstract: https://www.blackhat.com/asia-24/briefings/schedule/index.html#llmshell-discovering-and-exploiting-rce-vulnerabilities-in-real-world-llm-integrated-frameworks-and-apps-37215
• Tong Liu's related research: https://scholar.google.com/citations?hl=en&user=egWPi_IAAAAJ

can't wait for the crypto spammers to hit every web page with a ChatGPT prompt. AI vs Crypto: whoever loses, we win

Historian sneers at billionaire while HN trip over each other to lick boot.

Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid!

Any awful.systems sub may be subsneered in this subthread, techtakes or no.

If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post, there’s no quota for posting and the bar really isn’t that high

The post Xitter web has spawned soo many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.

who could have seen this coming, other than everyone who told the homebrew tree inverter guy this was a bad idea they absolutely shouldn’t do

An AI company has been generating porn with gamers' idle GPU time in exchange for Fortnite skins and Roblox gift cards

"some workloads may generate images, text or video of a mature nature", and that any adult content generated is wiped from a users system as soon as the workload is completed.

However, one of Salad's clients is CivitAi, a platform for sharing AI generated images which has previously been investigated by 404 media. It found that the service hosts image generating AI models of specific people, whose image can then be combined with pornographic AI models to generate non-consensual sexual images.

Investigation link: https://www.404media.co/inside-the-ai-porn-marketplace-where-everything-and-everyone-is-for-sale/

In this big thread, over and over, people praise the Zuck-man for releasing Llama 3's weights. How magnanimous! How courteous! How devious!

Of course, Meta is doing this so that they don't have to worry about another 4chan leak of weights via Bittorrent.

Context: https://www.consumerfinance.gov/about-us/newsroom/cfpb-takes-action-against-coding-boot-camp-bloomtech-and-ceo-austen-allred-for-deceiving-students-and-hiding-loan-costs/

Surely the CFPB has ulterior motives when they set their sights on this "school" whose Wikipedia page is 2/3 controversies: https://en.m.wikipedia.org/wiki/Bloom_Institute_of_Technology

Turns out software engineering cannot be easily solved with a ~~small shell script~~ large language model.

The author of the article appears to be a genuine ML engineer, although some of his takes aged like fine milk. He seems to be shilling Google a bit too much for my taste. However, the sneer content is good nonetheless.

First off, the "Devin solves a task on Upwork" demo is 1. cherry picked, 2. not even correctly solved.

Second, and this is the absolutely fantastic golden nugget here, to show off its "bug solving capability" it creates its own nonsensical bugs and then reverses them. It's the ideal corporate worker, able to appear busy by creating useless work for itself out of thin air.

It also takes over 6 hours to perform this task, which would be reasonable for an experienced software engineer, but an experienced software engineer's workflow doesn't include burning a small nuclear explosion worth of energy while coding and then not actually solving the task. We don't drink that much coffee.

The next demo is a bait-and-switch again. In this case I think the author of the article fails to sneer quite as much as it's worthy -- the task the AI solves is writing test cases for finding the Least Common Multiple modulo a number. Come on, that task is fucking trivial, all those tests are oneliners! It's famously much easier to verify modulo arithmetic than it is to actually compute it. And it takes the AI an hour to do it!

It is a bit refreshing though that it didn't turn out DEVIN is just Dinesh, Eesha, Vikram, Ishani, and Niranjan working for $2/h from a slum in India.

Tesla's troubled Cybertruck appears to have hit yet another speed bump. Over the weekend, dozens of waiting customers reported that their impending deliveries had been canceled due to "an unexpected delay regarding the preparation of your vehicle."

Tesla has not announced an official stop sale or recall, and as of now, the reason for the suspended deliveries is unknown. But it's possible the electric pickup truck has a problem with its accelerator. [...] Yesterday, a Cybertruck owner on TikTok posted a video showing how the metal cover of his accelerator pedal allegedly worked itself partially loose and became jammed underneath part of the dash. The driver was able to stop the car with the brakes and put it in park. At the beginning of the month, another Cybertruck owner claimed to have crashed into a light pole due to an unintended acceleration problem.

Meanwhile, layoffs!

The closer I look, the more depressed I get.

First of all, the entire thing feels off. Quoting one commenter:

So this seems to be some kind of universal package manager where most of the content is AI generated and it's all tied into some kind of reverse bug bounty thing thing that also has crypto built in for some reason? I feel like we need a new OSS license that excludes stuff like this. Imagine AI-generated curl | bash installers 🤮

The bug bounty thing in question apparently being tea.xyz. From what I can tell, the only things actually being AI-generated are descriptions and logos for packages as an experimental web frontend for the registry, not package contents nor build/distribution instructions (thank god).

Apparently pkgx (the package manager in question) is being built by the person who created brew. I leave it up to the reader's sensibilities to decide whether this is a good or bad omen for the project itself.

Now we get to the actual sneer-worthy content (in my view): the comments given by a certain user for whom it seems PKGX is the best thing since sliced bread, and that any criticism of using AI for the project's hosted content is just and who thinks we should all change our preferences and habits to accommodate this

PKGX didn't (and still doesn't) have a description and icon/logo field. However, from beginning (since when it was tea), it had a large number of packages (more than 1200 now). So, it would have been hard to write descriptions and add images to every single package. There's more than just adding packages to the pantry. PKGX Pantry is, unlike most registries, a fully-automated one. But upstreams often change their build methods, or do things that break packaging. So, some areas like a webpage for all packages get left out (it was added a lot later). Now, it needed images and descriptions. Updating descriptions and images for every single package wouldn't be that good. So, AI-based image and description generation might be the easiest and probably also the best for everyone approach. Additionally, the hardwork of developers working on this project and every Open-Source project should be appreciated.

I got whiplash from the speed at which they pivot from arguing "it would have been hard for a human to write all these descriptions" to "the hardwork of developers working on this projet [...] should be appreciated". So it's "hard" work that justifies letting people deal with spicy autocomplete in the product itself, but less hard than copying the descriptions that many of these projects make publicly available regardless??? Not to mention the packaged software probably has some descriptions that took time and effort to make, that this thing just disregards in favor of having Stochastic Polly guess what flavor of cracker it's about to feed you.

When others push back against AI-anything being so heavily involved in this package registry project, we get the next pearl of wisdom (emphasis mine):

But personally I think, a combination of both AI and human would be the best. Instead of AI directly writing, we can maybe make it do PR (for which, we'll need to add a description field). The PR can be reviewed. And if it's not correct, can also be corrected. That's just my opinion.

Surely the task of reviewing something written by an AI that can't be blindly trusted, a task that basically requires you to know what said AI is "supposed" to write in the first place to be able to trust its outpu, is bound to always be simpler and result in better work than if you sat down and wrote the thing yourself.

Icing on the cake, the displayed profile name for the above comment's author is rustdevbtw. Truly hitting as many of the "tech shitshow" bingo squares as we can! (no shade intended towards rust itself, I really like the language, I just thinking playing into cliques like this is not great).

My original post title was going to be something a bit more sensational like "Bored of dealing with actual human package maintainers? Want to get in on that AI craze? Use an LLM to generate descriptions for curl-piped-to-bash installations scraped from the web!" but in doing my due diligence I see the actual repo owner/maintainer shows up and is infinitely more reassuring with their comments, and imo shows a good level of responsibility in cleaning up the mess that spawned from this comments section on that github issue.

Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid!

Any awful.systems sub may be subsneered in this subthread, techtakes or no.

If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post, there’s no quota for posting and the bar really isn’t that high

The post Xitter web has spawned soo many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.

I'm not sure if this fully fits into TechTakes mission statement, but "CEO thinks it's a-okay to abuse certificate trust to sell data to advertisers" is, in my opinion, a great snapshot of what brain worms live inside those people's heads.

In short, Facebook wiretapped Snapchat by sending data through their VPN company, Onavo. Installing it on your machine would add their certificates as trusted. Onavo would then intercept all communication to Snapchat and pretend the connection is TLS-secure by forging a Snapchat certificate and signing it with its own.

"Whenever someone asks a question about Snapchat, the answer is usually that because their traffic is encrypted, we have no analytics about them," Facebook CEO Mark Zuckerberg wrote in a 2016 email to Javier Olivan.

"Given how quickly they're growing, it seems important to figure out a new way to get reliable analytics about them," Zuckerberg continued. "Perhaps we need to do panels or write custom software. You should figure out how to do this."

Zuckerberg ordered his engineers to "think outside the box" to break TLS encryption in a way that would allow them to quietly sell data to advertisers.

I'm sure the brave programmers that came up with and implemented this nonsense were very proud of their service. Jesus fucking cinammon crunch Christ.

It’s been a long couple of years with people going hard on building all their communications on the gamer chat company, disregarding all warnings and concerns because of shiny creature comforts

Soon: “we trusted you! We moved from slack to you! 😭”

Guess it’s only a handful of months before the tortured handwringing starts?

archive: https://archive.ph/Oz69t

Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid!

Any awful.systems sub may be subsneered in this subthread, techtakes or no.

If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post, there’s no quota for posting and the bar really isn’t that high

The post Xitter web has spawned soo many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.

Unfortunately I can’t snip from mobile easily now, but maybe someone else can archive it and comment with archive link?