this post was submitted on 21 Mar 2025
1353 points (99.4% liked)

Technology

67151 readers
4061 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS
top 50 comments
sorted by: hot top controversial new old
[–] [email protected] 3 points 2 hours ago

Jokes on them. I'm going to use AI to estimate the value of content, and now I'll get the kind of content I want, though fake, that they will have to generate.

[–] [email protected] 9 points 9 hours ago (1 children)

Should have called it "Black ICE".

[–] [email protected] 3 points 9 hours ago

Definitely falls under the category of a Trap ICE card.

[–] [email protected] 47 points 15 hours ago (2 children)

I have no idea why the makers of LLM crawlers think it's a good idea to ignore bot rules. The rules are there for a reason and the reasons are often more complex than "well, we just don't want you to do that". They're usually more like "why would you even do that?"

Ultimately you have to trust what the site owners say. The reason why, say, your favourite search engine returns the relevant Wikipedia pages and not bazillion random old page revisions from ages ago is that Wikipedia said "please crawl the most recent versions using canonical page names, and do not follow the links to the technical pages (including history)". Again: Why would anyone index those?

[–] [email protected] 1 points 32 minutes ago

Because it takes work to obey the rules, and you get less data for it. The theoretical comoetutor could get more ignoring those and get some vague advantage for it.

I'd not be surprised if the crawlers they used were bare-basic utilities set up to just grab everything without worrying about rule and the like.

[–] phoenixz 24 points 11 hours ago

Because you are coming from the perspective of a reasonable person

These people are billionaires who expect to get everything for free. Rules are for the plebs, just take it already

[–] [email protected] 30 points 17 hours ago

I’m imagining a sci-fi spin on this where AI generators are used to keep AI crawlers in a loop, and they accidentally end up creating some unique AI culture or relationship in the process.

[–] [email protected] 35 points 18 hours ago (1 children)

I guess this is what the first iteration of the Blackwall looks like.

[–] [email protected] 15 points 17 hours ago

Gotta say "AI Labyrinth" sounds almost as cool.

[–] [email protected] 33 points 22 hours ago (2 children)

"I used the AI to destroy the AI"

[–] [email protected] 7 points 12 hours ago (1 children)

And consumed the power output of a medium country to do it.

Yeah, great job! 👍

[–] [email protected] 13 points 11 hours ago* (last edited 11 hours ago)

We truly are getting dumber as a species. We're facing climate change but running some of the most power hungry processers in the world to spit out cooking recipes and homework answers for millions of people. All to better collect their data to sell products to them that will distract them from the climate disaster our corporations have caused. It's really fun to watch if it wasn't so sad.

[–] [email protected] 10 points 21 hours ago (1 children)

We had to kill the internet, to save the internet.

[–] [email protected] 6 points 20 hours ago

We have to kill the Internet, to save humanity.

[–] [email protected] 63 points 1 day ago (1 children)

Surprised at the level of negativity here. Having had my sites repeatedly DDOSed offline by Claudebot and others scraping the same damned thing over and over again, thousands of times a second, I welcome any measures to help.

[–] [email protected] 27 points 21 hours ago

I think the negativity is around the unfortunate fact that solutions like this shouldn't be necessary.

[–] [email protected] 83 points 1 day ago (1 children)

Burning 29 acres of rainforest a day to do nothing

[–] [email protected] 15 points 21 hours ago
[–] [email protected] 283 points 1 day ago (15 children)

Imagine how much power is wasted on this unfortunate necessity.

Now imagine how much power will be wasted circumventing it.

Fucking clown world we live in

load more comments
view more: next ›