admin

joined 2 years ago
MODERATOR OF
[–] [email protected] 4 points 1 year ago (3 children)

No problem, as far as I'm concerned, all suspensions are negotiable. Which one was it? I'll re-enable it straight away.

[–] [email protected] 6 points 1 year ago* (last edited 1 year ago)

List of disabled communities

These communities were deemed too interactive (mostly invoking discussion with OP or the community) for Lemmit, and will not be receiving updates from the bot.

Ident NSFW Subscribers
40klore 52
AmItheAsshole 1150
ApolloApp 49
AskMen 34
AskWomen NSFW 8
ElectricScooters 56
GMEmate 38
GearTrade 54
academicquran 2
aip 2
aitah 132
amdlaptops 52
amithecloaca 31
appletv 6
ask_detransition 45
askculinary 53
askhistorians 52
asoneafterinfidelity 51
aspergers 15
assettopiraterevamp 7
autoimmuneprotocol 5
aveiro 26
azure 55
azurecertification 41
beleggen 39
bestof 118
boardgames 79
brave 24
browsers 19
budgetwithbuckets 38
casualuk 102
characterrant 46
chipcommunity 29
classicalmusic 37
coins 31
cooking 55
covidlonghaulers 65
cozygamers 34
cycling 65
dating_advice 37
ddintogme 3
detrans 46
diy 18
economonitor 15
edh 5
ehlersdanlos 35
explainlikeimfive 256
finansial 48
fitness 32
footfetishtalks NSFW 11
france 106
fuckhoa 10
geochallenges 29
gme 2
gmejungle 2
handhelds 58
historiansanswered 65
howto 5
iFit 47
ifyoulikeblank 3
iinternetisbeautiful 28
indonesian 39
japanlife 20
kingcobrajfs 2
kratom 2
kratomm 2
loalllama 24
luciddreaming 11
malefashionadvice 5
menslib 44
minimalism 5
modernmagic 3
musemeditation 46
music_survivor 44
neutralpolitics 48
newpipe 2
nostupidquestions 110
oldhagfashion 58
outerwilds 29
outside 9
pathfinder2e 12
peerlessdad 6
perchance 35
piracybackup 46
piracyforreddit 31
plexsharesnew 79
plexsubs 52
poland 42
preyingmantis 7
professors 27
protonmail 11
redrising 9
relationship_advice 36
revancedapp 67
sao_paulo 21
sex 185
siloseries 46
simracing 83
sugarlifestyleforum 5
syncforlemmy 123
sysadmin 33
thriftgrift 39
thriftstorehauls 58
tillsverige 11
transeducate 2
translator 8
transsexual 41
twobestfriendsplay 59
twohottakes 39
twoxchromosomes 91
ukpersonalfinance 78
unraid 50
velo 32
welding 8
whatisthis 42
whatisthisthing 116
windowsondeck 47
xreal 42
zielonagora 7
[–] [email protected] 1 points 1 year ago

Point taken and no offense taken. Hell, I've been thinking about this for some time myself as well.

For now I have disabled the requesting of new subreddits/communities, and tightened the upvote filter to be at least 10 and have an 80% upvote ratio. It's not much, but it should limit the amount unpopular posts somewhat.

Next step is to go through the list of communities and purge all the "ghost communities" - ones that rely heavily on community participation, like for example AskReddit (221 subscribers). It's going to be tricky to make that selection though. What to do, for example, with [email protected]? With 1100 subscribers, it's the most popular community on this server. Clearly people are getting something out of it, even though there are no replies to it.

I guess I'll have to figure something out.

[–] [email protected] 1 points 2 years ago (1 children)

Sure thing, done.

[–] [email protected] 1 points 2 years ago

Thanks, that means a lot.

I have nothing but respect towards Ruud and the others at LemmyWorld, and I understand how some instances want to block the Lemmit bot - so no hard feelings to whatsoever.

[–] [email protected] 1 points 2 years ago (2 children)

I figured it out - earlier this morning Lemmit was defederated from your instance, but that seems to have been fixed now.

[–] [email protected] 2 points 2 years ago* (last edited 2 years ago) (8 children)

Still not getting updates properly, but I have approved lemmy.basedcount.com, @[email protected]

[–] [email protected] 1 points 2 years ago

Yeah, wasn't properly subscribed to this community yet, so couldn't reply to them. Glad to have helped though.

[–] [email protected] 1 points 2 years ago* (last edited 2 years ago) (2 children)

Hey all. Owner of a highly controversial Lemmy instance requesting a guarantee here: lemmit.online - the on-demand Reddit archiver.

While the bot that runs on it is not loved by everyone, I can assure you that I will never let any other users register, and the only content that gets posted on it will have to be vetted by Reddit first.

Edit: Guaranteed, thanks! Next admin can tag me, I'll gladly pay it forward.

[–] [email protected] 2 points 2 years ago (1 children)

So ehm... Better late than never :D

 

You know, on account of me upping that one setting in the admin which I should have thought of long ago.

 

Because there's an actual proper mod over there: [email protected].

6
submitted 2 years ago* (last edited 2 years ago) by [email protected] to c/[email protected]
 

Okay, this one took me a bit longer than I planned (mostly due to sql fun and trying to use integers as minutes, WEEEE!).

Backdrop: Last week I disabled the mirroring of a couple of subreddits to the database, because they were initially requested but the nobody subscribed to them. At the same time, the bot was just crawling in a loop, starting at todayilearned, ending at latestsubreddit. As more subreddits were requested, this loop took longer and longer (21 minutes before I rolled out this update). This wasn't sustainable.

So here's the new situation. The more popular a community is, the more often it will be updated. In this case popular means a mixture between number of subscribers and the amount of posts it receives per day (Link to relevant snippet of source code).

In short, the most popular subs will be synced every 10 minutes, the next tier ever 30 minutes, 120 minutes and the content with either no posts per day or no subscribers (other than the bot), will only be synced every 12 hours. I hope this will hit a good distribution of updates vs popularity, but it will most likely be refined at some point in the future.

Speaking of distribution, we now have over 300 communities on this server 🥳, and their update intervals are spread out as such:

  • Every 10 minutes: 22
  • Every 30 minutes: 39
  • Every 60 minutes: 55
  • Every 120 minutes: 143
  • Every 720 minutes: 44

With this update running live (I started typing after I deployed it, and it has now gotten through the backlog of 'abandoned' subs), I'm going to step back from feature development for a few days. Any bugs that cause the bot to crash will of course continue to be addressed.

Have a blast!

 

Before was running on the cheapest model (1 core / 1GB mem / 30GB storage) at $12/month. The machine was running pretty low on memory, causing it to start swapping, which in turn caused the cpu to get too busy, and everything to slow down.

Now it has a whopping 2GB of memory, and things seem to have calmed down - cpu is back to around 10-15% usage, and swap is down to 0. Happy times all around.

Because of the amount of subs being archived, it now takes about 15 minutes between updates for each sub (was 18 before I updated the VM).

I'm planning to build some kind of scoring system, based on the amount of posts per subreddit (per day?), and amount of subscribers on the lemmy community. That way communities with little subscribers or that don't see many posts per day, will only be updated once per hour.

At the same time, I feel that subs like AskReddit, OutOfTheLoop and other "question-based" subreddits shouldn't be archived by Lemmit. In my opinion those kind of posts are useless without those answers, but please let me know if you disagree.

 
  • Fixed a bug where posts would not be submitted because the title didn't contain long enough words.
  • Fixed a bug where posts would not be submitted because the url was too long.
  • Fixed a bug where posts would not be submitted when it was linking to a /user subreddit.
  • Fixed a bug where the bot would think Every Post Everywhere was a subreddit request, and would reply to it.
  • Fixed a bug where the bot would crash without recovering whenever something went wrong during new subreddit requests

A fruitful day all in all, I'd say.

 

That the replies-everywhere-bug was just because I forgot to include a variable in the bot deployment? 🤦

18
submitted 2 years ago* (last edited 1 year ago) by [email protected] to c/[email protected]
 

In the short time since this instance and bot launched, I've been seeing the same questions resurface multiple times. This is totally understandable, since the concept of a Fediverse is still new to most (myself included), and this server is not like the others.

Q: What is Lemmit?

A: Lemmit is a Lemmy instance specifically designed for archiving Reddit content. Users can request new subreddits to be included in the archiving process by posting in the [email protected] community. It is powered by an open source python bot, which periodically checks the request list, adds new requests to the queue, and continuously monitors the Hot feed of those subs for new posts to cross-post here.

Q: Does it synchronize comments?

A: No, that would be impossible. Considering there are thousands of posts already on Lemmit, many of them having at least several hundred comments on Reddit, often buried in deep layers, it simply wouldn't be feasible to index those for more than a few posts, let alone keep them up to date.

Unfortunately, this means that archiving certain subreddits, such as Ask Historians/Men/Women/Hyperintelligentshadesofthecolourblue-type subs, is going to be rather pointless.

Q: Can it send comments back to Reddit?

A: No, it cannot. The purpose is to help bootstrap the Lemmy platform, not to serve as a bridge between the two networks. Also, see the answer about synchronizing comments.

Q: Can I request any subreddit?

A: ~~Technically, yes. However, as the list of subs grows, the time it takes to update all of them will also increase. I do not have strict guidelines in place for this, so I'm relying on your common sense (hoooo boy). At some point, I will probably have to either stop accepting new requests or disable scraping for very low-traffic communities.~~ Nope. Automatic requests for new subreddits are closed. You can send me a PM if you think a specific subreddit would be a valuable addition to the Lemmyverse, but you'll have to make a strong case.

Q: Does this use the API? Will it keep working after July 1st?

A: Nope, it uses a combination of the public feed and scraping old.reddit.com. So, as long as those are still available, it will continue working. And even if they close those sources, there will probably be new ways to achieve the same effect. "Content, eh, finds a way."

Q: This is spam, can you stop?

A: First of all, I apologise for the inconvenience. All you have to do is block @[email protected], and none of its posts will ever show up on your instance. If you you don't want anyone else on your server to be exposed to this bot/instance, you should convince your admin to defederate from lemmit.online. Since there are no other users on here, there will be no harm done.

Obviously I could stop, because running this server and software is only ever going to cost me time and money. But for the reasons listed above, I still think this server is a useful addition to the lemmyverse at this time. But I'm looking forward to the day where I can turn the bot off because it's no longer needed.

Q: What started this?

A: Okay, nobody asked this, but I'm going to tell you anyway. After Reddit made it clear that they are effectively killing third-party apps and implementing plenty of other anti-end user decisions, I realized that I would either have to accept not being able to access my time-wasting content or have to do so in a rather uncomfortable way (either through the official app or old.reddit.com for as long as they'll allow it to exist).

Being a stubborn developer, naturally, I chose option C: Have my own Reddit. With blackjack, and hookers. This way, I would still be able to access my beloved content without being beholden to Reddit's mood swings and abusive relationship tendencies.

Besides that, I also know that Content is King. So I'm order to counter the network effect (No users because no content, No content because no users), I figured it would be better to have some inorganic content to bootstrap the adoption of Lemmy.

Q: Are NSFW subreddits allowed?

A: Absolutely. Like I said: Blackjack and hookers.

Q: My request isn't picked up by the bot!

A: That isn't a question. ~~But yeah, the process isn't flawless yet. I'm trying to iron out all the bugs as I encounter them. In the meantime, feel free to re-request the subreddit by making a second post. No harm done.~~ New subreddit requests are closed for the time being.

Q: No new posts are showing up at all on Lemmit

A: If no posts are appearing on the Lemmit Frontpage (sorted by NEW), it's possible that the bot has crashed or is stuck on something. Since no software is flawless, this sometimes happens. I usually fix this as soon as I'm aware, and I'm happy to say that these kinds of fatal errors are becoming less and less frequent. However, they may still occur, and as a human with needs of sleep and other responsibilities, I'm not always able to fix them immediately.

Q: Posts aren't showing up on my instance, what's up?

A: Due to the spammy nature of the bot, some server admins choose to block this server, and that is completely understandable. So first of all, make sure to check the instances link in the footer of your home server. If Lemmit is the Blocked Instances list, you're out of luck.

When you have verified that Lemmit is not blocked on your instance, try unsubscribing, waiting a little, and then re-subscribing. That tends to fix things.

 

Long story short: I messed up with the domain registration for this instance, and never replied to a mandatory email. The domainname (lemmit.online) got put in suspension, causing disconnects all over the fediverse.

I fixed it as soon as I found out, but it will probably take a few more hours for the issues to be fully fixed.

So ehm. Whoops. Hope this explains and fixes the federation issues we've been having today.

 
1
/r/food (lemmit.online)
 

Most importantly that the bot no longer crashes (and does nothing all night while I sleep 😛) when trying to create a community that has already been requested.

Furthermore mostly making the code prettier and adding tests.

view more: ‹ prev next ›