Lemmy Server Performance

8 readers

1 users here now

lemmy_server uses the Diesel ORM that automatically generates SQL statements. There are serious performance problems in June and July 2023 preventing Lemmy from scaling. Topics include caching, PostgreSQL extensions for troubleshooting, Client/Server Code/SQL Data/server operator apps/sever operator API (performance and storage monitoring), etc.

founded 2 years ago

MODERATORS

[email protected]

Lemmy PERFORMANCE CRISIS: popular instances with many remote instances following big communities could stop federating outbound for Votes on comments/posts - code path identified (lemmy.ml)

submitted 2 years ago* (last edited 2 years ago) by [email protected] to c/[email protected]

3 comments fedilink hide all child comments

I spent several hours tracing in production (updating the code a dozen times with extra logging) to identify the actual path the lemmy_server code uses for outbound federation of votes to subscribed servers.

Major popular servers, Beehaw, Leemy.world, Lemmy.ml - have a large number of instance servers subscribing to their communities to get copies of every post/comment. Comment votes/likes are the most common activity, and it is proposed that during the PERFORMANCE CRISIS that outbound vote/like sharing be turned off by these overwhelmed servers.

pull request for draft:

https://github.com/LemmyNet/lemmy/compare/main...RocketDerp:lemmy_comment_votes_nofed1:no_federation_of_votes_outbound0

EDIT: LEMMY_SKIP_FEDERATE_VOTES environment variable

top 3 comments

sorted by: hot top controversial new old

[–] [email protected] 1 points 2 years ago

Part of what makes Lemmy (and other voting link aggregators) work is the voting aspect. By taking away outbound vote federation, it forces further consolidation into these popular instances. Thereby further exacerbate the problem because now they’re even more consolidated and the posts and comments eventually becomes the bottleneck for the exact same underlying chatty protocol. For this reason, I’d be vehemently against this change without a pairing PR that allows this information to be requested via a batch pull that the protocol makes available.

[–] [email protected] 1 points 2 years ago (1 children)

Somewhat related, but why are we federating votes? Why not just federate the upvote count and downvote count? Does each server need to track the identity of every voter on a subscribed community?

Each server will track votes from their own users, preventing duplicate votes.

[–] [email protected] 1 points 2 years ago

Why not just federate the upvote count and downvote count?

I think the answer to that is that it isn't an optimized design.

Does each server need to track the identity of every voter on a subscribed community?

I think so. Which isn't a terrible assumption that user who votes will eventually comment/post and that profile will be of use.