Kris

joined 3 days ago
[–] [email protected] 2 points 2 hours ago

Yes, I also suspect that at a certain scale it would make a difference, but with a single Postgres database and no advanced clustering or so, the real-life performance metrics of a medium sized Lemmy instance strongly point to Postgres being the bottleneck and not the Python or Rust codebase of the rest of the software.

[–] [email protected] 3 points 2 hours ago* (last edited 22 minutes ago)

For regular users there is not a huge difference, but the web-interface is significantly faster and you can subscribe to topics that combine multiple communities. The disadvantage is less mobile app support. Right now only Interstellar supports Piefed.

The real advantage is for moderators and admins, as the Piefed developers actually listened to community feedback and implemented a lot of nice moderation features that require bots or annoying work-arounds on Lemmy.

And there are a few technical differences that make Piefed easier to administrate and troubleshoot from a sysadmin perspective.

[–] [email protected] 4 points 2 hours ago

I am genuinely confused. What makes you think you were ignored? We do have additional admins, the problem is physical access to the servers to replace hardware that seems to have broken.

We plan to add additional safeguards to such scenarios, but in the end it is a trade-off between reliability and costs. Running a Lemmy instance of our size on a fully managed rented server would easily triple the monthly costs. This would be beyond what I am willing to cover out of my own private pocket, thus making the instance dependent on donations. And that IMHO makes it more likely to be shut down in the future as donations are very variable.

I am not on vacations, but on a work deployment abroad, which further limits the time I can spend on this. Not that it would have made much of a difference, but at least the domain redirection and placeholder text was only possible to do on the weekend.

[–] [email protected] 2 points 3 hours ago

Sure, but they will not replace hardware and set up remote access for you, which seems like this would have been required in this case.

 

Due to the unfortunate down-time of slrpnk.net we are currently researching possible blockers for creating a Lemmy to Piefed migration guide or script for a relaunch in 1-2 months.

I have never done such a migration before, but generally it seems possible, although migrating image uploads from Pict-rs adds additional complexity (that might not be worth it).

I have created an issue on Codeberg about possible blockers here, but additional input on possible issues or examples of similar migration scripts would be appreciated.

If anyone else has similar plans, please let me know so that we can work together on this.

[–] [email protected] 2 points 4 hours ago (2 children)

It is run from a private residence in the DIY punk spririt (and this also allows us to run of a local solar PV system), but more or less the same would happen if you rent rack-space in a "real" data-center. Only if you rent a managed server or VPS someone else will be responsible to fix such issue and this comes at a significantly higher cost at the scale we operate at (slrpnk is part of a bigger project that also hosts other services).

[–] [email protected] 3 points 7 hours ago

Sorry that it took so long to get the word out. I guess we relied a bit too much on the xmpp server for internal communication.

[–] [email protected] 6 points 16 hours ago

Thanks.

Yeah at the very least this will result in some changes in the hardware setup to avoid similar situations. But of course it is impossible to forsee all possible problems.

[–] [email protected] 11 points 22 hours ago (8 children)

I think now that Piefed has an API for apps, we will see some of them adding support soon. Overall I think the benefits of a Piefed migration outweight the disadvantages, but it remains to be seen if it is doable.

[–] [email protected] 18 points 22 hours ago* (last edited 22 hours ago) (2 children)

Yeah, the xmpp server is down too. That is something that bugs me quite a bit and I will probably move that one to an external small VPS to retain a more secure backup communication channel.

[–] [email protected] 9 points 22 hours ago

Lets see. I think a relaunch on Piefed might interest some people to come back, and most slrpnk communities are rather niche and will probably stay. /c/climate might move though.

[–] [email protected] 38 points 22 hours ago* (last edited 22 hours ago) (6 children)

Slrpnk.net admin here.

The failure seems to have been in the main firewall, if it had been the server itself we could have easily restored it on another server from the backups on another machine. But as it stands, remote access is entirely cut off.

There usually is another person with hardware access, but they are on summer holidays. This seemed like an acceptable risk at the time...

An off-site backup would have been nice of course, but due to the costs involved in running an Lemmy instance of that size on a rented server, it would have not been a great option either.

I have plans to add a KVM to the main firewall via a secondary connection, but even that might have not helped in this case. I'll know more when I have physical access again.

[–] [email protected] 4 points 2 days ago

Sollte es zum Anschluss von Kanada kommen, gibt es wohl ein rechtliches Schlupfloch da Musk's Mutter gebürtige Kanadierin ist oder so.

340
Slrpnk.net outage (feddit.org)
submitted 3 days ago* (last edited 3 days ago) by [email protected] to c/[email protected]
 

Alt account of [email protected] here.

Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.

I am still trying a few things, but I am not very optimistic that I can get access.

The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.

As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable 😓

I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing 😥

Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.

view more: next ›