Reddthat Announcements

697 readers

1 users here now

Main Announcements related to Reddthat.

For all support relating to Reddthat, please go to [email protected]

founded 2 years ago

MODERATORS

[email protected]

Database Migrated to a new server (reddthat.com)

submitted 10 months ago by [email protected] to c/[email protected]

8 comments fedilink hide all child comments

There was a period on an hour where we migrated our database to it's own server in the same data centre.

This server has faster CPUs than our current one, 3.5GHz v 3.0GHz, and if all goes well we will migrate the extra CPUs we bought to the DB server.

The outage was prolonged due to a PEBKAC issue. I mistyped an IP in our new postgres authentication as such I kept getting a "permission error".

The decision of always migrating the DB to it's own server was always going to happen but was pushed forward with the recent donations we got from Guest on our Ko-Fi page! Thank-you again!

If you see any issues relating to ungodly load times please feel free to let us know!

Cheers,
Tiff

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 3 points 10 months ago (6 children)

Hmmmm that isn't great news. While testing it was working great, which is just my luck.

I can see that there are a lot more spikes now compared to before, and even when ssh'd to the server I can sometimes feel the lag.

I'll be investigating today as I really don't want to roll back to our single server again. There are probably some fine tuning items that need to be done on the network stack side now.

[–] [email protected] 3 points 10 months ago (5 children)

Wishing you the best of luck in your investigation! Hopefully it turns out to be something quick and easy, but having been through the wringer debugging my own projects, I totally get it if it's not.

[–] [email protected] 3 points 10 months ago (1 children)

Alright, I've had a look, and we are now HA. We now have a lemmy front end and back end instances across two different servers with the third server being the new database server.

This has reduced the number of connections through our one server to a more manageable number which seems to have helped with the lag spikes! From the logs it does look like the total number of lag spikes has disappeared but I'm not counting my chickens yet.

With regards to regular requests. With the added server, on average we are looking at

comments: 1-3s
users: 2-5s
Which again is still not ideal, but it's something to work towards.

One of our servers is also slower at responding in general. So I think we'll decommission it at the end of this month and use the money to buy 1-2 more frontend servers, (that's how big it is).

[–] [email protected] 3 points 10 months ago

Thank you for your work

load more comments (3 replies)