How to reduce the DiScourse server costs

I do. Happy to set it up.

2 Likes

:+1:

I’ll put together a colab agenda doc tomorrow and circulate to everyone involved in a practical way beforehand.

Sound good?

3 Likes
3 Likes

Here’s the agenda doc - add and chop and change as you like:

1 Like

Some thoughts on the stuff I said I’d look at:

Upgrades (both OS and discourse)

OS upgrades on Ubuntu can generally be managed in place (no need for downtime, a restart etc). Effectively just needs someone to log onto any servers and run the following two commands:

sudo apt update
sudo apt upgrade

In general this can be done monthly by anyone with a login to the actual server and a small amount of comfort with the linux command line.

Discourse upgrades are managed through discourse itself and can be done by anyone at administrator level.

Screenshot 2023-02-20 at 19.26.34

Discourse upgrades appear to be able to run in place, without downtime - (or at least the one I ran did). This ran in about 15 minutes.

Upgrading the docker_manager section (running in the screenshot) forces the site temporarily offline - it took about 10 minutes to run.

While we could work around this with a multi-server setup, I would suggest that we just plan a monthly maintenance window during which regulars know the site may be down briefly. Admin tasks can theoretically be shared around between trusted community members.

Plugin install

These are installed via command line on the server itself and therefore require server admin (not just discourse admin) rights.

They also require a site rebuild and therefore a little downtime to install, so we would probably want to schedule any additions/removal of plugins into maintenance windows.

The plugins that Discourse install on cloud hosted discourse are at Plugin directory | Discourse - Civilized Discussion, and I know @Sean posted a list of stuff that’s installed further up, so we need to decide which ones we actually want.

Component install

These are mainly UI components that can be used as part of different themes. Discourse comes with no components and a single theme by default - you can choose a dark or light variation of it when doing the initial install.

Backup & Restore

Haven’t set this up in full, but it seems the obvious option is to setup daily or weekly backups to Amazon S3 storage. This is resilient enough for disaster recovery in case we lost the server entirely.

Each backup by default includes the database and all images/uploads stored on the server. We can setup either discourse or S3 to expire backups after x number of days/weeks so we don’t overpay.

Image storage

This’ll be the bulk of our data storage requirements. Again, S3 seems the optimal solution and there are mechanisms to reduce the cost by either transferring files to cheaper storage classes after a period of not being used or deleting them after a certain period of time, and there’s always the option of deleting old threads, which would delete unused images.

Return email (bounceback, complaints, reply by email etc)

This is the one bit I mentioned above haven’t looked into yet. Next time.

4 Likes

Great work - I was out late last night and flying back to Sweden today, but will write some more once I’m back home.

@sean are you still okay to set up a room for this?

Yeah, how’s best to share the link?

could just post it in here, but pm us if you’d prefer!

Good news that the OS and Discourse updates / upgrades sound pretty straight forward.

Do you have any sort of idea of costs for S3 backups? Fasthosts / Ionos use Acronis - 500GB is £35 per month (might get away with 200GB, which is £15)

If images are pushed straight to S3 then would they still need to be included in the scope of the backups? Also, are they automatically served up via CloudFront, meaning we don’t need a separate CDN service? Would obviously need to get some idea of costs for all this too.

Can never decide whether we’ll get randoms trying to log in or not. Can post just before we start and have a waiting room if you like?

1 Like

Sure thing.

1 Like

$0.024 per GB a month on a pay-for-what-you use basis. Cost of Restore would then be $0.09 per GB as you’d have data egress charges, but realistically you’d hope to not be paying that.

Short answer, no, with a long answer of… maybe.

If we migrate from this instance to a new one and wanted to keep everything then any images that were hosted before S3 was setup (I think this happened at some point) would need to be included in the backups.

Not automatically, no, but cloudfront could be setup. Whether it’s worth having a CDN depends on cost, what options discourse has for setting up the urls etc. Will add to my list to look at :slight_smile:

1 Like

Is there a summary of main points for tonight and anything I might need to prepare?

still on, yeah?

2 Likes

I’m up for it (though I’m probably the one with the least to contribute so that may not mean much!)

1 Like

Crap. Sorry. I had this down as 7pm. Says 6pm
on the doc.

1 Like

everyone ready now, yeah? links in the agenda doc.

i’m in rn btw.