Finally got sick of the PS4 being so loud, and took it apart. Cleaned it out (a quarter of the heat sink was blocked up and gross) and replaced the thermal paste. This is how the original thermal paste looked after just 2 years:
Now it is so much quieter it’s beyond a night and day difference. It’s better than when it was new, which is kind if bizarre. Half an hour of TLOU2 and I could barely hear it, whereas before, just loading up the game caused the fan to go full speed. Would recommend anyone bothered by their PS4 noise to do this. Wish I’d done it immediately tbh. Annoyed they cut a corner in these older units, considering paste is cheap.