Long ago, back in the OS 7 and earlier days, it was a snap to perform what was formally called a "Clean Install" of the OS. It would utterly remove the OS and place a brand-spanking new one in its place. Applications and data (other than system preferences and such) would be untouched, and you rarely even had to reinstall a single application. Adjust a few system prefs and you were good as new in no time at all.
I believe this feature went away around OS 8, and since then I've never known of a good way to perform a clean install. Migration Assistant is useless and ranting about it is unhelpful here.
I have a mid-2011 iMac, running "low" Sierra from an external SSD (Thunderbolt). I've got vestiges of Mavericks and all kinds of applications I no longer use floating around, and I feel that it's time to clean up. But every time I've done this in every flavor of OS X, it's ended up being an utter nightmare of lost application files (prefs, extensions, etc.), lost system settings, lost chat logs (Messages, Skype, etc.) and so much more. It's as bad to clean my Mac these days as it always had been on Windows!
My approach to date has been to install a fresh OS on a newly created partition of my drive, and go through countless reboots back & forth between the two systems while I copy applications and their preferences and settings over. Still it's inevitable that something will get lost or not be able to transfer over like mail rules, chat histories, or all manner of other stuff. Eventually I can delete the original partition once I'm confident in the migration (though there are often problems doing that, too!)
So is there a "best practice" or reliable way to completely replace my current macOS Sierra installation with a fresh copy, without losing any application data including preferences, mail & chat histories, app settings, etc.? I'm OK with having to reset basic system settings as I expect that, but losing anything related to my user-installed applications, is really quite unacceptable IMO.