It was suggested sharing was okay because we are clean when we use them, so having another family member use the same one straight after is okay because the towel will still be clean, if a bit damp.
Following that logic, the towel will always be clean no matter how many times it is used. In which case, what is the point of ever washing them?