Because we as humans have the unique ability to reason, that makes us special. Because we're something special, we have an obligation to each other to be respectful of what we are; so when a person does something bad, even if they do it in private, they are shaming not only themselves but everyone.
I think the starting point is is a valid one. Our ability to reason makes us special, and we should hold onto that distinction. When one person does something shameful, it does diminish all of us. But what are the actions that bring shame on everyone? I think the shameful act is not the private sin that harms no one else, but rather denying another person the freedom to do the thing he wants to do only because it offends someone else's morals.