Society is best served if (almost) nobody ever commits crime in the first place, assuming of course a sane definition of "crime". When the population in general has a high confidence that criminals will be caught and punished, we can leave our cars unlocked and walk home from work at night; we all win.
When a particular individual commits a crime, we as society want to feel positively protected from that specific individual committing another crime. While people are in jail, they're not mugging old guys in the park.
(I know I'm using this word in my own idiosyncratic way; most people's definition would combine with my #1.) When (if) we release a particular individual from his punishment, we really want that individual to have a strong, personal feeling that the system really does work, and that committing another crime is a really bad idea.
When we've punished a criminal, we as a society really hope that, rather than just being scared into not committing any more crimes, he'll actually turn into a good person, a person we can trust because he no longer wants to commit crime. When we mentally hold this idea up against the stereotype of the hardened criminal, it seems idealistic to the point of silliness, but I think there's a way that it makes some sense. Although I don't believe that there's a simple black and white division between good and evil, I do think there's a bimodal distribution along a continuum that ranges from perfectly conscientious wonderful people to sociopaths. Most people are at least moderately respectful of the feelings of others, and a few don't much care who they hurt on their way to what they want. People who run afoul of the law are much more likely to be in the second peak, but a sizable minority are basically decent people who may have fallen in with the wrong people, been exposed to the wrong ideas, or had some bad judgment. These people can be saved.
The fact that almost all of us (even though many are ashamed to admit it) take real pleasure in seeing those who have hurt us hurt is an ugly reality of human nature, but to pretend it isn't there is foolish. While it's ethically unwise to intentionally appeal to that desire with our punishment, if we fail to assuage it at all, our system of punishment is weakened because people will feel that it's not serving its purpose. We don't want people taking justice into their own hands because they don't think the system punished the guy who hurt them enough. The public must feel that justice is done by the system, even at the karmic cost of a few people getting glee out of it, because rehabilitation isn't possible if the public feels the need to keep punishing the criminal for the rest of his life once the system has done its job.