My first Worldcon was Chicago in '82. After a hiatus, I went to New Orleans in '88, Chicago in '91, Winnipeg in '94, San Antonio in '97, Chicago in 2000, and Toronto in 2003. The system worked well for me: every three years there was a Worldcon I could go to. (If I can't drive to it, I can't go. The expense aside, it is not possible to take the stuff that I need for a con with me on a plane.) And even without spending money on airfare, Worldcons are too expensive for me to go every year. Unfortunately, the tiny clique that makes the rules decided to change them, eliminating the zone system that guaranteed there would be a Worldcon somewhere generally close to me every three years, and changing the time interval from three years to two, so that I was no longer allowed to enjoy the benefits of being a recurring attendee while only going every three years.
The way those changes (like all business meeting business) happen is the root of the problem with Worldcon's declining relevance to the rest of the world. To have a vote and a voice, you must be so dedicated to the process that you'll go to all the trouble and expense of going to the Worldcon, and then instead of going to the actual con, devote a large chunk of time at the con to a meeting. A nice catch-22 there; you have to think Worldcon is fun enough to devote what is for most people a significant chunk of the yearly travel budget and vacation time to going to the con, and then be willing to skip the fun in favor of the meeting. All that's left are a few strange people who actually think meetings are fun, and the true SMOFs in the worst sense of the word, so dedicated to maintaining their own little empire that going to the business meeting makes sense. And while I've never been to a business meeting myself, even when I knew about agenda items that were somewhat important to me, I think it's safe to assume that the people who have been devoting their Worldcon to this pursuit for years aren't going to be very welcoming to newcomers who have their own new ideas, because that's how humans work.
For site selection specifically, the problem is that the system is structured for the benefit of a small core of people who go to Worldcon (nearly) every year. A supporting membership in the current con plus the voting fee is quite enough of an expense to deter someone who's just hoping the bid for the con they would go to beats the bid for the one they wouldn't. It's only a good deal if you get on the train and stay on; since you were already at this year's convention you have the membership to vote, and since you'll be going to the next one whoever wins, the voting fee isn't a fee, it's part of getting the membership at the lowest possible price. Unfortunately, people who go to Worldcon every year don't actually vote for the con. They've been to enough Worldcons that the con itself isn't really that important, so they vote for the more interesting city. They know they and their Worldcon-every-year friends can have a good time being tourists if the con sucks.
Worldcon is mostly irrelevant to all of the world that doesn't go to Worldcon, and getting more so every year. The only way to change that would be to make it so that the people to whom it is less relevant (but care enough to have an opinion) are more able to have a say in making it more relevant. Replace the on-site business meeting with an on-line process, so that geography doesn't select who can be involved and people can be fully involved in the decision process without giving up the con, which was supposed to be the point. Lower the voting fee to $5, drop the membership requirement, and remove the automatic supporting membership in the winning con so that people who actually care about the con overwhelm the people who want their annual trip to be to a more exotic city.