I guess we're annoyed with different aspects of religion. I have a problem with adults clinging to childish notions of how the world functions.
People "exploiting" peoples faiths is just a natural development. Is NASA "exploiting" peoples belief in science that we can go to different worlds? Are the Greens "exploiting" that we believe in the global warming, (which finally last week got some serious proof backing it)?
We are social animals. We like doing things together and especially with people we have something in common with. If people believe in a supernatural god then they'll off-course like to hang out with others who share the idea. Off-course strong people are needed to hold them together. We can call them sect-leaders, popes, chairmen, doctors, proffessors or presidents. They all do the same thing but with different connotations but given power for different reasons.