No, I'm not. But there are limits to free speech, too. You can't shout "FIRE" in a crowded room, causing a panic. You can't incite others to riot. And you can't go bellowing through a bullhorn at 3 in the morning.
But religious leaders claim to speak with the authority of God, and for the most part their followers accept that authority. And when those leaders use that authority to restrict others' freedoms they are abusing that authority. True, some will, and have, turned away and left his church. But there will be those who will say, "Well, he speaks for God, so we have to do what God wants." Yes, we can say that that's naive. (I say it's stupid, but I'm a nasty atheist, after all.) But I think we all know that there are people who are like that. God's law is foremost, and the Vicar speaks for God.
And let's face it, we are seeing more and more cases of those same religious leaders, of all faiths, who don't live by their own rules. If they won't, why should anyone else?