Well, as I see it, you do not answer to anyone for either your feelings or your thoughts. What you do answer for, is your actions.
Is your life as a sub hurting anyone? No? Is it making you and your other happy? Yes? Well, it is not a moral defect to make anyone happy.
The morality comes from either religion or politics or ideology. We all have our own version for our own reason, our own sense of right and wrong. Nobody has a monopoly!
Now, if being turned on by non-consensual violence makes you go out and violate other people, it would be wrong. But if you keep it between consensual partners, I see no wrong myself. Even fantasies about actual non-consensual acts hurt noone, as long as you keep it fantasies. There is a difference between fantasy and reality, and one of the problems is that there are those who simply do not understand the difference.
I know there are people out there who think it against their political convictions, and that is bad for them.
Some are feminists, some are fundamentalists. You can recognize the fundamentalists by their tendency not to want to listen to other people, not to want to go out there and find out what it is actually all about on a less theoretical basis, and by their belief that even without doing these things they just know what is best for other people.
But it is the people doing it, who knows. Us. Oh, and the people who have actually studied this, and also found it harmless.