In many cases I think it is about insecurity, yes the dominant cannot just handle it.
I started out switch myself and were quite appalled by some things I heard. Over time I happened to put switch to the side, and it had nothing to do with what other people thought. Yet about that time there were a guy who were a major member of a BDSM organization who told me that "switch men become the best Masters since they understand what goes on in the head of the submissive" and that is something that I took with me.

Now that some remain switch is rather something I envy since they have the opportunity for twice as much good times than me. And I am serious when saying that I think so.