Welcome to the forum.
I also agree with those above, you need to talk to him about your concerns and if he does not seem willing to listen to them, then you have to seriously consider what to do. No one but you can make that final choice, but as I have been told before, if it is not fun anymore, than maybe it is time to make some changes. Good luck