I would say you need three things at this point, communication, communication and communication. Talk about what works for you, what does not, and what you want and expect out of the relationship and have him do the same. Talk to him about what limits you need. Talk about how to have your safe word fill the role that you need it to, ask yourself why you take it back and how that makes you feel. If you are upset after maybe you two need to come to the agreement that it cannot be taken back.