Communication is always the key. These worries and fears you have about him pushing your limits should be expressed to him, and he will explain what and how he thinks about it, if he is not willing to do that than if I were you I would move on.