A good rule of thumb to follow: if you have to ask if there's a problem, there usually is.

Granted, it's hard to see the dynamic of a four to five year relationship in a single post, but the important thing here to me is that you seem to have a gut feeling about this lad and his intentions, so my advice is to follow it. It can be confusing when you feel that the other person loves you but, as I'm sure you already know and as lily mentioned earlier, love doesn't always drive people to do what's best for the person they happen to love. I had a similar situation with an ex, and I ended up on the receiving end of three broken ribs and a few other various odd injuries. Long story short, follow your instincts and proceed carefully.

Best of luck.