I suggest talking to him. You have invested a lot in the relationship so you would probably regret just walking away. Give him a chance to explain and talk with him. Things might work out better than what you feel right now.