Tough to form an opinion in the abstract.
I can see myself pulling back from a new relationship if I thought there were unresolved issues that I wasn't qualified to deal with in a BDSM context. For instance, if I thought someone needed professional counseling, I'd try to guide them toward that rather than forming a deeper relationship.
In a committed relationship, I might curtail certain activities, but wouldn't try to end things, because there's a commitment to help someone through those issues.
The "... how he would handle hearing about them"-part might be meaningful -- it's possible that your experiences and feelings act as a trigger for something he experienced or felt that was negative for him. If that's the case, then it's conceivable that he could be "harmed". An analogy might be an alcoholic not wanting to be around another alcoholic because their talking about wanting a drink would set of his own cravings -- something like that.