I'm not sure I understand the problem. Is he with these four other subs now? Because if it's in the past, what difference does it make to your current relationship?

Have you discussed being exclusive with him previously?