(unless said g/f is aware & ok with it, then he's not a scumbag, just in a non-exclusive relationship)

But I would recommend getting away from him completely, because it is making you unhappy and uncomfortable.