"Respect" isn't unconditional and there are some acts which strike a strong, negative chord with people.
If a child-molester or rapist came here and started describing his exploits, would you "respect" him or speak out against him? No, I am not implying a moral equivalency between these and infidelity, but they are all things that some people consider to be wrong. Good people, when faced with wrong, speak out against it.