I have also been in that somewhat uncomfortable position of being caught by the husband. Now... they did indeed have an open marriage and he had asked to swap with me and the lady he thought was my girlfriend and lover... (which never happened.) But his wife had apparently never done the solo extra-marital affair before, (though he had.)
Now... if you know the joke about who has "savoir faire", let it be known, I was that third guy.