It's an affair. Your husband approves. Go for it and don't worry so much.

(IMO & remembering I'm a bit of a dawg.)