Have you tried Madam? Seems like maybe a good middle ground...?
Anyway, my master has me refer to him as Sir when we are alone. Though he makes it rather challenging, because sometimes when we're just lounging about, he prefers I go back to his first name. But then the atmosphere will change and suddenly I'll know that I should be addressing him by Sir. When in public, it's usually his first name, but occasionally he will tell me to address him formally in public.