This sounds very familiar to me....I am the exact same way. To me, it feels like I'm under scrutiny, and it scares the heck out of me to think that He will one of these days see what I see, and leave me. So I really don't think it's odd for you to feel that way, or we're both odd...that's possible too lol.
Have you talked to him about it? Tell him how it makes you feel, and see if he can offer you any insight into it since he is probably the one that knows you best. That's one thing that I'm gradually learning, that I have nothing to hide from my Sir. It's intimidating to talk to him, and admit the things that scare me or worry me, but afterwards I feel much better for having let Him know.
Anyway, I think I'm rambling...so I'm going to shut up now...