It sounds like a problem a friend of mine has, and she has a diagnosis called 'social phobia'. What you say sounds a lot like what she describes.

There is treatement to make it easier, for some to make it go away completely.

I gather this is not as unusual as you might think, it just isn't talked about much.