I agree, it's not an insult if neither party sees it as such. However, just because one person wants to say it doesn't mean the other person wants to hear it- different strokes and what not.