I'm sorry to say this, and I hate to make such a snap and harsh decision on someone I've never met, but a reply like that makes him sound rather callous.
On the other hand - maybe he felt like there was already some sort of trust level already there what with the mutual friend...
But in my opinion, if you don't want to answer something on a first date, vanilla or otherwise, then you damnwell shouldn't. It is entirely up to *you* and only you what and who you answer to, and a first date is *never* the appropriate time to be 'breaking barriers'.