Because a lot of people read Gor - even vanillas. I was aware of John Norman long before I was aware of BDSM because they are in the library on the same shelf as the conan books which I read at a very young age (and also have a bit of BDSM in them).

Though if the book is next to the bed and not, for example, on a shelf in the living room then it could indicate that they used it as a sex aid. Ok, the book that I am currently reading often ends up lying next to the bed but the only books that stay on the bedside cabinet all the time are the ones with erotic fiction in them...