Your instincts are good. Lashing out at others is not always the best response to someone who is taking advantage or abusing us. I just wrote out part of a long response giving advice on how you might handle similar situations, but I realized you weren't asking for advice. You have your own way of handling those situations and who is to say mine would be any better.
It sounds like your grandparents were emotionally abusive, that can be even worse than physical abuse. If they used words and emotions to hurt others, then no they were not nice people. They may have done some kind and generous things for you, but they definitely were not nice.