While technically true, there could come a time when he would have to justify that decision. That's why they have the impeachment process. The problem with that is that party politics tends to override intelligence. They would rather keep their bad president, just because he belongs to their party, than put him on trial for his indiscretions and risk losing the presidency to the other party.
The biggest problem with any democracy is not necessarily the number of people involved, but the number of intelligent people involved. I don't think I would want to live in a true democracy where the majority of people are voting with their gut rather than their brain.