Babies are thinking way before they can make words and express thoughts that way. They communicate through eye movements and gestures. I taught both my kids sign language (baby signs) for simple things before they were able to talk. For that matter, animals are thinking although we don't always know what they are thinking.

My thought was the reason none of us remember being babies and thinking baby thoughts is that there is no language ability to hang memory hooks on. That is, as adults, part of remembering things is the ability to rehearse them in your mind using language. And since babies don't have that ability yet they don't remember things in a clear manner, just possibly as sensation and emotion.