I've got a similar one. But I put a lot of work on distinguishing features. Things that others easily notice about them. Especially about their looks. This is different for different people depending on perspective, if it shifts in a story.
If we feel threatened, describing the eyes and how the person looks at others, and what the hands are doing I think is important. Ie, "The man with whitening knuckles said..."; while if it's just a character in the background then they're maybe distinguished by some particular piece of clothing or hat. "A man with grimy trucker hat leaned against the bar....".
For me that tells me a lot about what the viewer is feeling and needs to be logged. If the guy who bunches his fists when angry or trucker hat guy comes back, we need to keep track of it and make sure they do the same thing later.