Am I Overreacting?

Well I think you are over reacting and over thinking about that guy. Even a lady can say that statement as that is how some things are.

I also have somewhat similar behaviour like yours. In our place we have this sharing rickshaws. Co passengers sometimes get down in between and I am the only one left till the final stop. Majority of the times I have observed that the drivers start chatting when all others have got down. Till then he would be silent. This is not a particular driver but majority of them start talking. They will say anything like roads not good, climate not good, or they may ask time or say some random things here n there. I dont like to chit chat with them in the night and I always keep quiet. I dont feel odd if old guys talk but anyone in 30s 40s start talking I just get irritated. This is especially during the night time say after 9 / 10pm. On the other hand I have a friend who can sit just next to the driver(cab) and carry on conversations as with an old friend. Problem is with our own basic nature I feel. The way we have been raised, not speaking to opposite gender frankly in the childhood all lead to this kind of behaviour.

Also society has changed a lot, dressing sense has changed. So much of skin show is considered modern and covered up people are outdated. If I were in ur place I would ask him his wife's pictures and see if she is hugging anyone in her office.