Toddler self soothing by putting his hand down my shirt... still! Help!

So I nursed my son for a year, and starting a few months after I weaned him, he began to sooth himself when he was tired, upset, etc...by putting his hand down my shirt. It never was a big deal at all, but almost a year later, it is getting sort of old. I feel bad for even thinking this, but it is getting to a point that it has become an obsession of his.�
He wakes up about 5:30 am and I bring him to bed with me, and we cuddle, with his hands in my shirt, I am stuck on what to do. When I tell him it hurts mommy he will only listen if he isn't tired, but if he is tired, he gets so mad at me. Forget handing him anything else that I could substitute, a soft blanket, a doll. If I'm wearing a turtle neck he will just look at me, like, "hey! mom, what do you think you are doing?!?!" and pull my shirt out and somehow wiggle his hand down there...
i am really geeting stressed out with his habit...

can anyone help..