For some men, they can truly feel the difference. It feels like their penis is being 'choked'. They're not getting as much stimulation.
For some men, they see the condom as a literal barrier between them and the woman. They went to feel HER 100%...they want the complete skin-to-skin connection.
For others, the lack of condom adds an extra thrill...perhaps an element of danger (esp in the case of sex with a stranger), which some men might want.
And for others, it may stem from machismo. How dare a woman ask him to cover his penis in rubber, thereby telling him more or less that she doesn't want direct contact with his penis and/or fluids? When a woman tells a man to wear a condom, some macho or insecure men may see it as a sort of 'rejection' of him and/or his penis or precious semen.