Sorry, I should have said drift PIN punch.
I do not have any true drift punches. I find them useless, and yes, they would become stuck too.
All of my drift punches are technically drift pin punches. No taper, at all.
Ya, the one's that I used were not tapered at all and still got stick. I'll get it out though.