5 Speed Stick Shift Installation?

[Pooparu]

I recently did a 5 speed swap into my 1982 hatch and I'm almost finised with the installation but I can't seem to get the ball part of the stick shift to stay in place. I tryed to hammer it down into the plastic part sitting at the bottom of the rubber boot. It was so long ago when I pulled everything apart I can't remember how the heck it came out:-\.