If the OP's pen is like the flat-tops I've done, the center band fits the outside of the brass tube instead of inside (which is why the screwcap threads, resident to the center band, are now covered by the tube). So can't be punched out.
I haven't had to remove one, but after removing the clip, would consider trying to grip the outside of the band with something like a neoprene strap (non-marring friction surface) held by pliers, and similar for the cap portion, see if you could twist it off, being careful of the forces employed so not to crush or distort either.