You could use this as an excuse to get a new chuck and spindle adapter. I just swapped out my 25 year old 5/8" chuck, with a 1/2" ball bearing chuck and spindle. I thought I was having a bit of a runout issue. In reality, my problem was a 1/8" center drill I recently purchased was more than 1/8", I measured the chuck runout and blamed it on that. I do have less runout with the new chuck now, and I appreciate how smooth it spins.
I would like to comment on tapers being self holding. Never had an issue with my drill press, but I did recently add a collet chuck to my toys (er, tools), for the lathe, and I could not keep that thing in place. Luckily it was threaded (not tabbed) so I could add a draw bar.