Hi Dan: I use both. I found that when using a drill press, I tend to get a bit restless and really had to learn patience when drilling. Wood blanks are more forgiving and I find the increased speed of drilling them on the drill press makes it my preferred choice for that type of blank. Acrylics need to be kept cool, and I found that I do not have the urge to rush through the process when drilling on a lathe. I don't know if that's just me, or if others have experienced this.
I got a drill press before lathe drilling became popular. If I was just starting today, I am not sure if I would purchase a drill press, although I much prefer barrel trimming on a drill press over other setups.
Finally, unless space is critical, you can't have enough tools to do the things you want to do. I would never sell my drill press, as it comes in handy for so many other projects. In the same way, a vice takes up little space, but might be invaluable at some point.
I hope this helps.