The thing to check - IMHO - is the spindle throw. Most drill presses will work fine, but the killer is the throw. Make sure that it is at least 3" and even better - 4". I made the mistake of buying an inexpensive drill press only to find - and be very frustrated with the fact that it has a 2" throw. That means that I have to stop the drill and move the blank up on the drill to go all the way through it. BTW - it's an 8" drill press.
Next purchase for me is one with a deeper throw.
Other than the spindle throw - I can't think of anything else... except for stability and a good range of speed.
I agree with Alan, and I had to replace my old drill press when I started turning pens for the same reason. Get one with at least a 4" throw and you'll be happy.
I use an 8" press and I don't find it a big deal to drill about 1" into the blank then raise the bit and the blank up and place a piece of 3/4" scrap underneath it to continue on .
A lot could depend on your finances , a drill press with a 4" throw is quite a bit more money than one with a 2" throw.
Just my $0.02 worth .
If you can afford it and are willing to make the investment, the longer throw is great, but I agree with Doug. I got a drill press from HF for $40 on special at the store. It only has a 2" throw, which is a pain, but considering that the next cheapest press I saw with a longer throw was four to six times that price, I'll stick with my HF for now.
I also have the 12" Ryobi. For the money, it is a great press. The Quill stroke is 2 & 3/8" , but seems to be just fine for drilling blanks. This press along with a centering jig works real well. I usually dont have to raise the blank part way through drilling.