Hey Mark,
Welcome to the fun turning pepper mills... I use a chuck for both drilling and turning, because I have one, but as Ed said if you have a drill press capable of drilling the blanks, you can use jam chucks in the holes.... you could create a jam chuck for the headstock that is screwed to a faceplate and then another for the tail stock to hold the mill between centers... you could also use a screw chuck for the headstock to hold your jam chuck... only problem I see is the remounting of the jam chucks for future mills... if you be sure to mark and remount to the exact same screw locations on the face plate each time, would work. I still use a jam chuck at the head stock that remounts in my 4 jaw chuck and another at the tail stock.
Not knowing your budget, Grizzly has a pretty nice 4 jaw chuck for under $50 that works pretty nicely.... it will open to take a 2 1/2 inch tenon, and has a set of tommy bars to close the jaws. Only draw back I've found is the jaws are not interchangeable with other jaws like the PSI Barracuda II.