Collet chuck is good and most accurate AFTER you have turned your blanks round.
To get them round you need to mount some kind of drive center in your headstock, and you need a live center in your tailstock.
As I see it, the only problem with using a Collet chuck and collets is that you are restricted in the size (diameter) of what you turn because the collets only come in fixed sizes. . Plus you need the appropriate collet chuck to hold them.
On the other hand, an adjustable lathe headstock chuck, like a Nova G3, can handle any size (up to 2" diam, or so). And with the correct jaws can grip shapes other than round.
If you don't buy the pin jaws for the G3, you still need a drive center for the process of turning non-round blanks to get them round.
And even after they are round, you can use the drive center when turning them down to the correct size/shape.
The drive center can be just a straight-sided 60 degree taper that fits in your chuck jaws. The 60 degree taper is not really designed for this kind of job but it does work if you apply quite a bit of pressure from the tailstock and spin at a fair speed.
Instead of the plain steel 60 degree taper, I use a 60 degree diamond coated cone (which is really intended for sharpening mortise chisels). The diamond cone is more successful as a drive center because of the friction between the diamond coating and your blank material.
To do the drilling on your lathe you will need a Jacobs chuck with arbor to fit your tailstock. The necessary arbor will probably be an Morse Taper 2 (MT2) as stated already above. You hold the blank, while drilling, in the headstock chuck (which could be a collet chuck for best accuracy).
Lots of different things to think about, really.