I must have grown up with a different experience on drilling holes. I am not a "machinist" but know how to drill holes even in metal without it skating.
1. IF the chuck is in the tail stock, LOCK the tail stock down tight. Put a drill bit in it deep enough so that there is a good bite on the bit, and tighten good. With thumb and forefinger only, hold the bit fairly tight and see if you can wiggle it. If so, there is a problem somewhere mechanical. Look for it.
2. Add the blank to the head stock chuck. Tighten it. With thumb and forefinger, can you wiggle it? If so, then check to see of the blank is square. Work on getting the blank into the chuck and with enough pressure / tightness so that it does not wiggle with moderate pressure.
3. Pull the tailstock up and see if the tip of the bit lines up with the center of end of the blank.
4. Caveat: This does not work with some 300+ lb weight lifters. :biggrin: (Of course, - they can wiggle the whole lathe with their thumb and forefinger.)
5. IN most cases for me, I start drilling with high speed and very light touch feed - metal or wood and find that works on the (my Grizzly & Rikon) lathes too. But I will say that I have had years of experience.
I don't have the most expensive lathes, but it is not much different than a drill press. Learning to tighten, insert the bit deep enough into the chuck, check everything mechanical, lock down to keep things from moving BEFORE drilling is necessary. I have all kinds of bits and don't have a problem with holes getting started (on the lathe) where they need to be - because I tighten and check each part before drilling.
Don't assume that because the bit is in the chuck and is snug and that the tail stock is locked - that they are tight.