The big issue (for me) isn't so much that the hole starts off off-center, but rather that the blank is square in the vise. In one of my previous set-ups, I could (unintentionally) misalign the blank in the vise, and on more than one occasion the bit would drill through the blank at an angle. When it came out, the bit would have wandered enough that I could no longer use the blank (wasn't enough wood left near the hole). Part of that was a poorly squared table, but part is also just a bad vise design. Like several of the others, I now use Paul Huffman's vise. I align it when I first put it on my table (simply clamp a Harbor Freight transfer punch in the vise, then grab the punch using the drill press's chuck, then align the table to provide appropriate support), and then I don't look back. It works great regardless of the diameter/width of the blank.