OK, Jack. That is a good start. You did not mention if this is a problem that has just started or noticed or is one that has been with you for a while? What size holes were you drilling when you had the problem? Are we talking a 7mm drill bit or a bigger one like 1/2�?
Anyway, just because the table is perpendicular to the drill bit, does not mean that the blank is properly oriented relative to the drill bit. There may be a problem with the vise not being parallel and perpendicular and it is also possible that the blank is not lined up exactly right in the vise. Also a possibility is some play on the vise that allows the blank to move when the blank is drilled.
First off let's find out what kind of vise you have. Is it one of those pen vises made specifically for drilling pen blanks or is it a more general purpose vise like a machinist’s vise? Next let me say that being off by a 16th is not that much of a problem. It could be better; but it could also be a lot worse. I did a little math and being off by that amount is the equivalent of an error of only 1.5 degrees.
Let's assume for the moment that it is a pen vise. When you secure a blank in the vise, is it firmly held in place or does it move around a little due to slack in the vise itself? If there is slack in the vise are there any nuts, bolts or screws that can be tightened up to reduce or eliminate the slack? Another possibility if it is the pen vise is that the grove in the jaws of the vise is not cut exactly vertical and the blank is being held at a slight angle to the base of the vise.
This may be a little tricky to do; but, mount a blank in your vise and then move the vise on the table so a vertical face of the blank rests against the side of the drill bit that has been extended down onto drilling position. Do this for all four faces. Just like when you were checking the table with the machinists square, the faces of the blank should touch the edge of the drill bit and be uniform along the entire contact…should be no gap at the top or bottom. The idea here being to see if the vise is holding the blank in such a position that the drill bit is parallel to the faces of the blank. When moving the vise around. Try not to rotate it; but keep it in the same relative orientation…ie handle always to the left to the right or to the front… whatever you choose. If there are some gaps then the blank is not being held properly. Depending upon where the gaps are, you can put some shims on the bottom of the base of the vise to correct the misalignment. (You could also return the vise, but these vises have a rep for not being exactly perfect and the replacement may very well have the same problem.)
Another way to deal with a vise misalignment would be to make up some dummy blanks out of scrap wood(assuming you have the tools to do so) and put some reference marks on the test blanks. Then drill 2 or 3 of them and note the direction of the drilling error at the bottom of the blank. Add thin shims to the bottom of the vise in the direction that the drill is trying to wander and that will push the hole back toward the center of the blank. You may have to do this several times and play with the thickness of the shims a little before you get the hole back in the center of the blank; but it is really much simpler to do than it is to read about it. I hope all of this is not too confusing. If so, holler, and I will try to clarify.