This is true if you use the table of the sander upon which to directly rest the pen blank. But, a perfectly drilled hole parallel etc to one side of the blank is not necessary. The only requirement for a blank to be square is the hole in the blank must be parallel to the table and perpendicular to the disk while squaring, assuming the table is perpendicular to the disk, level, and the groove is parallel to the disk. Of course, a jig is required but not spot on dead accurate drilling, if the jig is used for squaring. The best jig I've used is this one:
http://content.penturners.org/articles/2005/blanksquaring.pdf and I've made several and purchased two or three. This is the only one I now have and I use it to square all of my pen blanks.
Do a good turn daily!
Don