This is a problem that I have given some thought to as the table on my drill press has the same problem. I never went so far as to measure the actual amount it was out as it has never given me a real problem. (I use a machinist's vice to hold my blanks and only have to tip the blank a little in the vice to have it line up perfectly parallel to the drill bit.)
I have looked pretty closely at my DP and have never come up with a way to shim the head or the table collar that I thought would be an effective and permanent solution to the problem. It's true, as others have suggested, that you could mount your vice on a plywood base and shim it to be perpendicular to the drill bit; but this is only a half-a**** solution to the problem and the error would still exist for any other type of work that you were doing on your drill press.
Were it me, I would get a piece of 3/4" or 1" plywood and cut it to the size of the DP table or larger and then bolt that piece of wood to the DP table and shim it in the process so it was perpendicular to the drill bit. That way you would have a surface that was perp. to the drilling axis regardless of what you were drilling.
That being said, it may be that you really don't have a problem at all. Consider that you have an error of 0.045" in 5 ". If you look at the kits that are commonly available, about 95+ per cent of then have barrel sections that are about 2" long. The error you quoted above translates to 0.018" in 2" or just slightly less than 1/2 mm.....a negligible amount for most pen crafters. Only place I can see where that might be a problem is drilling blanks for Celtic knots or other detailed segmented blanks. I've never done a Celtic knot so I don't know exactly how much of an error is allowable before the effect is noticeable in the final product.