Troy,
I have found that for the larger bits, (over 1/2, even if it is metric)
that you can find the lower grade bits at a much lower price. it requires shoping for each individual bit though. they will not be right out front at the hardware stor and they usually will not progress to the very next size. say they woudl go from 19mm to 21mm. I can be sure of that becouse you mentioned you need a 20 mm. and that means in my experience that it will be that size they don't carry.
anything in a 64th that is larger than a half inch will have to be found someplace like Enco mentioned before. they start jumping by either a 16th or even an 8th of an ench in the commonly found bits. finding all the bits for the large pens was a real doozy.
I found many of them by digging in a drawer under the bit display at a local ace hardware.