I have several decent fractional sets (bullet point, brad tip, split point, etc).
For the metric and letter sizes that I know I'm going to use frequently, I get good quality bits and store them in drill indexes I got from Enco.
You really have to balance cost against need. With shipping costs and the cost of gasoline what they are today, running to the store or ordering online for one or two drill bits doesn't necessarily make a lot of sense.
That's why I just ordered the HF (every size known to man) set so that I can continue working if I suddenly need a new size. I can then make a list over time and put together a larger order to minimize travel or shipping costs.
Just my 50th of a dollar.