I have a full set of imperial bits - fractional, lettered and numbered - that came in a metal bit file. I reserve those bits exclusively for use in wood. The fact that each bit has a dedicated place means that it's easy to find specific bits for an application.
I also have a random bunch of standard jobber-length twist bits that I have purchased over the years or inherited from my dad or FIL that I keep in two plastic boxes that originally came from the deli and contained sliced, baked ham. One box holds bits 1/4" and smaller, and the other holds larger bits. This setup is a bit more difficult to use, but it works for me. These are the bits that I reserve for use when drilling metal.
Then I have another of the plastic boxes that I use for specialty bits - brad point, 'aircraft' length, etc. I have fewer of them, so finding the bit that I need is not a problem.
And I have another plastic box (we're big on ham sandwiches in our house) that contains spade bits. Don't use them often, but sometimes they are convenient to have.
I bought a set of forstner bits at the big box store - 1/4" through 1 1/2" - that live in a wooden box. But I have a few additional odd-size forstners that reside in a rack on the wall. Likewise, I have a set of large diameter Silver and Deming bits (1/2" - 1" with reduced diameter shanks) that also came in a wooden box.