I find many of the recommended drill bits are not the best fit. The fit should be a close sliding slip fit. There should be little wobble or movement when the tube is in the blank.
Gorilla glue original (Polyurethane) does expand during the cure process but the curing process can make the tubes slide out of the blank. You need to check the blank(s) about 20 min and 1 hr after gluing.
For CA glue use thick for gluing in tubes.
Lastly, if the brass tube is smooth, scuff the tube with some sandpaper to give a better bond to the blank.
Drilling isn't a straightforward as it might seem! If your holes are too big for the tube, it might be your drilling setup. The hole might be bigger than the drill bit. The drill could slightly bent, or you might be inadvertently enlarging the hole as you repeatedly reinsert the bit into the blank. And you
should be withdrawing and repeatedly inserting the drill bit to let it cool off during the drilling process as well as to clear the flutes. An overheated drill bit is especially bad on acrylics which can melt inside. I reinsert my drill bit before I turn the drill press on which ensures that I am following the original hole.
As for undersizing the drill bit, yes, but remember if you are reverse painting, you'll need to leave room for the thickness of the paint (which may be pretty close to the amount the hole is oversized)
Many here drill on the lathe-it seems to help, but I worry about the amount of play in my tailstock causing holes bigger than the drill bit.
Scuffing the tube is controversial-some here say it is essential, others never do it. I have no opinion. Try it both ways and see what works for you