I start off with bushings, then switch to the tube alone when nearly to final size and shape, then use calipers to get final dimensions. I need more pressure when first rounding the blank, but after that less is needed. I also sand and apply the finish without the bushings, which prevents bushing dust contaminating the blank, as well as doing away with any problems with the blank getting glued to the bushings as the finish is applied. This way the bushings will last indefinitely too.
I am very careful to use minimal pressure from the live center when going without bushings, and it requires surprisingly little by that point. I haven't had any problem with deformity or fit difficulties, but you might if you're not careful.