I do settle for .001 on cast resin and on wood but I shoot for .000 and generally get it it. If I had a machining lathe (metal), I most certainly would dial in finer settings in many situations. Just to be clear, I over turn my blanks - down to .003 or .004 below the size and build them back up with CA. At this point it isn't about how many times I add CA on with paper towel on to the pen, It is about how much thickness of CA that I have. It is difficult to build up perfect matching CA levels by doing x number of CA wipe ons. It isn't about the x number of CA layers, it is about the CA thickness - to bring it to the finished level that matches pen components after sanding and buffing. (BTW, I don't used PT that much as the PT drinks 10 times more CA than is added onto the blank.
Feel:
There were several posts on the subject some time ago on what the hand can feel. Seems like it was .0002 or .0004 or something like that. It has been ages since I last saw or commented on such a post.
A hint that someone passed on a long time ago: When you get a troublesome blank and work and work on it, taking finish off, adding it back on: If the finished blank is .002 or .003 larger than the finial, CB or nib, slightly round over the ends with MM 6000 and re-polish. This works well with pens that have curves as well as straight, and especially ones with non smooth (engraved/designed) components.
Getting back to your original: I don't settle .003 differences. I will settle for .002 on occasion, such as for kids (young grand kids that are going to lose them) but that is not for a pen that I would sell.
I think since you are looking at perfect fits, this is the kind of attitude that will greatly enhance your pen turning skills and output. Some people will not know the difference, but some people will fill your pen and KNOW that you make a superb pen. I used one of my pens in a wedding I performed; later two separate guests that I did not know came and asked to see the pen. Both commented on the design, and both commented on the "fit and finish" and how smoothly it transitioned from the blanks to the components. They did not make pens but they were pen aficionados.