Lathe "on" or "off?": Both. Sand with the lathe on slow to get ridges or spots out, to get the finish consistent. Then turn lathe off and sand lengthwise with fine MM or sandpaper.
It sounds like you are sanding either too much or with too low of a grit of sandpaper.
Once the finish stage starts, sanding should be at 400+. I usually use MicroMesh, but on occasion I use standard sandpaper. When I do, I use 600 (automotive type) to smooth any spots out and move up to 800 or 1000 grit before changing over to MM 3600 - 12000.
Also, if you are sanding it off, you may not be getting enough CA on the blank. I think most people average 8 to 12 coats, but some do it with less.
Just a note here - some woodworkers who start turning pens hang on to an old idea - "too fine of sanding will not let the finish stick properly, therefore nothing higher than 400". That is not true, it is OK to go to a higher grit and finish sticks just fine, - CA, lacquer, poly, tung oil etc.