I use the barrel trimmer before turning, then after turning, I use the barrel trimmer with the head turned upside down on the 7mm shaft with self adhesive sandpaper on the back of the trimmer head to touch up and make sure it is nice and square after turning. This also helps take off any excessive CA or other finish that may be on the ends (I only do TBC for finishing, so the ends are not protected by bushings..)or take away the slight rounding that happens on the edges when you sand without bushings. Using the 7mm shaft and holding the larger diameter tube against the side works nicely to not use just a ring of the sandpaper. I roll the blank against the shaft as it sands also to help keep things even. The 7mm shaft is also just about the size of a standard hole punch so it's pretty easy to cut squares and punch the hole to get a nice patch of sandpaper.
I do have issues with some of the adhesive staying on the trimmer head when I have to change paper, but I just scrape it off, and if it gives me a hard time, a bit of acetone usually takes it off. You've found the drawback to any sort of soft base to the sanding surface, it needs to be solid to get a good flat and square edge.