I use both methods. I prefer using a pen mill for plastics, but sanding for wood (especially burls and cross-grained wood).
I speculate that most frustrations with pen mills happen because the cutters get dull, people feel the need to apply more pressure, the cutters grab, and the material splits or cracks or chips.
Sharpen the cutters; it's easy if you
follow this tutorial.
Sanding with the lathe works great, but I find having to switch between the drill bits (I have a 115 piece set) in the chuck annoying when I'm making a run of pens with different barrel tube diameters. I purchased a
barrel trimming sleeve set, which makes changing between pen tube diameters easy. This isn't an endorsement of one brand over another.
I wish the pen mill shaft came with a quick connect so that I could use one of my smaller drills. Sure, I could use an adapter, but that adds one more source of slop that I don't want to introduce.