Just did my first one last weekend. SHARP cutters, light cuts. Hadn't been that long since I rotated the carbide, and I was thinking the antler was rock hard (it was, to be sure). Then, toward the end, I rotated the cutter--made life a LOT easier. What may be sharp for ebony may not be sharp enough for antler.
My novice observation--definitely not an expert!!
I find that cutting and drilling is the trickiest part due to odd shapes. Turning isn't so bad, but it does blunt the tools pretty quick. I use HSS spindle gouge, I use up the whole cutting edge and usually sharpen twice by the time I'm down to size. Depending on the size of the antler, it can be pretty porous. If its looking porous I usually fill with thin CA when I am a few thousands oversize then turn down to fit -- this just speeds up the finishing process for me.
The antler dust is a bit worse for the lungs compared to sawdust, so be sure to use a good DC or respirator (or both).
Last comment -- I would avoid using fresh antler. It holds moisture and can crack like wood during drying. I think typical drying time is 3 to 6 months. Many guys say they use fresh antler all the time without issues, others report cracking -- for me it just isn't worth the risk.
Selecting the right piece of antler is critical. Some people like tips that will make a nice hard and white surface, I feel that beam pieces make more interesting, and better selling, pens. My choice is always gnarly porous beam pieces. Fill them with CA 2-3 times as you turn them if necessary. I've made several hundreds of antler pens and those from beam stock, exposed bark and unusual coloration are always what sells best.
Drilling is a challenge but turning them round before drilling is not necessary. Mark the desired entrance and exit locations and drill a starter hole for each. I mount a drill bit in the headstock of my lathe, hold the antler piece with a large pliers, and use a live center in the tailstock quill to push the antler into the bit.
Some people claim they smell, if the antler is good and dry it won't smell as much. Definitely use a dust collector.
It looks like Boone and Crockett calls for 60 days drying time before measurements, so that's probably a good indication that most movement is done by that point. Given it isn't a problem that is often discussed in penturning, it's probably not a major concern, but it's also a risk if you don't allow for any drying. Cutting and turning round to 3/4" should speed up drying. You could then put on the dash of your car or a sunlit window, or bake in a toaster oven to speed up drying if you're in a hurry.