Now that you explained it, there is something to watch for, and in general it works better to take the whole finish off with a sharp slightly rounded end square scraper (like radiused square carbide insert), and then start over. Or you can lay a skew on its side and use that to scrape the finish down to the wood. Turn fast and barely touch taking of incremental amounts down to the wood.
The reason is that once a section or spot wears down to the wood (or is sanded through) OR in your case resizing the fit, the new spot of wood covered with the same finish will appear a shade different, usually lighter. Some people don't mind, and for some it is hard to see, but for those with an eagle eye, the light spots/shade jumps out as opposed to the section where the finish was not removed. On some woods like cherry, - about 6 months down the road the cherry will turn a beautiful shade darker and new sanded wood will look very different.
Occasionally someone will post a picture and ask what caused the light spot. It usually is because they finished the pen with CA or other and then sanded or turned through the area and refinished the spot as well as the whole pen. The newly finished spot shows through.